About App Microscope

App Microscope is a tool to display the Internet Safety Labs safety label (ISL Safety Label) for mobile applications. The information in the ISL Safety Label is designed to highlight safety risks found in the app when using it as it was designed to be used. ISL calls these inherent risks “programmatic harms”.

The current version of App Microscope contains 1722 apps studied in the ISL 2022 K-12 EdTech safety benchmark. Over time, App Microscope will be able to generate safety labels for all apps.

App Microscope is designed for app developers, school technology decision-makers, journalists, privacy advocates and regulatory enforcers, but can be used by everyone to better understand the hidden behaviors of mobile apps.

App Microscope is a free public service, funded in part by a generous grant from the Internet Society Foundation. ISL is a US-based 501(c)(3) non-profit organization.

About the ISL Safety Label

The purpose of the ISL Safety Label is to help people understand what’s happening “under the hood” of mobile apps.

The safety label is organized into Risk Modules that identify safety risks to people using the app as it was designed to be used. The label is designed to add on harm/risk modules over time. Currently, the label includes only one risk module: Privacy Risks.

Other envisioned risk modules include:

• Harmful UX Pattern Risks and
• Automated Decision-making risks

Note that the Apple and Android logos are used only to communicate the version of the app and do not constitute an endorsement or affiliation with Internet Safety Labs. Each mark is the property of its respective owner.

ISL Safety Label Structure

The high-level structure of the ISL Safety Label looks like this:

ISL Safety Score & Triggers

The ISL Safety Score is a “stoplight” score with four possible values:

• Some Risk (green)
• High Risk (yellow)
• Very High Risk (red)
• Not Scored (blue)

Here are the detailed explanations of each score.

Some Risk:

This represents the “safest” of all safety scores. Note that “no risk” is not an option in the ISL scoring rubric as all apps entail some level of risk.

High Risk:

Apps that receive this rating meet at least one of the following high risk criteria:

Presence of high risk SDKs

(at least one ISL scored Very High Risk or High Risk SDK).

App’s use of Webview APIs

to launch web content within the app. The main problem with Webview APIs are that they launch the platform’s default browser, not the user’s preferred browser, including preferred privacy settings. See What is Webview for more details.

Presence of data aggregators, Google or Apple,

as determined from either the presence of SDKs or network traffic analysis.

Presence of one or more dangling domains in the app.

A dangling domain is domain that is available for purchase. See this blog post for more information.

Very High Risk:

This score represents the least safe apps and ISL recommends that these apps are not safe for students. Apps receive this score if they meet at least one of the following criteria:

Presence of advertising

(of any kind). The safety score doesn’t distinguish between contextual and behavioral advertising, since no matter what kind of advertising is present, there’s a very high likelihood that user data is being shared into advertising networks, though it has not yet been confirmed. The current adtech infrastructure is risky because there is no way for the public to inspect where the data goes or how it’s used.

Presence of one or more Data Broker’s SDKs

(per the California and Vermont Data Broker registries).

Presence of data aggregators as determined by presence of SDKs or from network traffic analysis: Adobe, Amazon, Facebook, or Twitter.

Presence of MaxPreps.

Refer to Spotlight Report #4 which examines the extremely risky behavior of MaxPreps, an advertising-supported school sports platform. MaxPreps is owned by CBS/Viacom, parent to Disney and used by hundreds of schools.

Suspicious permission behavior:

Safety tester observed inconsistencies related to app’s permission behavior, such as the app’s use of resources without asking for permission.

Note that this score was previously referred to as “Do Not Use” in the first two 2022 K-12 Edtech Safety Benchmark findings reports.

Not Scored:

Apps that were unscored for one of the following reasons:

• App required school login credentials in order to exercise even basic functionality.
• App was broken.
• App was a paid app.

Third Parties Receiving Data

This section provides information on the number and riskiness of SDKs in the app and the number of aggregator platforms receiving data as observed in network traffic.

Total # of SDKs:

This is the total number of SDKs found in the app. [Source: AppFigures]

Risky SDKs:

This is the number of High Risk and Very High Risk SDKs found in the app. [Source: SDKs from AppFigures; ISL SDK Risk Database (to be published in 2023)]

Data Broker SDKs:

This is the total number of SDKs found in the app that are owned by registered data brokers in either the California or the Vermont data broker registries. [Source: California and Vermont data broker registries.]

Aggregator Platforms:

This is the number of aggregator platforms, 0-6, with whom the app communicated during the testing session, based on collected network traffic. The aggregator platforms included are:

• Adobe
• Amazon
• Apple
• Facebook
• Google
• Twitter

[Source: From network traffic collection performed by ISL.]

App Category Average

The App Category Average column displays the average number of SDKs and aggregator platforms for the category of apps. Note that the category is the smallest sub-category as shown in the top part of the label, such as “EdTech -> Other -> Games”.

The “n=number” is the number of apps in that category/subcategory.

This information provides context as to whether the chosen app is behaving better or worse than the category with respect to third parties.

[Source: calculated from ISL app database.]

Risky Behaviors

This section indicates if risky behaviors were observed during manual app test. The following behaviors are key risks:

Ads:

If digital ads were observed in the app, this field will show “Yes”; if no digital ads were observed in the app, the field will show “No”. In this case, “no” is safer and “yes” is riskier. ISL regards digital ads risky due to the infrastructure of digital advertising and real-time bidding, in particular, which has the capability to share personal information to countless entities in the ad network. [Source: observed by ISL product safety testers.]

Behavioral Ads:

This field reports whether the tester observed behavioral ads in the app. Behavioral advertising refers to the capability to anonymously ‘follow’ consumers across websites, all over the Web. Behavioral ads are ads that clearly rely on information that has followed the user from another site. In other words, behavioral ads are using personal information to customize ads being displayed. This behavior is extremely risky. As above, a “yes” is riskier than a “no”. [Source: observed by ISL product safety testers.]

WebView:

This field indicates if the app uses Webview to display webpages within the app. [Source: observed by ISL product safety testers.]

• From pp 8-9 in Spotlight Report #4:

  What is WebView? All mobile apps with an “in-app browser” are using “Webview” code/APIs, and the rules and protections for these in-app browsers differ somewhat between Apple iOS and Android platforms. WebView is a low-level set of functions (APIs) provided by an operating system within a mobile device that allows native apps to present web pages, woven seamlessly into the application without having to open or close a separate browser. WebView allows native apps to selectively become a browser while keeping the user in the app. A 2011 paper on WebView attacks against Android devices explains this legacy development and data supply chain strategy: “WebView is an essential component in both Android and iOS platforms, enabling smartphone and tablet apps to embed a simple but powerful browser inside them. To achieve a better interaction between apps and their embedded “browsers”, WebView provides a number of APIs, allowing code in apps to invoke and be invoked by the JavaScript code within the web pages, intercept their events, and modify those events. Using these features, apps can become customized “browsers” for their intended web applications.” WebView in iOS relies on WebKit, an open-source browser “engine” first introduced 23 years ago in 1998, which later evolved into the Safari engine used by Apple around 2013. The WebView APIs are therefore limited by WebKit capabilities supported in the OS. iOS WebView is restricted and protected by iOS WebKit (built on Safari) limitations, and Android WebView is restricted and protected by a slightly similar concept with Chromium limitations. However, they both ultimately support in-app browsing as a seamless user experience without giving users full control over their own data sharing preferences within these in-app browsers.

App Category %

The numbers in this column display the percentage of apps in the named category that are positive for the behavior. For example, 60% shown in the Ads Present row means 60% of the apps in the category have ads present. The number of apps in the category (n=32, e.g.) is shown at the very top of the column.

[Source: Calculated from ISL app database.]

User Data

This section lists the categories of sensitive permissions the app asks for. ISL has created these categories for ease of understanding the overall risks related to the app’s use of user data accessible from the device. There are eight permission categories (listed in alphabetical order):

Crash Logs

include permissions that allow the app publisher to receive information when the app crashes. There is a risk of this information including personal details.

Files

include any permission that allows apps to list user data files or their contents, whether in the cloud or on device. This access is risky both because files and filenames can include personal information and because it can be used to fingerprint and reidentify a user even if they have reset other identifiers.

Join User Identifiers

includes any permission that directly assists advertising networks that wish to track users across apps or across device, such as with Apple’s ID for Advertising (IDFA).

Location

includes any permission that potentially allows apps to determine the user’s geographic location. Permissions such as wifi network names and bluetooth connections are included in this category because in many cases these names are distinctive and can be compared against databases to guess the location.

Phone Service

includes permissions that reveal who the user’s carrier is or whether they currently have service. This can serve as a proxy for location. It may also reveal financial wellbeing.

Physical Environment

includes permissions that reveal information about the user’s physical environment, such as granting access to the camera and microphone.

Social Information

includes permissions that reveal who the user associates with, as well as when or where they do so. This includes calendar and contacts.

User Behavior

permissions include anything that would be useful to advertising networks seeking to learn more about a user, such as their psychology or interests.

[Source: AppFigures, iOS App Store, Google Play Store; permission categories created by ISL.]

3rd Party Sharing Details

This section provides details on the 3rd parties likely to be receiving user data. This section has two main parts: (1) details on the SDKs and (2) details on the Aggregator Platforms observed in the network traffic.

Riskiness of 3rd Party SDKs – by SDK Risk Score:

This section shows the companies behind the SDKs included in the app, sorted by SDK risk score. Similar to the Safety Score, the SDK risk score is a “stoplight” score with Very High Risk (red) being the riskiest category, High Risk (yellow) the middle category, and Some Risk (green) being the safest category.

ISL is unable to share the names of the SDKs in the apps, so this section displays the names of the companies behind the SDKs, as well as the ISL designated category.

These are the current SDK categories:

• Advertising
• Analytics
• AR/VR
• Authentication
• Computer Vision
• Crash Reporting
• Customer Chat
• Debugging
• Location
• Map
• Marketing
• Messaging
• Storage
• Utility
• Voice

[Source: AppFigures, ISL SDK Risk Database]

Aggregator Platforms Receiving Data:

This section lists the aggregator platform companies observed in the app network traffic. The six platforms are: Adobe, Amazon, Apple, Facebook, Google, and Twitter.

[Source: From network traffic collection performed by ISL.]

User Data Details

This section contains detailed information regarding what user information the app accesses. This part of the safety label is currently comprised of a single section: Riskiness of App Permissions.

Riskiness of App Permissions:

This section lists all the permissions requested by the app. Permissions are grouped by Permission Category. [Source: ISL]

Permission Category:

ISL has created permission categories based on the kind of information accessed. Note that the same permission may be included in multiple categories. See above for the list of permission categories. [Source: ISL]

Permission:

This section lists all the permissions that the app either requests access to or simply accesses without permission. [Source: AppFigures, iOS App Store, Google Play Store]

iOS Apps:

In iOS apps, there are two types of permissions shown in the safety label: (1) permissions that start with “NS”, and (2) permissions that don’t start with NS. Permissions that start with NS actuallyaren'tpermissions but map to fields in the Apple Privacy Nutrition Label, which are voluntary disclosures made by the app publisher and are subject to various exceptions that allow the publisher to not disclose use of certain data as per the fine print in Apple's policies.

Android Apps:

The permissions shown for Android apps reflect the actual permissions requested by the app.

App Category %

This column shows the percentage of apps in the app category that use permissions in this permission category. [Source: Calculated from ISL app database.]

This information provides a rough approximation of app category behavioral norms for the permission category. When this number is low (15% or less), and there are one or more permissions shown for the app, it can be viewed as atypical behavior for the category. This may or may not be significant depending on the exact purpose of the app.

K12 Edtech Typology

The following Edtech classification schemas were considered in deriving our final Edtech typology:

• Valuates Reports': K-12 Education Technology Market Research Report, January 2022, K12 Education Technology Market Size, Share, Growth, Forecast 2021 - 2026 | Valuates Reports
• EdSurge Product Index, Homepage | EdSurge Product Index
• LMS Hero, "What Is Edtech and How Is It Shaping the Future of Learning", What Is Edtech and How Is It Shaping The Future Of Learning - LMS Hero
• "Understanding the Edtech Product Landscape [+Infographic]", Ashmeet Singh, April 19, 2018, https://medium.com/the-edtech-world/edtechlandscape-743716608675
• EdtechImpact.com Categories, https://edtechimpact.com/categories
• Learn Platform, "Edtech Top 40 Mid-Year Report 2021-2022", Edtech Top 40 Mid-Year Report - LearnPlatform
• G2 Edtech Categories, https://www.g2.com/categories/education

After reviewing all the references above, we chose to use the G2 Edtech Categories as the basis for classifying apps in our benchmark, but adding two new categories: Other, for educational-other apps, and NES for Non-Education Specific apps.

1. Include multimedia messaging options
2. Provide mass messaging and push notifications
3. Facilitate two-way parent-teacher messaging
4. Sync messages to multiple platforms, including email
5. Examples: PowerSchool Mobile, School Messenger

1. Provide tools for administrators and parents to communicate
2. Include a news feed of recent things happening at the school, for the benefit of both students and parents
3. Primary function is to serve as a communication platform
4. Not classified as something else (like SMS)
5. Examples: Nearpod, Minga

1. Be designed for use by instructors at K-12 schools or higher education institutions
2. Deliver interactive educational lessons
3. Include multimedia elements designed to increase student engagement
4. Personalize the learning experience for each student
5. Generate reports based on student performance data (Optional)
6. Examples: Edmodo, Quizizz

1. Provide a platform for educators to deliver online course content to students
2. Distribute assignments to students and allow instructors to grade student work
3. Administer digital assessments to students
4. Facilitate individualized feedback on student work, such as through written comments or grading rubrics
5. Generate performance dashboards for tracking student progress
6. Contain gradebook functionality or integrate with third-party gradebooks
7. Examples: Canvas Student, Google Classroom, Schoology

1. Include a database that can be used to store and manage information on different types of content assets (books, magazines, movies, music records, and more) in different formats (print, electronic, video, etc.)
2. Manage patron and member information including profiles, present and past loans, payments, and penalties
3. Allow users to find information from public sources like OPAC (Online Public Access Catalog) or WorldCat
4. Manage asset inventory and loans across multiple physical locations
5. Provide statistics on loans, inventory, late returns, or lost documents
6. Examples: Destiny Discover, hoopla, SORA, Libby

1. Either not an edtech application or does not fit any categories
2. Add subcategories of NES
   1. News
      1. Examples: NY Times, KQED
   2. References
      1. Examples: TED talks, Encyclopedia Britannica
   3. Productivity
      1. Examples: Outlook, Google Documents

1. Edtech/edtech-adjacent applications that do not fit criteria for other edtech categories (e.g. educational games, music lesson apps)
2. Add subcategory for Games.
   1. Examples: zoo-phonics, Teach Your Monster to Read
3. Add subcategory for Sports
   1. Examples: ScoreStream, SBLive Sports, NFHS Network

1. Be designed to manage school transportation programs
2. Create optimized bus routes and schedules
3. Assign students and drivers to bus routes
4. Examples: WheresTheBus, Z Pass+, Versatrans My Stop

1. Allows reporting of school specific safety information to school security personnel
2. Reporting is anonymous
3. Examples: WeTip, Vector Alert, P3 Tips

1. Allows users to use one login to access multiple applications ordatabases in one portal
2. Example: Clever, ClassLink LaunchPad

1. Provide tools to improve staff communication
2. Have features designed to improve efficiency
3. Include functionality designed to help manage school operations in areas such as facilities, IT management, program management, document management, attendance, food service and payment technologies, hall pass management.
4. Examples: Nutrislice, MySchoolBucks

1. Monitor relevant student data
2. Include a portal for parents to access information about their students
3. Offer reporting capabilities
4. Handle student admissions
5. Provides a module for school staff
6. Examples: OnCourse Connect, Skyward Mobile Access

1. Have features specifically for test preparation
2. Include various study methods
3. Be accessible for students and educators
4. Examples: Sporcle, ProProf Quizzes, Kahoot!

1. Contain live video streaming capability
2. Provide screen sharing
3. Contain an online whiteboard feature
4. Provide a comprehensive online classroom environment designed for use by educational institutions as well as individual teachers and tutors
5. Stream live rich media interactive presentations
6. Examples: Zoom, Microsoft Teams, Google Meet