About App Microscope

App Microscope is a tool to display the Internet Safety Labs safety label (ISL Safety Label) for mobile applications. The information in the ISL Safety Label is designed to highlight safety risks found in the app when using it as it was designed to be used. ISL calls these inherent risks “programmatic harms”.

Most of the labels in App Microscope are from the more than 1,722 apps studied in the ISL 2022 K-12 EdTech safety benchmark. New labels are being added all the time.

App Microscope is designed for app developers, technology decision-makers, journalists, privacy advocates and regulators, but can be used by everyone to better understand the hidden risks of mobile apps.

App Microscope is a free public service, funded in part by a generous grant from the Internet Society Foundation. ISL is a US-based 501(c)(3) non-profit organization.

The ISL Safety Label

The purpose of the ISL Safety Label is to help various audiences understand what’s happening “under the hood” of mobile apps.

The safety label is organized into risk categories that identify safety risks for people when using the app as it was designed to be used. The label is designed to add risk categories over time. Currently, the label includes only one risk module: Privacy Risks, and the team is working on adding additional risks. Note that the Apple and Android logos are used only to communicate the version of the app and do not constitute an endorsement or affiliation with Internet Safety Labs. Each mark is the property of its respective owner.

ISL Safety Label Structure

The high-level structure of the ISL Safety Label looks like this:

ISL Safety Score & Triggers

All mobile apps carry some risk. The Safety Score and color indicate the level of privacy risk. Note that the ISL app score currently reflects only privacy risks, in particular, the extent of personal data monetization observed in the app.

The ISL Safety Score is a “stoplight” score with five possible values:

• Some Risk (green triangle)
• Medium Risk (yellow diamond)
• High Risk (orange pentagon)
• Critical Risk (red hexagon)
• Not Scored (grey square)

Here are the detailed explanations of each score.

Some Risk (green):

This represents the “safest” of all safety scores. Note that “no risk” is not an option in the ISL scoring rubric as all apps entail some level of risk.

Medium Risk (yellow):

This represents an app with relatively few risky privacy leaks. Apps receive this score when no high-risk or critical-risk triggers are found.

High Risk (orange):

An app will receive a High Risk score when any of the following are observed AND there are no Critical Risk triggers:

• There are more than 15 unique companies found in the network traffic, but none of them are Critical Risk.
• A Customer Data Platform (CDP) is found in the network traffic, but there are no Critical Risk triggers.

Critical Risk (red):

ISL deems the following app behaviors to be of critical risk to users and any app with any of these behaviors will be scored as “Critical Risk”:

• The presence of behavioral ads,
• The presence of network traffic to the advertising realtime-bidding “bidstream”,
• The presence of network traffic to registered data brokers,
• The presence of network traffic to identity resolution platforms,
• The presence of network traffic to a company that performs screen recording,
• The presence of network traffic to a subdomain that is determined to be critical risk for another reason.

Apps that were unable to be fully scored are marked as Not Scored (grey). Apps are unable to be fully scored if we couldn't fully exercise the app. This happens if: (1) we tested a version of the appwithout using scool-provided credentials, (2) the app was found to be broken, or (3) the app was a paid app and we didn't pay for it.

Safety Score Calculation

The app safety score is fairly complex. The flow chart below illustrates the high level sequence of calculations and checks.

Please contact us if you'd like more information about how risk scores are calculated.

Privacy Risks Summary

This section provides information on the number and riskiness of third parties found in either observed network traffic or as SDKs in the app. First, the risk assessment based on the observed network traffic.

Observed Network Traffic:

Aggregator Platforms:

This is the number of aggregator platforms, 0-7, with whom the app communicated during the testing session, based on collected network traffic. The aggregator platforms included are:

• Adobe
• Amazon
• Apple
• Facebook
• Google
• Microsoft
• Twitter

Total # of Companies:

The total number of unique companies observed in the network traffic.

Data Broker Companies:

The number of registered data brokers observed in the network traffic. Note that ISL continually adds and assesses against state registries as new state laws come into effect.

Total # of Subdomains:

The total number of domains observed in the network traffic. Note that a subdomain is of the form prefix.XZY.com for example.

Risky Subdomains:

The total number of High or Critical risk subdomains observed in the network traffic.

Data Broker Subdomains:

The total number of registered data broker subdomains observed in the network traffic.

[Source: From network traffic collection performed by ISL.] Next, the risk assessment of included SDKs.

Total # of SDKs:

This is the total number of SDKs found in the app. [Source: AppFigures]

Risky SDKs:

This is the number of High Risk and Very High Risk SDKs found in the app. [Source: SDKs from AppFigures; ISL SDK Risk Database (to be published in 2023)]

Data Broker SDKs:

This is the total number of SDKs found in the app that are owned by registered data brokers in either the California or the Vermont data broker registries. [Source: California and Vermont data broker registries.]

App Category Average

The App Category Average column displays the average number of SDKs and aggregator platforms for the category of apps. Note that the category is the smallest sub-category as shown in the top part of the label, such as “EdTech -> Safety Platform”.

The “n=number” is the number of apps in that category/subcategory.

This information provides context as to whether the chosen app is behaving better or worse than the category with respect to third parties.

[Source: calculated from ISL app database.]

Risky Behaviors

This section indicates if risky behaviors were observed during manual app test. The following behaviors are key risks:

Ads:

If digital ads were observed in the app, this field will show “Yes”; if no digital ads were observed in the app, the field will show “No”. In this case, “no” is safer and “yes” is riskier. ISL regards digital ads as risky due to the infrastructure of digital advertising and real-time bidding, in particular, which has the capability to share personal information to countless entities in the ad network. [Source: observed by ISL product safety testers.]

Behavioral Ads:

This field reports whether the tester observed behavioral ads in the app. Behavioral advertising refers to the capability to anonymously ‘follow’ consumers across websites, all over the Web. Behavioral ads are ads that clearly rely on information that has followed the user from another site. In other words, behavioral ads are using personal information to customize ads being displayed. This behavior is extremely risky. As above, a “yes” is riskier than a “no”. [Source: observed by ISL product safety testers.]

WebView:

This field indicates if the app uses Webview to display webpages within the app. [Source: observed by ISL product safety testers.]

• From pp 8-9 in Spotlight Report #4:

  What is WebView? All mobile apps with an “in-app browser” are using “Webview” code/APIs, and the rules and protections for these in-app browsers differ somewhat between Apple iOS and Android platforms. WebView is a low-level set of functions (APIs) provided by an operating system within a mobile device that allows native apps to present web pages, woven seamlessly into the application without having to open or close a separate browser. WebView allows native apps to selectively become a browser while keeping the user in the app. A 2011 paper on WebView attacks against Android devices explains this legacy development and data supply chain strategy: “WebView is an essential component in both Android and iOS platforms, enabling smartphone and tablet apps to embed a simple but powerful browser inside them. To achieve a better interaction between apps and their embedded “browsers”, WebView provides a number of APIs, allowing code in apps to invoke and be invoked by the JavaScript code within the web pages, intercept their events, and modify those events. Using these features, apps can become customized “browsers” for their intended web applications.” WebView in iOS relies on WebKit, an open-source browser “engine” first introduced 23 years ago in 1998, which later evolved into the Safari engine used by Apple around 2013. The WebView APIs are therefore limited by WebKit capabilities supported in the OS. iOS WebView is restricted and protected by iOS WebKit (built on Safari) limitations, and Android WebView is restricted and protected by a slightly similar concept with Chromium limitations. However, they both ultimately support in-app browsing as a seamless user experience without giving users full control over their own data sharing preferences within these in-app browsers.

App Category %

The numbers in this column display the percentage of apps in the named category that are positive for the behavior. For example, 60% shown in the Ads Present row means 60% of the apps in the category have ads present. The number of apps in the category (n=32, e.g.) is shown at the very top of the column.

[Source: Calculated from ISL app database.]

User Data

This section lists the categories of sensitive permissions the app asks for. ISL has created these categories for ease of understanding the overall risks related to the app’s use of user data accessible from the device. There are eight permission categories (listed in alphabetical order):

Crash Logs

include permissions that allow the app publisher to receive information when the app crashes. There is a risk of this information including personal details.

Files

include any permission that allows apps to list user data files or their contents, whether in the cloud or on device. This access is risky both because files and filenames can include personal information and because it can be used to fingerprint and reidentify a user even if they have reset other identifiers.

Join User Identifiers

includes any permission that directly assists advertising networks that wish to track users across apps or across device, such as with Apple’s ID for Advertising (IDFA).

Location

includes any permission that potentially allows apps to determine the user’s geographic location. Permissions such as wifi network names and bluetooth connections are included in this category because in many cases these names are distinctive and can be compared against databases to guess the location.

Phone Service

includes permissions that reveal who the user’s carrier is or whether they currently have service. This can serve as a proxy for location. It may also reveal financial wellbeing.

Physical Environment

includes permissions that reveal information about the user’s physical environment, such as granting access to the camera and microphone.

Social Information

includes permissions that reveal who the user associates with, as well as when or where they do so. This includes calendar and contacts.

User Behavior

permissions include anything that would be useful to advertising networks seeking to learn more about a user, such as their psychology or interests.

[Source: AppFigures, iOS App Store, Google Play Store; permission categories created by ISL.]

3rd Party Sharing Details

This section provides details on the 3rd parties likely to be receiving user data. This section has two main parts: (1) details on the observed network traffic, and (2) details on the SDKs.

Both the observed subdomains and SDKs are scored using the same risk tiers and stoplight scores as the app itself (Some Risk (lowest risk), Medium Risk, High Risk, and Critical Risk (highest risk)).

Observed Network Traffic:

This section presents details on the observed subdomains in the network traffic.

Aggregator Platforms Receiving Data:

This section names the aggregator platforms whose subdomains were observed in the network traffic: Adobe, Amazon, Apple, Facebook, Google, Microsoft, and Twitter.

Riskiness of Subdomains by Risk Score:

This section shows the corporate owner for each observed subdomain by risk score.

[Source: From network traffic collection performed by ISL.]

Riskiness of 3rd Party SDKs – by SDK Risk Score:

This section shows the companies behind the SDKs included in the app, sorted by SDK risk score.

ISL is unable to share the names of the SDKs in the apps, so this section displays the names of the companies behind the SDKs, as well as the ISL designated category.

[Source: AppFigures, ISL SDK Risk Database]

User Data Details

This section contains detailed information regarding what user information the app accesses. This part of the safety label is currently comprised of a single section: Riskiness of App Permissions.

Riskiness of App Permissions:

This section lists all the permissions requested by the app. Permissions are grouped by Permission Category. [Source: ISL]

Permission Category:

ISL has created permission categories based on the kind of information accessed. Note that the same permission may be included in multiple categories. See above for the list of permission categories. [Source: ISL]

Permission:

This section lists all the permissions that the app either requests access to or simply accesses without permission. [Source: AppFigures, iOS App Store, Google Play Store]

iOS Apps:

In iOS apps, there are two types of permissions shown in the safety label: (1) permissions that start with “NS”, and (2) permissions that don’t start with NS. Permissions that start with NS actually aren't permissions but map to fields in the Apple Privacy Nutrition Label, which are voluntary disclosures made by the app publisher and are subject to various exceptions that allow the publisher to not disclose use of certain data as per the fine print in Apple's policies.

Android Apps:

The permissions shown for Android apps reflect the actual permissions requested by the app.

App Category %

This column shows the percentage of apps in the app category that use permissions in this permission category. [Source: Calculated from ISL app database.]

This information provides a rough approximation of app category behavioral norms for the permission category. When this number is low (15% or less), and there are one or more permissions shown for the app, it can be viewed as atypical behavior for the category. This may or may not be significant depending on the exact purpose of the app.