Pinterest Europe Ltd.
Government orders to act against illegal content
Article 15(1)(a)
Notices received from users and flaggers
Article 16
Own-initiative moderation
Article 15(1)(c) and (d)
Restriction types applied (terms & conditions)
Account-level actions
Article 15(1)(d)
| Account suspensions | 325 |
|---|---|
| Account terminations | 2,043,236 |
| Total account actions | 2,043,561 |
Automated detection accuracy
Pinterest reports the accuracy of its automated detection. These are its own figures, measured against its own method and denominators, and are not comparable with other providers'. The detection tool or method is shown as filed.
| Tool or method | Scope | Accuracy | Precision | Recall |
|---|---|---|---|---|
| Accuracy provides an overview of how many overall decisions taken by our automation - both enforcements and non-enforcements - are correct. Our automated processes utilize a combination of signals and systems to inform content moderation decisions. Accordingly, we provide an aggregate measure of accuracy that reflects the overall performance of our automation. We calculate the indicator of accuracy for our automated content moderation measures as follows: Accuracy = (True Positives + True Negatives ) / (True positives + True Negatives + False Positives + False Negatives) Additional details about our approach can be found in the 11_qualitative tab. | Total number | 92.0% | — | — |
| Accuracy provides an overview of how many overall decisions taken by our automation - both enforcements and non-enforcements - are correct. Our automated processes utilize a combination of signals and systems to inform content moderation decisions. Accordingly, we provide an aggregate measure of accuracy that reflects the overall performance of our automation. We calculate the indicator of accuracy for our automated content moderation measures as follows: Accuracy = (True Positives + True Negatives ) / (True positives + True Negatives + False Positives + False Negatives) Additional details about our approach can be found in the 11_qualitative tab. | Own-initiative | 92.0% | — | — |
| Precision reflects how often our automated enforcements are correct. Our automated processes utilize a combination of signals and systems to inform content moderation decisions. Accordingly, we provide an aggregate measure of precision that reflects the overall performance of our automation. We calculate the indicator of precision for our automated content moderation measures as follows: Precision = True Positives / (True Positives + False Positives) Additional details about our approach can be found in the 11_qualitative tab. | Own-initiative | — | 96.0% | — |
| Precision reflects how often our automated enforcements are correct. Our automated processes utilize a combination of signals and systems to inform content moderation decisions. Accordingly, we provide an aggregate measure of precision that reflects the overall performance of our automation. We calculate the indicator of precision for our automated content moderation measures as follows: Precision = True Positives / (True Positives + False Positives) Additional details about our approach can be found in the 11_qualitative tab. | Total number | — | 96.0% | — |
| Recall measures the proportion of policy violations successfully identified by automation. Our automated processes utilize a combination of signals and systems to inform content moderation decisions. Accordingly, we provide an aggregate measure of recall that reflects the overall performance of our automation. We calculate the indicator of recall for our automated content moderation measures as follows: Recall = True Positives / (True Positives + False Negatives) Additional details about our approach can be found in the 11_qualitative tab. | Own-initiative | — | — | 93.0% |
| Recall measures the proportion of policy violations successfully identified by automation. Our automated processes utilize a combination of signals and systems to inform content moderation decisions. Accordingly, we provide an aggregate measure of recall that reflects the overall performance of our automation. We calculate the indicator of recall for our automated content moderation measures as follows: Recall = True Positives / (True Positives + False Negatives) Additional details about our approach can be found in the 11_qualitative tab. | Total number | — | — | 93.0% |
Full per-tool and per-language detection figures are inExplore (automated_means_accuracy).
In Pinterest's words
Pinterest's long-form answers to the standard qualitative questions every platform must answer (Article 42 of the DSA). How it moderates content, how it measures accuracy, how its teams are resourced. Its own words. Expand each to read. (Short notes pinned to individual figures are under "Footnotes from Pinterest" below.)
High-level description of the content moderation governance structure
Meaningful and comprehensible information regarding content moderation engaged in at the providers' own initiative
Methodology used to compute the number of human resources dedicated to content moderation
Qualifications of the human resources dedicated to content moderation
Qualitative description of indicators of accuracy and possible rate of error of automated means
Qualitative description of the automated means
Safeguards applied to the use of automated means
Specification of the precise purposes to apply automated means
Summary of the content moderation engaged in at the providers’ own initiative
Support given to human resources dedicated to content moderation
Training given to human resources dedicated to content moderation
Raw data
Every figure on this page comes from Pinterest's filing as loaded into RTFP's public database. You can query the underlying data directly via thepublic API. The original filing is linked from thesources page.
Footnotes from Pinterest
Short notes Pinterest pinned to specific figures in its filing. Definitions, clarifications and corrections written against individual numbers, shown verbatim. (For its longer descriptions of how it moderates, see "In Pinterest's words" above.)
Show 66 notes
Article 16 notices
- Actions on the basis of lawThis is the total number of actions that were taken following receipt of a notice, that were based on the illegality of the content.
- Actions on the basis of law (Trusted Flaggers)This is the total number of actions that were taken following receipt of a notice from a Trusted Flagger, that were based on the illegality of the content.
- Actions on the basis of termsThis is the total number of actions that were taken following receipt of notice, that were based on the incompatibility of the information with Pinterest’s Terms of Service.
- Actions on the basis of terms (Trusted Flaggers)This is the total number of actions that were taken following receipt of notice from a Trusted Flagger, that were based on the incompatibility of the information with Pinterest’s Terms of Service.
- Items in noticesThis is the total number of specific items of information that were included in the total number of notices received.
- Items in notices (Trusted Flaggers)This is the total number of specific items of information that were included in the total number of notices received from Trusted Flaggers.
- Median time to actionDuring the reporting period, we did not receive any notices related to this category. The median time is listed as 0 - as it applies to Pinterest - but the result of that practice during the reporting period is zero.
- Median time to actionThe median time to take action is calculated based on the time the notice was received until the time the notice was addressed. This includes notices that Pinterest took no action on after determining no violation was found and the content was not illegal. The median time is indicated in hours.
- Median time to action (Trusted Flaggers)During the reporting period, we did not receive any notices related to this category. The median time is listed as 0 - as it applies to Pinterest - but the result of that practice during the reporting period is zero.
- Median time to action (Trusted Flaggers)The median time to take action is calculated based on the time the notice was received until the time the notice was addressed. This includes notices that Pinterest took no action on after determining no violation was found and the content was not illegal. The median time is indicated in hours.
- Notices receivedThis category encompasses notices received through our European Union Illegal Content Reporting Form for “nudity or pornography” and “other.” In our European Union Illegal Content Reporting Form, users can select “nudity or pornography” as a reporting basis. We have included this option under this category because there is no more suitable category available on this tab for such reports. Of the 2,821 total notices received, 520 related to “nudity or pornography” and 2,301 utilised the "other" reporting option.
- Notices receivedThis includes the total number of notices received during the reporting period through our European Union Illegal Content Reporting Form. Please note the total number of notices received does not equal the total numbers outlined in the rows below. This is because one notice can contain multiple items with separate reporting reasons per item. When one notice includes multiple reporting reasons, we count them separately.
- Notices received (Trusted Flaggers)This is the total number of notices received from Trusted Flaggers during the reporting period. Please note the total number of notices received does not equal the total numbers outlined in the rows below. This is because one notice can contain multiple items with separate reporting reasons per item. When one notice includes multiple reporting reasons, we count them separately.
Complaints, appeals & disputes
- Complaint regarding a decision not to take action on a notice submitted by a Trusted Flagger in accordance with Article 16The median time is indicated in hours.
- Complaint regarding a decision not to take action on a notice submitted in accordance with Article 16The median time is indicated in hours.
- Complaint regarding a decision to remove or disable access to or restrict visibility of informationThe median time is indicated in hours.
- Complaint regarding a decision to remove or disable access to or restrict visibility of informationThis is the number of complaints received regarding a decision to remove or disable access to or restrict visibility of information. Please note that the numbers reported in rows 10-12 below do not equal the total reported in this row. This is due to the fact that we have included “decisions omitted” in the total reported, which include appeals where we are either awaiting additional information or identity verification from the user or the appeal had not been processed by the end of the reporting period. It also includes appeals where the content had been deleted by the user, duplicate appeals, or cases where the content was otherwise unavailable at the time of Pinterest’s review.
- Complaint regarding a decision to restrict the ability to monetise informationPinterest does not have enforcements that map to this restriction type, thus this is not applicable.
- Complaint regarding a decision to suspend or terminate an accountThe median time is indicated in hours.
- Complaint regarding a decision to suspend or terminate an accountThis is the number of complaints received regarding a decision to suspend or terminate an account. Please note that the numbers reported in rows 20-22 below do not equal the total reported in this row. This is due to the fact that we have included “decisions omitted” in the total reported, which include appeals where we are either awaiting additional information or identity verification from the user or the appeal had not been processed by the end of the reporting period. It also includes appeals where the content had been deleted by the user, duplicate appeals, or cases where the content was otherwise unavailable at the time of Pinterest’s review.
- Complaint regarding a decision to suspend or terminate the provision of the servicePinterest does not have enforcements that map to this restriction type, thus this is not applicable.
- Number of complaints submitted to the internal-complaints mechanismDecisions omitted include appeals where we are awaiting additional information or identity verification from the user, or where the appeal had not been processed by the close of the reporting period. This also includes appeals where the content has been deleted by the user, duplicate appeals, or cases where the content was otherwise unavailable at the time of Pinterest’s review. Please note that in this reporting period, 4,460 of these appeals were duplicates. An additional 4,308 appeals related to cases where we work with a trusted third party partner to process appeals, most of which are still in the waiting stage because the user had not completed the identification verification process and our partner was not able to proceed with their appeal.
- Number of complaints submitted to the internal-complaints mechanismDecisions partially reversed include situations where we determined our original decision was too strict and reversed some restrictions on the content.
- Number of complaints submitted to the internal-complaints mechanismDecisions reversed include situations where we reversed our original decision.
- Number of complaints submitted to the internal-complaints mechanismDecisions upheld include situations where we upheld our original decision. It also includes situations where we determined our original decision was not strict enough and imposed further restrictions.
- Number of complaints submitted to the internal-complaints mechanismThe median time is indicated in hours.
- Number of complaints submitted to the internal-complaints mechanismThis includes appeals from EU users regarding content moderation restrictions. Also included are all appeals concerning decisions taken following a notice received through our European Union Illegal Content Reporting Form. Please note this may include appeals from non-EU users that resulted from a decision based on a notice submitted via this form.
- Number of disputes submitted to out-of-court dispute settlement bodiesDecision omitted includes out of scope and duplicate disputes and situations where Pinterest reversed its original decision before a decision was reached by the out-of-court dispute settlement body.
- Number of disputes submitted to out-of-court dispute settlement bodiesDuring the reporting period we did not receive any decisions from out-of-court dispute settlement bodies, therefore the median time it took to reach a decision is not applicable in this report.
- Number of disputes submitted to out-of-court dispute settlement bodiesDuring the reporting period we did not receive any decisions from out-of-court dispute settlement bodies.
- Number of disputes submitted to out-of-court dispute settlement bodiesThis is the number of disputes concerning a content moderation restriction or a decision on a notice that were submitted to an out-of-court dispute settlement body, including out of scope and duplicate disputes. Please note that, at the end of the reporting period, 8 of these disputes were still pending and Pinterest was awaiting a decision from the out-of-court dispute settlement body.
- Number of restrictions newly imposed as a result of an internal complaintThis includes situations where, upon review of a complaint, we determined our original decision was not strict enough and imposed further restrictions.
- Number of suspensions enacted for the provision of manifestly illegal contentIf a user frequently posts manifestly illegal content, we warn the individual that continuation of the behaviour will result in deactivation of the user’s account. This includes users who were deactivated due to repeated violations of our Community Guidelines, and this volume may overstate the number of deactivations specifically for manifestly illegal content.
- Number of suspensions enacted for the provision of manifestly unfounded complaintsIf a reporter frequently submits manifestly unfounded appeals, we warn the individual that continuation of the behaviour will result in suspension of their ability to use the abused systems. If, following this warning, the individual continues to abuse the systems, Pinterest will suspend for a reasonable period of time their ability to use the abused systems.
- Number of suspensions enacted for the provision of manifestly unfounded noticesIf a reporter frequently submits manifestly unfounded reports, we warn the individual that continuation of the behaviour will result in suspension of their ability to use the abused systems. If, following this warning, the individual continues to abuse the systems, Pinterest will suspend for a reasonable period of time their ability to use the abused systems.
Government orders
- Article 10 orders receivedThese are orders to provide information from EU Member States' authorities where the issuing authority omitted a reference to a legal ground.
- Article 10 orders receivedThis is the total number of orders to provide information received from EU Member States’ authorities in the reporting period. These orders may include law enforcement requests, such as emergency disclosure requests, and requests issued pursuant to local law.
- Article 10: median time to give effectDuring the reporting period, we did not receive any orders related to this category. The median time is listed as 0 - as it applies to Pinterest - but the result of that practice during the reporting period is zero.
- Article 10: median time to give effectThe median time to give effect to the order to provide information is calculated based on the time the order was successfully transmitted to Pinterest, until the time the order was addressed. The median time is only calculated based on orders where effect is given to the order, thus this is not based on the total number of orders received. The median time is indicated in hours.
- Article 10: median time to give effectThe median time to give effect to the order to provide information is listed as 0 because we did not provide information in response to this request.
- Article 10: median time to inform of receiptDuring the reporting period, we did not receive any orders related to this category. The median time is listed as 0 - as it applies to Pinterest - but the result of that practice during the reporting period is zero.
- Article 10: median time to inform of receiptThe median time to inform the authority of the receipt of the order to provide information is listed as 0 because our confirmation is automated and immediate.
- Article 9 orders receivedThis is the total number of orders received from EU Member States’ authorities to act against illegal content in the reporting period. We review the orders to determine if the content identified violates our Community Guidelines or local law. Our teams take action on violations, ranging from deactivating the content globally to restricting access to the content within the relevant country if it appears to violate local law, but does not violate our policies.
- Article 9: items in ordersThis is the total number of specific items of information that were included in the total number of orders received from EU Member States’ authorities.
- Article 9: median time to give effectDuring the reporting period, we did not receive any orders related to this category. The median time is listed as 0 - as it applies to Pinterest - but the result of that practice during the reporting period is zero.
- Article 9: median time to give effectThe median time to give effect to the order to act against illegal content is calculated based on the time the order was successfully transmitted to Pinterest, until the time the order was addressed. The median time is indicated in hours.
- Article 9: median time to inform of receiptDuring the reporting period, we did not receive any orders related to this category. The median time is listed as 0 - as it applies to Pinterest - but the result of that practice during the reporting period is zero.
- Article 9: median time to inform of receiptThe median time to inform the authority of the receipt of the order to act against illegal content is listed as 0 because our confirmation is automated and immediate.
Human resources
- Number of total moderators with sufficient linguistic expertiseThis is the number of total moderators available, both internally and externally, that have sufficient linguistic expertise in the indicated language. Sufficient linguistic expertise is defined as at least equivalent to level B2 of the Common European Framework of Reference for Languages (CEFR). Pinterest is primarily a visual platform, nevertheless every member of our review team speaks at least one official language of a Member State of the EU, and we have additional external resources available to assist with any EU Member State official language not covered by our moderators.
Own-initiative (illegal content)
- Measures (total)Pinterest engages in content moderation at its own initiative to detect and address content that violates our policies. However, we do not engage in content moderation at our own initiative specifically for the purpose of identifying illegal content. Any actions we take related to illegal content are conducted either in accordance with our policies or in response to an order from a Member State authority or a notice submitted through our European Union Illegal Content Reporting Form. Therefore, for the purposes of this report, Pinterest has not taken any measures at its own initiative in respect of illegal content.
Own-initiative (terms of service)
- Account restriction: suspensionThis restriction type includes restrictions for copyright violations. When a user has received multiple complaints about certain copyright violations, we restrict the ability for the user to post and save content for 14 days so that they can audit their account and remove any further instances of infringement. This temporary restriction is applied manually and automatically lifts after the 14 day period; it acts as a final warning after which any further unrefuted claims of infringement on their account will result in account termination. This restriction type also includes suspensions applied as a result of the provision of manifestly unfounded complaints and notices.
- Account restriction: terminationThis restriction type includes account deactivations. Any account, regardless of privacy settings, may be deactivated for violating our policies. When an account is deactivated, all of its Pins and boards are also deactivated. That means that if you search for them on Pinterest or click on an old link to their profile, that profile won’t show up anymore. Their Pins won’t appear anywhere on Pinterest. And the deactivated user won’t be able to access their own Pins or boards, either. This restriction type also covers account deactivations imposed where a user frequently provides manifestly illegal content or repeatedly violates our policies.
- Measures (total)This is the number of measures that have been taken at Pinterest's own initiative for violations of our Terms of Service. Own initiative means that the decision to impose a restriction on an account or content has not been preceded by an order from a Member State authority or a notice submitted through our European Union Illegal Content Reporting Form.
- Measures solely automatedThis is the number of measures that were taken at Pinterest's own initiative using automated means. More information about our automated means can be found in the "qualitative description of the automated means" section on 11_qualitative tab.
- Monetary restriction: otherWe have not mapped any enforcement types to this restriction category.
- Monetary restriction: suspensionWe have not mapped any enforcement types to this restriction category.
- Monetary restriction: terminationWe have not mapped any enforcement types to this restriction category.
- Service restriction: suspensionWe have not mapped any enforcement types to this restriction category.
- Service restriction: terminationWe have not mapped any enforcement types to this restriction category.
- Visibility restriction: age-restrictThis restriction type includes ads limited approval solely or partially based on age. Limited approval: Limited approval means that an ad may run, but with restrictions - for example, it may only be shown to people over a certain age.
- Visibility restriction: demoteThis restriction type includes limited distribution of Pins, boards, accounts and comments. It also includes advertiser offboardings, merchant rejections and catalogue restrictions. Limiting distribution: When we limit the distribution of content, that content will continue to be accessible to the user who posted it, but it won’t be distributed through recommendation or discovery surfaces such as Pinterest search results or home feed. Advertiser offboarding: When we offboard an advertiser, they no longer have access to Pinterest’s advertising tools. This doesn’t affect their ability to use other, non-advertising Pinterest products. Catalogue restrictions: When we prevent the distribution of catalogues, they cannot be used in ads. When we limit the distribution of catalogues, they can only be allowed in ads if they comply with our Advertising Guidelines.
- Visibility restriction: disableThis restriction type includes domain blocks, music mutes and ad rejections. Domain blocks: We may block domains (links) that lead to content or are associated with content that may violate our policies. Music mutes: Pinterest works with a third-party partner to proactively and automatically identify audio that matches a file provided by a copyright holder to the third-party partner. When our third-party partner matches the digital signature of an audio file on Pinterest with a file in their library, the audio on the Pin is muted automatically. Ad rejections: When an ad is rejected, it will not be shown to users. We may reject an ad before ever showing it to anyone, or after the campaign has already begun to run - for example, in response to a report from a user who saw the ad.
- Visibility restriction: labelWe have not mapped any enforcement types to this restriction category.
- Visibility restriction: limit interactionWe have not mapped any enforcement types to this restriction category.
- Visibility restriction: otherThis restriction type includes content and accounts that have been blocked in territory or provided limited approval. This also includes ads with limited approval based on any reason other than age. Blocked in territory: We may restrict access to content within a relevant territory if it is determined to violate local law but does not violate our policies. The blocked in territory actions reported in this tab are limited to restrictions applied to content or accounts of users located in the EU, where the enforcement action was taken based on a violation of law outside the EU and its Member States. Limited approval: Limited approval means that an ad may run, but with restrictions - for example, it may only be shown to people in certain regions.
- Visibility restriction: removalThis restriction type includes deactivated boards, Pins and comments. When we deactivate content, that content is no longer available to anyone on the platform. Deactivation may also be referred to as “removal.”