AI content moderation uses machine learning and automated protocols to identify and remove prohibited speech at scale, but often results in over-removal and systemic bias. According to a 2025 joint declaration by special rapporteurs and representatives from the UN, OSCE, OAS, and ACHPR, these systems risk erasing linguistic diversity and amplifying existing inequalities through opaque training processes.
The Shift Toward Permanent AI Governance
Social media platforms have transitioned from using AI as a supplementary tool to making it a permanent feature of how platforms govern speech online. In 2018, Mark Zuckerberg appeared before the U.S. Senate’s Commerce and Judiciary Committees and disclosed that 99 percent of the ISIS and Al Qaida content removed by Facebook was flagged by AI before any human sees it. This marked a shift toward proactive, automated detection.
The COVID-19 pandemic in 2020 accelerated this trend. A surge in social media use and a spike in misinformation, combined with a reduction in human moderation staff, led platforms—particularly Meta—to rely more heavily on automation.
Systemic Risks: Over-Removal and Algorithmic Bias
Automated systems struggle with the nuance of human language, leading to significant collateral damage in free expression. The 2025 joint declaration by special rapporteurs and representatives of the UN, OSCE, OAS, and ACHPR warns that reliance on biased datasets leads to the “homogenisation of expression.”
Specific impacts include:
- Human Rights Documentation: A 2019 paper co-authored by EFF, Witness, and Syrian Archive found that automated extremist content regulations impact human rights documentation. Human Rights Watch corroborated this in 2020, stating there is no way of knowing how much potential evidence of serious crimes is disappearing without anyone’s knowledge.
- Marginalized Communities: GLAAD has highlighted that when moderation systems lack nuance, transparency, and human oversight, they can fail to curb harassment and wrongly suppress legitimate LGBTQ content.
- The “Low-Resource” Language Gap: The Center for Democracy and Technology reports persistent inequities in the Global South. AI models for “low-resource” languages—those with a relative scarcity of training data—are more difficult to develop for equitable and accurate results.
The Trade-off: Human Wellness vs. Algorithmic Accuracy
Platforms justify the expansion of AI by citing the mental health toll on human moderators who must review content that varies from whimsical to horrific. Outsourcing this work to the bots can offer some relief.
However, this benefit is partially offset by the conditions of the AI supply chain. The humans hired to train the AI models face a similar dynamic, often for little pay and with devastating mental health consequences.
Comparison: Human vs. AI Moderation
| Feature | Human Moderation | AI Moderation |
|---|---|---|
| Speed/Scale | N/A | N/A |
| Contextual Nuance | N/A | N/A |
| Psychological Impact | Severe mental health consequences | N/A (though trainers are affected) |
| Consistency | N/A | N/A |
The Requirement for Transparency and Accountability
The question today is not whether companies will use AI to moderate content, but under what conditions they should do so. Civil society, researchers, and human rights experts argue that the current approach—where platforms deploy models without fully realized transparency, accountability, and due process safeguards—is unsustainable. For AI to serve expression rather than stifle it, platforms must ensure that automated decisions are not erasing cultural or political diversity.