Government Urges Safeguards in AI Apps to Protect Women Online

The Indian government is increasing pressure on tech companies to ensure the safety of women on online platforms, particularly those utilizing artificial intelligence (AI). Recent directives from the Ministry of Electronics and Information Technology (MeitY) signal a growing concern over obscene and harmful content, and a demand for more robust safeguards within AI-enabled applications.

recent Government Directives

On Monday, MeitY issued a directive to all social media companies, demanding the removal of obscene content and warning of strict action for non-compliance. This advisory, the third issued in 2024, reiterates the platforms’ obligations under the Information Technology (IT) Rules and the IT Act.

The ministry emphasized the need for greater “rigour” in monitoring vulgar and obscene content. It specifically called for intermediaries to improve their “identification, reporting and expeditious removal” of content deemed obscene, indecent, vulgar, pornographic, paedophilic, harmful to children, or otherwise unlawful. This directive builds upon previous concerns regarding the proliferation of non-consensual intimate images (NCII) and online harassment.

Focus on AI-Enabled Applications

Beyond general content moderation, the government is specifically concerned about the potential for AI to exacerbate online harms against women. AI-powered features like deepfakes, image manipulation tools, and automated content generation can be used to create and disseminate abusive material at scale. The government is reportedly considering engaging directly with tech companies – denoted as “X” in initial reports – to ensure that AI-enabled apps are designed with robust safeguards to prevent misuse and protect users.

The Risks Posed by AI

• Deepfakes: AI can create realistic but fabricated videos or images,often used to depict individuals in compromising situations without their consent.
• Automated Harassment: AI-powered bots can be used to target individuals with abusive messages and coordinated harassment campaigns.
• Image manipulation: AI tools can easily alter images to create non-consensual intimate images (NCII) or to defame individuals.
• Algorithmic Bias: AI algorithms can perpetuate and amplify existing societal biases, leading to discriminatory outcomes.

IT Rules and Intermediary Obligations

The IT rules, particularly IT Rule 4(1)(d), outline the due diligence obligations of social media intermediaries. These obligations include proactively identifying and removing unlawful content,responding to complaints,and assisting law enforcement agencies. The recent advisory from MeitY underscores the government’s expectation that platforms will take these obligations seriously and implement more effective content moderation practices.

Key Takeaways

• The Indian government is actively addressing online safety concerns, particularly for women.
• Recent directives from MeitY demand stricter content moderation on social media platforms.
• There is a growing focus on the risks posed by AI-enabled applications and the need for built-in safeguards.
• Social media intermediaries are legally obligated to comply with the IT Rules and proactively address unlawful content.

Looking Ahead

The government’s increased scrutiny of online platforms and its focus on AI safeguards represent a notable step towards creating a safer online environment for women. Further engagement between the government and tech companies, along with the progress of clear regulatory frameworks for AI, will be crucial in addressing these evolving challenges. The effectiveness of these measures will depend on the willingness of platforms to prioritize user safety and invest in robust content moderation technologies and practices.