Artificial Intelligence has revolutionised how digital media is created, edited and distributed. However, the same tools that power visual effects, advertising and creative production are also being misused to manipulate identities online.

One of the most serious emerging threats is deepfake and AI morphed photo and media, synthetic images or videos where a person’s face, voice or likeness is digitally replaced using machine learning models. These manipulations are becoming increasingly realistic and easier to create.

Celebrities and public figures are often the first targets because their photographs, interviews and videos are widely available online. But the implications go far beyond celebrity culture. Deepfake abuse is rapidly evolving into a broader cybercrime affecting professionals, influencers and private individuals.

This article examines the rise of AI-generated manipulation in India, recent high-profile incidents, the legal framework currently available to victims, and how harmful content can be reported and removed.

The Rise of AI Morphed Photo and Deepfake Technology

Deepfakes are typically created using deep learning systems, particularly Generative Adversarial Networks (GANs) and diffusion-based models. These systems analyse large datasets of images and videos to learn facial features, expressions and voice patterns.

A typical deepfake creation workflow includes:

1. Collecting images and videos of the target person
2. Training an AI model to replicate facial movements and expressions
3. Replacing the person’s face or voice in another video or image
4. Rendering a highly realistic manipulated output

What once required advanced technical expertise can now be produced using consumer-grade AI tools and mobile apps, making the technology accessible to a much wider audience.

Where Deepfake Content Spreads

AI-generated manipulations can spread extremely quickly across digital ecosystems. Common distribution channels include:

• Social media platforms
• Messaging apps such as WhatsApp and Telegram
• Video-sharing websites
• Anonymous online forums
• Private piracy and file-sharing communities

Because these platforms enable rapid sharing and re-uploads, manipulated content can reach millions of viewers within hours.

Recent Celebrity Deepfake Incidents in India

Several incidents involving well-known actors and public figures have triggered widespread debate about AI misuse and digital safety.

Rashmika Mandanna Deepfake Video

One of the most widely discussed AI morphed photo manipulation cases in India involved actress Rashmika Mandanna.

A viral video appeared to show the actress entering an elevator wearing a black outfit. Later investigations revealed that the original clip belonged to a social media influencer and Mandanna’s face had been digitally superimposed using AI tools.

The video spread rapidly across social media platforms before being debunked. The incident sparked national outrage and prompted calls for stronger regulation of deepfake technology. The case eventually led to a police investigation and arrest of a suspect, marking one of the first widely recognised deepfake investigations in India.

Another incident involves an action still of Katrina Kaif from Tiger 3 which was digitally altered online to appear more revealing, while a viral video falsely featuring Alia Bhatt used AI to insert her face onto another woman’s body, highlighting growing concerns around digital impersonation, identity misuse, and the spread of manipulated content on social media platforms. A criminal complaint raised by actress Anupama Parameswaran, traced the source to a 20-year-old individual accused of creating and distributing the morphed images.

Legal Implications of Deepfake Abuse in India

India currently does not have a single comprehensive law specifically regulating deepfakes. However, several existing legal provisions can be used to address AI-generated abuse to an extend.

Information Technology Act, 2000

The IT Act is the primary cyber law governing online offences in India. Relevant provisions may include:

• Section 66C – Identity theft
• Section 66D – Cheating by personation using computer resources
• Section 67 – Publishing or transmitting obscene content online
• Section 67A – Publishing sexually explicit material

Deepfake images involving explicit or morphed content may fall under these provisions.

Bharatiya Nyaya Sanhita (BNS)

Under India’s updated criminal law framework, offences involving:

• Obscenity
• Harassment
• Defamation
• Digital impersonation

It can potentially apply to deepfake creators and distributors.

Personality Rights

Indian courts increasingly recognise personality rights, which protect an individual’s:

• Name
• Image
• Coice
• Likeness
• Reputation

These rights are linked to the right to privacy recognised by the Supreme Court in the landmark case of: Justice K. S. Puttaswamy v. Union of India. Courts have repeatedly held that unauthorised commercial or reputational exploitation of a person’s identity may violate constitutional privacy protections.

Key Legal Challenges in Deepfake Enforcement

Despite these legal protections, several enforcement challenges remain.

1. Lack of Specific Deepfake Legislation

Current laws were written before generative AI technologies became widespread, leaving legal ambiguity in AI-specific offences.

2. Anonymous Offenders

Perpetrators often hide behind:

• Fake social media accounts
• VPN services
• Anonymous forums
• Encrypted messaging platforms

This makes identification difficult.

3. Jurisdictional Issues

Deepfake content is often hosted on servers outside India, creating cross-border enforcement challenges.

4. Rapid Content Spread

Even if the original post is removed, copies and mirrors can continue circulating, making complete removal difficult.

How Victims Can Report Deepfake Abuse Online

If someone becomes a victim of AI-generated abuse or identity manipulation like AI morphed photo circulated online defaming them, several reporting mechanisms are available.

1. Report to the National Cyber Crime Portal

Victims can file a complaint through the Government of India’s official cybercrime reporting system: https://cybercrime.gov.in

Cases involving women and children receive priority investigation.

2. Contact the Digital Saheli Helpline

Digital Saheli is a support initiative that assists women facing online abuse, including image-based harassment and digital identity misuse: Call 1930

The helpline provides guidance on reporting cybercrime, digital safety and legal assistance.

3. Request Platform Takedown

Victims should also immediately report the content to the platform hosting it, such as Instagram, X (Twitter), Facebook, YouTube, Telegram, etc. Most major platforms have policies prohibiting non-consensual intimate imagery and impersonation.

4. Professional Anti-Piracy & Content Protection Support

For high-profile individuals, public figures, studios, or brands facing large-scale distribution of manipulated content, specialised digital enforcement services may be necessary.

BLOCK X provides:

• Real-time tracking of unauthorised content
• Deep web and social media monitoring
• Platform takedown enforcement
• Evidence documentation for legal action
• Piracy and impersonation disruption

This ensures that no links remain unchecked across the internet ecosystem, enabling faster containment of harmful content. For assistance, [email protected]

Why Stronger AI Regulations Are Needed

Legal experts increasingly argue that India needs dedicated legislation addressing deepfake technology and AI-driven impersonation.

Potential regulatory reforms may include:

• Legal definitions for deepfakes
• Criminal penalties for non-consensual AI-generated imagery
• Mandatory AI-generated content disclosures
• Platform obligations for deepfake detection
• Faster takedown frameworks

Such measures would strengthen digital identity protection and online safety in the AI era.

Deepfake technology represents one of the most complex challenges in the digital age. While it offers legitimate creative applications, its misuse has created serious risks involving privacy violations, identity theft, reputational harm and cyber harassment.

One of the biggest legal gaps in India is the absence of legislation specifically addressing deepfake technology. Current laws such as the IT Act and the BNS do not specifically include provisions dealing with AI-related offences.

Addressing this threat requires coordinated efforts between law enforcement, courts, technology platforms and digital protection organisations.

As AI tools become more powerful, protecting digital identity and consent will become one of the most important legal challenges of the coming decade.

To get help with bulk removals, contact us.