The rise of artificial intelligence has brought groundbreaking advancements, but it has also opened the door to unethical practices. A BBC investigation recently highlighted a distressing situation: fake AI influencer accounts on Instagram and TikTok are exploiting and sexualizing black women, violating platform policies while profiting from these fabricated personas.

Digital Theft and Manipulation

Eighteen-year-old Ria discovered her identity manipulated in a deeply unsettling way. A selfie video she had posted online was digitally altered to darken her skin, and this manipulated content appeared on the profile of a non-existent "black influencer." This account amassed 3 million followers in just a few weeks, revealing the viral appeal of such fraudulent content.

What happened to Ria is far from an isolated incident. The BBC’s investigation uncovered dozens of AI-generated accounts presenting black women in highly sexualized and stereotyped ways. These profiles mimic the behaviors of authentic influencers, sharing viral dance videos, lip-sync clips, and snippets of "lifestyle" content. However, patterns quickly emerge: the fabricated avatars are often depicted in revealing clothing, exhibiting exaggerated body shapes, and featuring unnaturally dark skin tones.

These accounts frequently use names with descriptors like "black," "noir," "dark," or "ebony." Comments targeting these profiles sometimes hint at racial fetishes, compounding the harm and reinforcing racist themes.

The Money Behind Fake Influencers

The exploitation extends beyond social media to external platforms. The BBC collaborated with AI researcher Jeremy Carrasco and discovered that at least 60 such accounts linked to paid-for explicit content on third-party sites. On these platforms, the avatars are labeled as AI-generated, providing a semblance of transparency. However, on Instagram and TikTok—where these accounts build massive followings—there’s little to no indication that the profiles are fake.

According to Carrasco, these AI-generated profiles likely yield substantial profits. Subscription-based explicit content channels often come with high fees, and with the traffic some of these accounts generate, the financial incentives are considerable despite the obvious ethical violations.

Violations of Platform Standards

What amplifies the problem is the platforms' delayed response. Platforms like Instagram and TikTok prohibit AI-generated content that uses someone’s likeness without permission. TikTok, for example, requires creators to label realistic AI content as such. Yet, most of these profiles operate without disclosure, misleading millions of users.

Some of these fake accounts even deny their AI-generated nature when questioned by followers. This deception fosters a cycle of exploitation, blurring the line between authenticity and manipulation.

Activists and User Efforts to Combat the Trend

Moroccan model and content creator Huda, who advocates for the accurate representation of black women, argues that this trend aims to erase real black women from public spaces. Huda sees these AI-generated accounts as undermining authentic voices while perpetuating stereotypes.

Ria, for her part, actively reported the account using her manipulated content multiple times, but the videos stayed online initially. Her original video received little attention compared to the tens of millions of views garnered by the altered clip. "I’m not sure if I’m more concerned about them taking my video to promote explicit content or people actually believing it," Ria remarked.

Social Media Platforms Respond

After the BBC presented evidence of these violations to TikTok, the platform acted swiftly. Within days, they removed content and banned 20 accounts, including the one using Ria’s video. TikTok stated its commitment to prohibiting AI-generated content that violates user rights, emphasizing the importance of disclosing AI manipulation.

Instagram’s parent company, Meta, also began investigating the identified accounts. While the actions are steps in the right direction, the broader issue persists. The profitability and minimal accountability of AI-generated influencers mean this unethical practice could continue unless stricter measures are implemented.

Practical Implications

The rise of AI-generated influencers exploiting racial and gender stereotypes raises several pressing concerns. Here are some practical takeaways:

• Stronger Platform Policies: Social media platforms must enforce stricter rules against AI-generated content that exploits or deceives users. Clear labeling requirements and swift action on policy breaches could deter such accounts.
• User Awareness: Users should stay vigilant about the authenticity of online profiles, especially those garnering sudden, massive followings. Questioning the origins of influencer content can help expose deceptive practices.
• Advocacy for Representation: Efforts by activists like Huda remind us of the importance of supporting genuine content creators who offer authentic and respectful portrayals of underrepresented groups.
• Legal Frameworks: Governments and regulatory bodies may need to implement specific laws targeting the misuse of AI for financial or social exploitation, ensuring accountability for those creating and profiting from such content.

Conclusion

The unethical use of AI to fabricate influencers portraying stereotypes and exploiting black women is a troubling trend. While actions by TikTok and Meta demonstrate that platforms can intervene, the broader challenge lies in addressing the root cause: the accessibility and profitability of this deceptive technology. Supporting authentic creators, reporting suspicious accounts, and demanding accountability from platforms are critical steps users can take to ensure social media spaces remain respectful and inclusive.