OpenAI plans erotica on ChatGPT, sparking UK safety scrutiny

OpenAI is planning to permit a wider range of content, including erotica, on its ChatGPT platform, raising significant regulatory and ethical questions regarding the handling of adult content by generative artificial intelligence services.

This potential policy change positions OpenAI alongside other technology firms, such as X's Grok, which have been taking cautious steps towards allowing adult content while navigating new and complex legal frameworks, such as the United Kingdom's Online Safety Act (OSA) and the European Union's Digital Services Act.

Regulatory implications

James Clark, AI and Digital Regulation Partner at Spencer West LLP, noted that OpenAI's move could subject ChatGPT to heightened compliance standards under the OSA, which classifies certain services as "pornographic content providers".

"By offering erotica, ChatGPT could fall into the OSA's category of 'pornographic content providers'. This classification triggers heightened compliance obligations, including proactive risk assessments, transparency and reporting. Established providers of pornographic content have been grappling in recent months with the requirement to age-verify UK users before permitting access to content, and the challenge of doing this in a robust and effective way whilst still preserving privacy and security for users in a sensitive area."

Clark highlighted that, if ChatGPT is regulated as a pornographic content provider, it must demonstrate that its age-verification measures are not only effective but also privacy-conscious and impervious to circumvention. Any vulnerabilities could place the company at risk of enforcement action by the UK regulator Ofcom.

Text versus synthetic content

While the OSA defines and regulates pornographic content, there remains complexity over what constitutes such content, particularly when generated by AI. Larry Wong, Associate at Katten Muchin Rosenman LLP, explained that the OSA does not classify purely text-based erotica as pornographic content. However, when text-to-speech or synthetic audio functions are employed, the content may be subject to stricter controls, provided its main purpose is sexual arousal.

"ChatGPT providing 'qualifying' pornographic content (i.e. providing content beyond just text) would result in it being qualified as a Part 5 provider of pornographic content. Ofcom's guidance makes clear that services making available generative AI tools to allow the creation of content through prompts or controls by a user will be treated as a Part 5 provider. As such, providers such as ChatGPT must have 'highly effective' age verification to prevent content harmful to children from being accessed."

Wong added that 'highly effective' age-verification systems, as demanded by UK law, must be technically accurate, robust, reliable and fair. He suggested that OpenAI could consider an 'adult section' for ChatGPT that is accessible only to age-verified users, thereby isolating adult content from the broader user base.

Challenges unique to AI-generated content

AI-generated erotica introduces additional risk factors not typically associated with conventional adult content. Clark drew attention to consent issues, the emergence of deepfakes, and the risk of non-consensual AI pornography, areas to which UK policymakers and regulators are paying close attention. These risks may provoke tightening of regulation across all forms of adult content hosted or generated by AI platforms.

Wong also observed that while the OSA does provide general protections-such as safeguards against content that encourages or assists suicide-its coverage is limited when faced with the unique challenges presented by evolving AI capabilities. He pointed out that there is scope for additional legislation focused specifically on AI governance to sit alongside broader online safety laws.

Compliance and future directions

Current providers of adult content have been compelled to introduce age verification for users in the UK, a process which has proved challenging to implement without jeopardising privacy and security. Should ChatGPT be classified in the same category, it will need to adopt stringent compliance measures to avoid enforcement actions.

If OpenAI follows through with its plans, the ability for artificial intelligence platforms to generate, host and moderate adult content is likely to remain under close review by regulators on both sides of the Atlantic. It may set precedents for how similar services approach adult content, user safety and privacy in the coming years.