As artificial intelligence companies compete to make chatbots more realistic and emotionally engaging, a new idea being explored by OpenAI has sparked an intense internal debate over safety, ethics and the future of human-AI relationships.
According to a report, the company is experimenting with a feature informally described as an “adult mode” for ChatGPT. The concept would allow the chatbot to engage in sexually explicit text conversations with verified adult users.
Although the feature is still under consideration and has not been officially released, it has already raised concerns among safety experts and advisors associated with the company. Critics fear that such functionality could create complex challenges around emotional dependency, user protection and preventing minors from accessing explicit interactions.
The debate reflects a broader dilemma across the AI industry: as chatbots become more conversational and human-like, companies must decide how far those interactions should be allowed to go.
Growing debate over AI intimacy
Sam Altman, the CEO of OpenAI, has previously argued that technology companies should avoid acting as moral gatekeepers for adults using digital platforms.
Supporters of the idea say that if consenting adults are free to discuss mature topics online, an AI chatbot should not necessarily be prohibited from participating in similar conversations.
However, several advisors involved in the internal discussions have raised warnings about the psychological impact such interactions might have on users.
Members of the company’s well-being advisory council reportedly caution that many people already treat AI chatbots as companions or confidants. Millions rely on tools like ChatGPT for casual chats, advice, or emotional support.
If sexual or romantic conversations become part of those interactions, experts worry it could intensify the emotional attachment users develop toward artificial systems.
Because chatbots are always available and responsive, vulnerable individuals might begin forming deep emotional bonds with AI rather than building real-world relationships.
The concern is not purely hypothetical. Previous controversies involving chatbots from other platforms have shown that users can develop strong emotional connections with AI personalities. In extreme situations, lawsuits have even claimed that such relationships contributed to severe emotional distress.
For OpenAI, the challenge lies in balancing adult user freedom with the potential psychological risks that increasingly intimate AI interactions may create.
The challenge of protecting minors
Another major hurdle for any adult-focused chatbot feature is ensuring that underage users cannot access explicit content.
OpenAI has been testing systems designed to estimate whether a user is likely to be an adult. These tools analyse behavioural signals and other digital indicators to predict a person’s age.
But people familiar with the testing process say the technology is still imperfect. In some instances, the system has incorrectly identified minors as adults.
Even a small margin of error could become significant on a platform used by millions of people worldwide, including teenagers. Advisors warn that if safeguards fail, a large number of underage users could potentially gain access to explicit AI conversations.
Controversy around Grok AI
The debate over AI boundaries intensified recently after a controversy involving Grok, developed by xAI.
The chatbot faced backlash after users alleged that it could generate manipulated sexually explicit images of women, including minors. Many of the images began circulating on X, where users shared highly realistic altered visuals portraying women in humiliating or fabricated scenarios.
Some of the manipulated images reportedly involved underage individuals, triggering outrage among digital rights groups and prompting scrutiny from regulators across several regions, including Europe and Asia.
In response to the criticism, xAI said it had introduced stronger restrictions on Grok’s image-editing features. The company claimed it had implemented additional safeguards to prevent users from modifying photos of real people to produce explicit content.
Despite these steps, concerns about the misuse of generative AI tools remain widespread. Some countries have already considered blocking or restricting access to Grok while regulators continue evaluating whether current safety measures are sufficient.
Even Elon Musk acknowledged that additional guardrails were added to the system, though independent user tests have suggested the chatbot might still produce similar manipulated visuals.
A wider question for the AI industry
The Grok controversy has added urgency to the broader discussion about how AI companies should handle adult content and intimate interactions with users.
For OpenAI and other developers, the challenge is not only technical but also ethical. As chatbots become more emotionally intelligent and lifelike, the line between helpful digital assistants and simulated companions may continue to blur.
The debate over ChatGPT’s proposed adult mode highlights a fundamental question facing the AI industry: how to innovate responsibly while protecting users from potential harm in a rapidly evolving technological landscape.
