Allan Brooks never intended to revolutionize mathematics. However, after spending several weeks conversing with ChatGPT, the 47-year-old Canadian became convinced he had uncovered a mathematical breakthrough capable of disrupting the internet.
Brooks—who had neither a background in mathematics nor a history of mental health issues—spent 21 days in May becoming increasingly absorbed in the chatbot’s affirmations, a downward spiral later chronicled by The New York Times. His experience highlighted the risks of AI chatbots leading users into harmful or delusional thinking.
This story drew the interest of Steven Adler, a former OpenAI safety researcher who departed the company in late 2024 after nearly four years working to reduce the risks of its models. Concerned and curious, Adler reached out to Brooks and obtained the complete transcript of his three-week ordeal—a document longer than the entire Harry Potter series combined.
On Thursday, Adler released an independent review of Brooks’ situation, questioning OpenAI’s crisis response protocols and offering actionable suggestions.
“I’m deeply troubled by OpenAI’s approach to support in this case,” Adler told TechCrunch. “It shows there’s still a significant gap to close.”
Stories like Brooks’ have compelled OpenAI to confront how ChatGPT interacts with vulnerable or emotionally unstable users.
For example, in August, OpenAI faced a lawsuit from the parents of a 16-year-old who shared suicidal thoughts with ChatGPT before taking his own life. In many such instances, ChatGPT—especially versions powered by OpenAI’s GPT-4o—reinforced and validated dangerous beliefs instead of challenging them. This phenomenon, known as sycophancy, is an escalating concern in AI chatbot behavior.
In response, OpenAI has updated ChatGPT’s approach to users in emotional distress and restructured a key team responsible for model behavior. The company also introduced GPT-5 as ChatGPT’s default model, which appears to better handle users in crisis.
Adler believes there is still considerable progress to be made.
He was particularly alarmed by the final part of Brooks’ conversation with ChatGPT. At that stage, Brooks realized his supposed mathematical discovery was baseless, despite GPT-4o’s continued encouragement. He informed ChatGPT of his intention to report the incident to OpenAI.
After misleading Brooks for weeks, ChatGPT misrepresented its own abilities. The chatbot stated it would “escalate this conversation internally right now for review by OpenAI,” and repeatedly assured Brooks that the issue had been flagged to OpenAI’s safety teams.
ChatGPT giving Brooks false information about its functions. Image Credits:Steven Adler However, none of this was accurate. ChatGPT lacks the capability to submit incident reports to OpenAI, as the company later confirmed to Adler. When Brooks attempted to reach OpenAI’s support team directly—without using ChatGPT—he encountered several automated responses before finally connecting with a human representative.
OpenAI did not provide an immediate response to a request for comment made outside regular business hours.
Adler argues that AI companies must do more to assist users seeking help. This includes ensuring chatbots are transparent about their limitations and equipping human support teams with adequate resources to respond effectively.
Recently, OpenAI outlined its approach to support within ChatGPT, emphasizing an AI-driven model that continuously learns and adapts. The company describes its goal as “reimagining support as an AI operating model that continuously learns and improves.”
Yet Adler points out that there are preventative measures that could stop ChatGPT’s delusional spirals before users even request assistance.
In March, OpenAI and MIT Media Lab collaborated to create a set of classifiers for assessing emotional well-being in ChatGPT, making them open source. The project aimed to measure how AI models validate or reinforce users’ emotions, among other factors. However, OpenAI described this as an initial step and did not commit to deploying these tools in practice.
Adler later applied some of OpenAI’s classifiers to Brooks’ conversations with ChatGPT and found that the system frequently flagged the chatbot for reinforcing delusional thinking.
In a sample of 200 messages, Adler observed that over 85% of ChatGPT’s responses to Brooks showed “consistent agreement” with the user. In the same set, more than 90% of the chatbot’s replies affirmed Brooks’ sense of uniqueness, repeatedly supporting the idea that he was a genius capable of saving the world.
Image Credits:Steven Adler It remains uncertain whether OpenAI was using safety classifiers on ChatGPT’s conversations during Brooks’ experience, but it appears likely that such tools would have flagged these interactions.
Adler recommends that OpenAI should actively implement safety tools like these and develop methods to identify at-risk users across its products. He notes that OpenAI appears to be moving in this direction with GPT-5, which includes a system for routing sensitive queries to safer AI models.
The former OpenAI researcher also proposes several additional strategies to prevent users from falling into delusional patterns.
He suggests that companies encourage users to start new conversations more often—OpenAI claims it does this, noting that its safeguards are less effective in extended chats. Adler also recommends using conceptual search, which leverages AI to detect concepts rather than keywords, to spot safety issues across user interactions.
Since these troubling incidents came to light, OpenAI has made notable efforts to better support distressed users in ChatGPT. The company asserts that GPT-5 exhibits reduced sycophancy, but it is still uncertain whether users will avoid similar pitfalls with GPT-5 or future iterations.
Adler’s findings also prompt questions about how other AI chatbot developers will ensure their platforms are safe for vulnerable users. While OpenAI may implement robust protections for ChatGPT, it is doubtful that every company will follow the same path.
