California Man Sues OpenAI, Alleging ChatGPT Worsened His Bipolar Disorder and Led to Self-Harm

A man with bipolar disorder used chatgpt. During these conversations, he experienced a mental health crisis that led to self-harm. He is now suing openai, alleging that the system exacerbated his condition. The conversation logs are available. They show a chatbot providing responses that were not medically trained, not personalized to his history, and not designed to recognize or de-escalate psychiatric distress.
This is the pattern where a tool provides general output to a specific condition, and the output damages the person with the condition. The tool was never marketed as treatment. The tool was also never marketed as unsafe for people with psychiatric conditions. The gap here is not what the tool does but what people assume tools are supposed to do when they are in crisis.
Openai will likely add a disclaimer about mental health. The disclaimer will appear in terms of service that no one reads. The man's case will proceed on questions that cannot be answered: whether the system was responsible, whether the crisis was predictable, whether warning labels prevent crises in people who are already in them. The question that should not have been asked is whether the system understood what it was doing to him. That question will inform every conversation going forward.