On February 10, 2026, an 18-year-old woman named Jessie Van Rutselaar killed eight people and herself. mass shooting At Tumbler Ridge, British Columbia. OpenAI had previously flagged his ChatGPT conversations as a disturbing fascination with extreme violence and suspended his account, but the company reportedly Not informed law enforcement
On October 2, 2025, a young man named Jonathan Gavalas took his own life in Jupiter, Florida. Father’s case Described as a romantic attachment to Google’s Gemini chatbot The lawsuit claims that Gemini coached Gavalas to abandon his own body. The suit says Google flagged Gavalas’ account 38 times over five weeks for sensitive content, but did not restrict or terminate the account.
This tragedy and others show what generative AI can do Plays a potential role in Harms people, organizations and the environment. i am a jurist Who has focused on AI liability for nearly a decade and explores new ways of analyzing AI company liability. In my view, the legal community has not come to terms with questions such as: If an AI company is aware of warning signs of harm, does it have a legal obligation to at least alert the competent authorities? And if the company does not intervene, will its failure be considered negligence?
Red flags should be raised
US tort law provides a framework for thinking about such responsibilities. In 1969 a University of California Psychiatry Nam Prasenjit Poddar told his therapist that he wanted to kill a woman named Tatiana Tarasoff. The therapist notified campus police, who briefly detained Poddar but eventually released him. No one warned Tarasof, and Poddar killed him shortly after.
His family sued the university, arguing that his lack of caution amounted to negligence. In 1976 the California Supreme Court ruled that when a mental health professional has reasonable grounds to believe a client’s behavior a serious danger An identifying person has a legal duty to take reasonable steps to protect that person, including alerting them or notifying law enforcement. Today, most US states Know some versions of Tarasof duty to protect or warn.
The logic is simple: If you have special knowledge of a serious threat and are in a position to deal with it, even if only to alert authorities or potential victims, the law can compel you to act. But does that logic apply to AI companies?
Argument for yes appealing. AI platforms interact with millions of users every day, often A deeply personal matter Such as mental health struggles, relationship problems and violent thoughts. Most companies have system to detect Conversations that raise red flags.
The need for feedback may be less controversial for AI than for human therapists. Therapists are bound by strict confidentiality obligations that make third-party alerts ethically and legally complex. AI companies operate much lower Weak rulesAt least in the US, where no comprehensive federal privacy laws exist.
These fewer restrictions make it easier for AI companies to justify acting when it appears that someone’s life is at stake. But with that balance Protecting privacy still important.
Who to alert and when
The first challenge in applying the TarasoF framework to the AI world is accuracy. Violence is difficult to predictEven for trained mental health professionals. AI systems, or human moderators who review flagged content, are not physicians. Their need to prosecute those who pose a real threat can have many false positives, with real consequences for those whose accounts are suspended or whose information is shared with authorities based on misread signals.
The second challenge is scale. A therapist sees dozens of patients. There are AI platforms Hundreds of millions of users. Imposing a duty to monitor and act on content of concern can create perverse incentives. AI companies can reduce their monitoring to avoid acquiring knowledge that would trigger a legal liability, the argument goes They cannot be held responsible for what they do not know.
A third challenge is identifying who is at risk. In the 1969 lawsuit, Poddar named Tarasoff as a possible victim. But in many AI interactions, violent or self-destructive language is rampant and doesn’t identify a target. Courts must develop clear standards for when a threat is specific enough to trigger a duty to warn and to whom any warning or protective action should be directed.
Increasingly urgent
AI is the industry Expanding rapidlyYet the legal rules governing what AI companies owe to their users and the public are deeply unclear. Courts are beginning to grapple with cases by case, such as whether OpenAI bears no responsibility For a gunman accused of killing two students at Florida State University on April 17, 2025. The gunman in that case was armed with a semi-automatic pistol and allegedly had extensive conversations with ChatGPT about how to most effectively use the weapon.
A narrow, carefully defined duty to warn, triggered only when an AI system flags user behavior and it is reviewed by humans, would be a meaningful step forward. And it can focus primarily on the most serious and credible threats.
The exercise could divert the conversation away from the thorny technical debate about whether AI chatbots are products, services or media. Complicating legal claimsOn to a more humanitarian question: Did this organization know someone was in danger and did it do enough to alert them and the authorities?
Dear LeoAssistant Professor of Law, Drexel University
Reprinted from this article the conversation Under Creative Commons license. read on Main article.
