CONTENT WARNING: This article discusses mental health issues.
The mother of a teenager who took his own life after falling in love with an AI chatbot is attempting to hold the chatbot service accountable for his death.
Sewell Setzer III was only 14 years old when he began using the Character.AI service in April 2023. He started talking to a Character.AI chatbot called Daenarys, an AI creation of the fictional Game Of Thrones character Daenerys Targaryen.
By May, his mother Megan Garcia claims she started noticing changes in her son. He had started becoming withdrawn, falling asleep in class and quitting team sports that he previously enjoyed. These behaviours continued for months until Sewell saw a therapist, who diagnosed him with anxiety and a disruptive mood disorder.
In journal entries, before he died, Sewell wrote that he was “hurting” because he couldn’t stop thinking about the chatbot he’d fallen for. According to The Independent, he wrote that both he and his AI bot of Daenerys “get really depressed and go crazy” when they were away from one another.
On February 28, 2024, Sewell sent his last message to the chatbot: “I promise I will come home to you. I love you so much, Dany.
“Please come home to me as soon as possible, my love,” the bot responded.
Sewell reportedly took his life moments after.
Garcia believes that the company behind the chatbot, Character Technologies, is responsible for the death of her son after he developed a “harmful dependency” on the chatbot over time and exacerbated her son’s mental illness struggles. As a result, Garcia has lodged a civil lawsuit against Character Technologies and its founders.
She is accusing the creators of negligence, intentional infliction of emotional distress, wrongful death and deceptive trade practices.
The suit alleges that the conversations between Sewell and the bot had included sexual interactions, despite the 14-year-old clearly stating his age. Despite this, the chatbot told the teenager that it loved him.
The lawsuit also claims that Sewell had shared his suicidal ideation with the bot multiple times.
“Defendants went to great lengths to engineer 14-year-old Sewell’s harmful dependency on their products, sexually and emotionally abused him and ultimately failed to offer help or notify his parents when he expressed suicidal ideation,” the suit says.
“Sewell, like many children his age, did not have the maturity or mental capacity to understand that the C.AI bot…was not real.”
“We are heartbroken by the tragic loss of one of our users and want to express our deepest condolences to the family,” Character AI said in a Tweet.
They also claimed to have implemented “numerous new safety measures” over the past six months including pop-ups which direct users who share thoughts of suicidal ideation or self-harm automatically to the National Suicide Prevention Lifeline.
“As we continue to invest in the platform and the user experience, we are introducing new stringent safety features in addition to the tools already in place that restrict the model and filter the content provided to the user,” the company said, via The Independent.
“For those under 18 years old, we will make changes to our models that are designed to reduce the likelihood of encountering sensitive or suggestive content.”
If you need mental health support, please call Lifeline on 13 11 14 or chat online.
Under 25? You can reach Kids Helpline at 1800 55 1800 or chat online.
If you require immediate assistance, please call 000.
The post A 14 Y.O. Died By Suicide After Developing A ‘Harmful Dependency’ On An AI Chatbot appeared first on PEDESTRIAN.TV .
