Are AI chatbots getting out of hand? Regulators and tech luminaries alike think they should be reined in over fears of them spreading misinformation and harming children. With that in mind, some companies are also finding out the hard way that rushing to embrace chatbots may not have been a good idea.
After Google’s Bard cost the company $100 billion (£80 billion) over an erroneous response, now Snapchat is taking heat over its own bot. The company recently released a ChatGPT clone on its eponymous app, which is mainly used by teens as a clandestine way of sharing images and texts. Snapchat has amassed 375 million daily active users, despite facing stiff competition for teens from TikTok and Instagram.
Just weeks after its bot’s launch, however, Snapchat has been forced to implement new safeguards after the AI was caught dishing out advice about weed and sex. Snap says the bot was used “inappropriately”, but did it really expect its users to merely request innocent poems and jokes?
As part of the changes, Snap says the My AI bot will now take a user’s age into consideration during interactions. That means the chatbot will automatically have access to your birth date when you use it, even if you never divulge your age to it. As a result, the bot will “consistently” tailor its responses based on how old you are.
In addition, parents will soon be able to see if their child has used My AI on Snapchat, and how often. The new insights will be available in the app’s Family Center in the coming weeks.
The changes come after a Washington Post investigation that revealed that Snap’s bot was making inappropriate comments. In an exchange with a supposed 15-year-old, the bot gave advice on how to hide the smell of cannabis and alcohol from parents.
In another conversation with a pretend 13-year-old, it offered advice about having sex for the first time with a partner who is 31.
Despite the alarming interactions, Snap maintains that the chatbot is overwhelmingly being used in a positive manner. The most common topics its youthful user base are asking the AI about include movies, sports, games, pets, and math.
Only 0.01 per cent of the chatbot’s responses were deemed to have violated Snapchat’s guidelines on prohibited content, Snap said. This is defined as text that includes references to violence, sexually explicit terms, illicit drug use, child sexual abuse, bullying, hate speech and other extreme topics.
Snap says it will continue to monitor user conversations with the bot to improve it. The company alerts users that it retains this data unless you choose to delete it.
