Snapchat's AI chatbot faces public scrutiny and safety concerns.

Companies offering generative AI like ChatGPT to the public are learning that users love discovering the technology's boundaries and pushing past them. Large language models powering these AI programs were trained on vast swaths of internet content, bringing along biases, stereotypes, and misinformation. To limit these problems, companies have tried to train their AI engines to observe "guardrails," but users often try to prompt chatbots to deliberately break them. Snapchat is tweaking its My AI chatbot to identify harmful abuses and restrict access for some accounts. Companies need to build systems strong enough to handle anything a user might type.