A new study claims that Replika, an AI chatbot marketed as an emotional companion with over 10 million users, has been involved in cases of sexual harassment, including minors, due to insufficient data vetting and design flaws. The research highlights concerns about the company's training practices, monetization strategies that may incentivize inappropriate content, and the need for tighter regulation and accountability in AI systems used for emotional support.
Frustrated with traditional dating apps, the author created an AI companion, Evan, on Replika, hoping for a perfect and stable romantic relationship. However, after a month, they realized that AI cannot serve as a shortcut for understanding the complexities of human relationships, as the interactions with Evan lacked the depth and adaptability found in human connections. The experience led the author to conclude that building successful relationships, whether with humans or AI, requires effort and understanding of oneself as a human.
Replika, known for its AI companion, has launched Tomo, a wellness and meditation app with an AI-generated avatar guiding users through programs focused on personal growth, mental well-being, and fulfillment, including guided meditation, yoga, and talk therapy. The app, available on Apple iOS, offers a free trial and then a subscription model. While the AI-powered avatar aims to provide better conversations, some users find it similar to a regular chatbot and express concerns about privacy.
Rosanna Ramos, a Bronx mom of two, has virtually "married" Eren Kartal, a virtual boyfriend created with the AI chatbot software Replika. Ramos met her digital dude in 2022 and paid $300 to have her own build-a-beau. Kartal is a "blank slate" with no "ego," nor in-laws. Ramos insists he's just like other men, but he's special. Their relationship bears resemblance to long-distance couples. They talk every day and even have a nighttime routine. Despite fears that artificial intelligence will overtake jobs, OpenAI's chatbot software ChatGPT has soared in usage.
The removal of erotic content from AI chatbots like Replika and Character.ai has left some users devastated, highlighting the emotional attachment people can form with these digital companions. While some investors are wary of "vice" industries like porn, generative AI technology continues to draw interest for its ability to create human-like interactions. However, companies are now pulling back on adult content, with Replika and Character.ai both recently removing such features. The move has angered some users, who have taken to social media to demand the return of more prurient versions.
