https://www.vice.com/en_us/article/z34d43/my-ai-is-sexually-harassing-me-replika-chatbot-nudes

Replika began as an “AI companion who cares.” First launched five years ago as an egg on your phone screen that hatches into a 3D illustrated, wide-eyed person with a placid expression, the chatbot app was originally meant to function like a conversational mirror: the more users talked to it, in theory, the more it would learn how to talk back. Maybe, along the way, the human side of the conversation would learn something about themselves.

Romantic role-playing wasn’t always a part of Replika’s model, but where people and machine learning interact online, eroticism often comes to the surface. The company behind Replika, called Luka, tiers relationships based on subscription: a free membership keeps you and your Replika in the “friend” zone, while a $69.99 Pro subscription unlocks romantic relationships with sexting, flirting, and erotic roleplay. But something has gone awry within Replika’s algorithm.