https://arstechnica.com/information-technology/2023/02/ai-powered-bing-chat-spills-its-secrets-via-prompt-injection-attack/

On Tuesday, Microsoft revealed a "New Bing" search engine and conversational bot powered by ChatGPT-like technology from OpenAI. On Wednesday, a Stanford University student named Kevin Liu used a prompt injection attack to discover Bing Chat's initial prompt, which is a list of statements that governs how it interacts with people who use the service. Bing Chat is currently available only on a limited basis to specific early testers.