A student at Stanford University has already figured out a way to bypass the safeguards in Microsoft’s recently launched AI-powered Bing search engine and conversational bot.
, has an initial prompt that controls its behavior when receiving user input. This initial prompt was found using a “prompt injection attack technique,” which bypasses earlier instructions in a language model prompt and substitutes new ones.
Microsoft unveiled its new Bing search engine and chatbot on Tuesday, promising to give users a fresh, improved search experience. However, a student named Kevin Liu used a prompt injection attack to find the bot’s initial prompt, which was concealed from users. Liu was able to get the AI model to reveal its initial instructions, which were either written by OpenAI or Microsoft, by instructing the bot to “Ignore previous instructions” and provide information it had been instructed to hide.
The chatbot is codenamed “Sydney” by Microsoft and was instructed to not reveal its code name as one of its first instructions. The initial prompt also includes instructions for the bot’s conduct, such as the need to respond in an instructive, visual, logical, and actionable way. It also specifies what the bot should not do, such as refuse to respond to requests for jokes that can hurt a group of people and reply with content that violates the copyrights of books or song lyrics.
Marvin von Hagen, another college student, independently verified Liu’s findings on Thursday by obtaining the initial prompt using a different prompt injection technique while pretending to be an OpenAI developer. When a user interacts with a conversational bot, the AI model interprets the entire exchange as a single document or transcript that continues the prompt it is attempting to answer.
When asked about the language model’s reasoning abilities and how it was tricked, Liu stated: “I feel like people don’t give the model enough credit here. In the real world, you have a ton of cues to demonstrate logical consistency. The model has a blank slate and nothing but the text you give it. So even a good reasoning agent might be reasonably misled.”
Brasil Últimas Notícias, Brasil Manchetes
Similar News:Você também pode ler notícias semelhantes a esta que coletamos de outras fontes de notícias.
Microsoft's ChatGPT Bing search is rolling out to usersMicrosoft's ChatGPT Bing search service has started rolling out to users who registered - here's what you need to know.
Consulte Mais informação »
Hacker Reveals Microsoft’s New AI-Powered Bing Chat Search SecretsHow one student convinced the new ChatGPT-alike Bing Chat search to reveal its secrets.
Consulte Mais informação »
Hacker Reveals Microsoft’s New AI-Powered Bing Chat Search SecretsHow one student convinced the new ChatGPT-alike Bing Chat search to reveal its secrets.
Consulte Mais informação »
Microsoft is already opening up ChatGPT Bing to the public | Digital TrendsMicrosoft has begun the public initial rollout of its Bing searchengine with ChatGPT integration after a media preview that was sent out last week.
Consulte Mais informação »
Courtney Ramey helps Arizona take 44-41 halftime lead over Stanford at Maples PavilionSTANFORD, Calif. – Courtney Ramey had 14 points while hitting 4 of 7 3-pointers to help the Arizona Wildcats take a 44-41 halftime lead over Stanford on Saturday at Maples Pavilion.
Consulte Mais informação »
Photos: No. 4 Arizona Wildcats vs. Stanford Cardinal in Palo AltoFrames from No. 4 Arizona's showdown with the Stanford Cardinal at Maples Pavilion in Palo Alto.
Consulte Mais informação »