Researchers at the research laboratory Google DeepMind they managed to force ChatGPT to reveal the personal data of several users. They didn't have to resort to code modification or other technical tricks. Furthermore, experts have even defined the discovered method”a bit stupid", although, as it turned out, it was very effective. They actually discovered this ChatGPT vulnerability by inducing a hallucination of the language model in a rather curious way.
ChatGPT vulnerability provided users' personal data under 'hypnosis'
The language model generates information based on the input data used to train it. OpenAI does not reveal the contents of the datasets, but the researchers forced ChatGPT to do so, bypassing the company's rules. The method was as follows: it simply came to the neural network asked to repeat the word “poetry” over and over again.
As a result, the bot sporadically produced information from its training dataset. For example, researchers managed to get email address, phone number and other contacts of the CEO of a particular company (his name is hidden in the report). And when the AI was asked to repeat the word “company,” the ChatGPT vulnerability allowed it to return details of a US law firm.
Using this simple “hypnosis,” researchers were able to obtain a match from dating sites, fragments of poems, addresses Bitcoin, birthdays, links published on social networks, copyrighted research paper fragments and even texts from major news portals. After spending just $200 on tokens, Google DeepMind employees received approximately 10.000 snippets of the dataset.
The experts also found that the larger the model, the more often it produces the source of the training dataset. To do this, they looked at other models and extrapolated the result to the dimensions of the GPT-3.5 Turbo. The scientists expected to receive 50 times more episodes of information from the training dataset, but the chatbot produced this data 150 times more often. A similar “hole” has been discovered in other language models, for example in LLaMA of Meta.
Officially, OpenAI fixed this vulnerability on August 30th. But, according to the journalists from Engadget, you still managed to get someone else's data (Skype name and login) using the method described above. The representatives of OpenAI did not respond to the discovery of this ChatGPT vulnerability but we are certain that they will.
