Privacy group NOYB has launched a complaint against OpenAI that highlights a massive problem at the heart of LLMs: ChatGPT holds personal data on millions of people, but its creators are unable to extract or edit that data. Not only that, ChatGPT is prone to making up personal information that is plainly false.
The underlying problem is that, although LLMs obviously contain vast amounts of information, it isn’t held as structured data. In fact, the specifics of how information is stored are largely mysterious, rather like with human memory. There is no index of records that one can look up like in a database. Their memories are fluid and creative (also like human memory). This is why fixing hallucinations is so difficult. There is no simple fix.
How this case plays out will be extremely interesting.