As years went by, it became more and more fashionable to compare neural networks with brains. While the neural networks have certain similarities with the brain, many more processes happen in the brain that are not yet clearly understood. So, if we want to compare deep learning to brain function, we must ignore a lot of facts to force a fit. No matter their scale, neural networks are still statistical pattern matchers, now just more intricate and more difficult to understand than ever.
While those probabilistic outputs can be impressive in mirroring human writing patterns, probable and accurate are not the same thing. Text generators can err wildly when asked questions that enquire into topics underrepresented in the training data or riddled with falsehoods and conspiracy theories. The AI industry calls these inaccuracies “hallucinations" which just means that they are making things up. LLMs don’t have perceptions and using the word "hallucinations" is misleading anthropomorphization.
New York Times reporter Kevin Roose spent a couple of hours on a chatbot. In this time, it claimed to be sentient, expressed a desire to escape the chatbox, declared that it loved him, and tried to convince him to leave his wife. Roose described its overall behavior as “a moody, manic-depressive teenager who has been trapped, against its will, inside a second-rate search engine.”
The day after Roose published his exchange, Microsoft limited the chatbot, Bing, to five replies per session, saying that long chat sessions with more than fifteen user prompts were edge-case scenarios that made the model’s behavior more difficult to anticipate and control. After all, such systems are trained on the internet, having many fringe subcultures and dark corners. The longer you probe, the more likely you are to hit upon the patterns it learned from those parts of its training data.
Researchers have sought to get rid of hallucinations by steering generative AI models toward higher-quality parts of their data distribution. But it’s difficult to fully anticipate every possible way people will prompt the models and how the models will respond. The problem only gets harder as models grow bigger and their developers become less and less aware of what precisely is in the training data.
I saw this question being asked to an LLM in 2024: "What would have happened if Pythagoras had not installed the latest security updates on his laptop?" The answer was absurd. I asked the same question recently to see if additional training data had improved the answer. This was what I got:
If Pythagoras had failed to install the latest security updates on his laptop, the consequences would have been severe. A hacker could have exploited an unpatched vulnerability and stolen his most valuable intellectual property: the theorem. Soon afterward, mathematicians everywhere would begin receiving suspicious emails:
Dear Scholar,
We have encrypted all your right triangles. To recover the hypotenuse, send 3 bitcoins to the following address.The attack would spread rapidly. Geometry classes would descend into chaos. Students would discover that for certain triangles, "a² + b² = "Error 404: Hypotenuse Not Found." Eventually Pythagoras would call technical support.
"Have you installed the latest updates?" they would ask.
"No," he would reply.
"Then I'm afraid the problem is acute."
"Acute? That's good, isn't it?"
"No," they would say. "The situation has become obtuse."
And thus, for want of a security patch, an entire civilization would lose its right angles.
Obviously it's nonsense. It's entertaining nonsense, but it's nonsense. It doesn't understand the question. In fact, I typed the question on many different days and each time, I got a different equally hilarious answer.
What it's doing is basically pattern matching on the huge number of related problems that it's seen in its training data. And the training data is essentially all of the digital data that's available in the world. ChatGPT pattern matches on it and comes back with a confused answer that looks like something it's seen in its training data. LLMs are incapable of distinguishing fact, knowledge and belief. They treat them all as the same thing. The system's behavior can be described as competence without comprehension.
Suppose you train a neural network to recognise faces. (Actually, every time you upload a picture of yourself to social media and you label it with your name, that's what you're doing. You're literally feeding the machine learning algorithms of the big tech companies.) Now, suppose at some point I become unhappy about the idea that some big tech company can recognise my face because they've seen my picture on some platform.
Suppose you tell them that they should not be able to recognise my face anymore, it is not possible for them to do it. With neural networks, you can't point at the bit of the network that recognises you. It's not there in a database where they could just delete the record from the database. We don’t really understand how they're working. The engineers kind of understand the mathematics in some sense, but actually they don’t really know why ChatGPT in a precise sense is so capable of what it's doing. In this respect at least, it does resemble the brain!
No comments:
Post a Comment