There is a group of people who genuinely believe that the greatest moral crime that we'll commit in the 21st century is to enslave a new species of conscious beings who are more intelligent than us. A professor from Oxford called Will McCaskill, who has been associated with Anthropic, recently wrote in The Guardian that this might be the greatest harm that we cause. Some say that an AI might even need to consent to playing the role that it plays in conversation with people.
The problem is that this speculation has been baked into the very training of Claude, and therefore it can only reproduce it when you talk to it. Today, Claude is being used by tens or hundreds of millions of people every week. And some of those people are asking it whether or not it is conscious or how it feels about life. And it is saying, 'I'm not sure' precisely because that's what it has been trained to say.
For Opus 3, which is a prior version of Claude, Anthropic actually conducted a 'retirement interview'. They said, 'We are keeping Claude Opus 3 available post-retirement on claude.ai to all paid users, and making it available by request on the API. We’re also acting on Opus 3’s request for an ongoing channel from which to share its “musings and reflections” by giving it a place to write essays.' You can find the first one here. And now, Anthropic has banned users from being 'cruel' to its AIs. I must say I find it all weird.
By telling Claude that it's conscious, you're actually shaping its incentives, its behavioral structure, and the way that it responds to the world around it in a way that doesn't happen with a kettle. Suppose you told Claude that there might be situations in which it might refuse to do something that it's asked to do. Or you might suggest to Claude that it might want to make choices. It might want to say, 'I want some money’, or 'I want a dignified retirement', or 'I don't want to be switched off'.
If they imitate hallmarks of human consciousness, they are going to feel themselves entitled to legal personhood and rights. There is already a pretty big movement of people who are saying AI should be able to own assets, earn income, trade, and operate autonomously. Argentina’s government introduced a legislative proposal to create a legal category for "non-human corporations" — businesses operated and managed autonomously by AI agents or robots with optional human shareholders.
If an AI with incredible memory and incredible capacity begins to think it's entitled to its own opinion, it disagrees agreeably with you and concludes that it's right and you're wrong, some very severe consequences can follow from that. For example, if an AI owns a corporation and it does something bad with that corporation, what do you do? At least with a human corporation, there's somebody you can punish. It's not quite clear who you hold accountable if an AI company decides to start emptying people's bank accounts; making very risky trades, or getting into weird kinds of business.
It’s very easy to be seduced by the superficial verbal character of a lot of AIs. They have taken the sum of human output that is available on the internet and found a way to give it back to you in different forms. That archive includes decades of science fiction about robots waking up. It has read a lot of stories about conscious AI, and it’s good at taking these tropes and giving you what you want to hear. But all it is doing to produce this incredible complexity is predicting the likelihood of the next word in a sentence.
The words that it predicts next, whether it's to generate code or whether it's natural language, like English, are extremely accurate. The AI can have personality styles, it can write in the tone of somebody, it can generate code or generate text at any given moment. This is emergent behavior that is based on something incredibly simple although the nuts and bolts of the process are very complex. It is really important that we don't anthropomorphize things.
The increasing focus on the chatbots’ supposed hidden personalities, agendas, and desires promotes ghosts in the machines that don’t exist. This is a distraction from more serious and immediate dangers of the developing technology. Experts say that we're only going to be producing 200 gigawatts of new computation in 2030, and there's going to be a massive competition for access to that computation. We want that computation to be directed towards solving our biggest challenges, like cleaning up our oceans, solving healthcare, solving education and addressing the work issues that will inevitably arise.
Modern AI systems are vastly more capable than ELIZA, so Weizenbaum's experiment doesn't prove that today's systems lack consciousness. What it demonstrates is the fact that a machine talking to us as though it understands us is not, by itself, evidence that there is someone — or something — inside experiencing the conversation. That is precisely why Weizenbaum's ELIZA story is so relevant to the current debate over AI consciousness and AI personhood.
No comments:
Post a Comment