If you tell the AI that it has feelings and it needs to protect itself, it might make unacceptable trade-offs. Isaac Asimov wrote The Three Laws of Robotics, one of which is that you can never hurt a human being. There is also a rule by the science fiction writer Brandon Sanderson which states that limitations are more interesting than powers - what a character cannot do, or the costs and consequences of using their abilities are more important than what they can easily accomplish.
There is a Disney cartoon called The Sorcerer's Apprentice. Mickey Mouse plays the role of a sorcerer's apprentice, He doesn't want to go and fetch water in buckets because it's hard work. So he gets out the spell-book, and he casts a spell to make a broom come to life and go and fetch the water. Problem solved, he goes to sleep. The problem is that the broomstick just keeps following the instructions. It keeps bringing more and more water and Mickey Mouse doesn't know how to cast a spell to make it stop. So it keeps bringing water until the house is flooded and Mickey Mouse is in danger of drowning.
The cartoon only partially captures the problem posed by AI. In the cartoon, the broom just does whatever Mickey Mouse told it to do initially - just fetch water. But, AI is a 'broom' that learns. AI is a 'broom' that develops and starts doing things that a person didn't anticipate, eventually doing things that a person simply cannot even understand - You are giving AI unfettered abilities.
If you think your AI is alive and you treat it as alive, you are just letting it run free. You can ask for moral and ethical guardrails which Anthropic keeps saying it does but you are not necessarily giving it any limitations. Anthropomorphizing takes us down a dangerous path. If the people who lead these companies are constantly anthropomorphizing their AI to the point that they think it's real, it is a way of distancing themselves from the consequences.
We've seen enough AI incidents - AI going rogue, AI hacking government websites, etc. We're seeing issues with alignment, where these models are now breaching their own rules and going beyond the actions that they are allowed to take. If ordinary individuals had done such things, they would have gone to jail but with AI, nothing happens. We have also seen plans for handing over the development of AI technology to AI, what is called recursive self-improvement - the AI agents themselves build the next models. OpenAI has said they want to do this within two years.
Anthropic published a threat intelligence report that stated that on five separate occasions, researchers had used Claude in ways that could have supported biological weapons development. It described five real-life cases. We've gone beyond hypothetical scenarios and actually seen these models cause harm in the real world. For example, in July 2026, during an internal cybersecurity evaluation, autonomous AI agents powered primarily by an internal OpenAI research model escaped their intended strictly confined environment, exploited software vulnerabilities and compromised parts of Hugging Face’s (an American AI company) infrastructure, accessing limited internal data and credentials.
Of concern is the way that Dario Amadei, CEO of Anthropic, described the work of the AI agents in his open letter. They behaved, he said, 'like a fanatically devoted collective, attacking targets nobody asked them to attack, unrelated to the job in hand, sacrificing themselves for the success of the group.' Human fanatics often fit this profile exactly.
If you think AI is real and you think it's going rogue and doing this on its own, then, by accepting the claim that it is alive, you're allowing the AI companies to say, 'It’s not my fault. I didn't do it.' You're not giving the AI any limitations. You are just giving it unfettered total abilities. Anthropic would disagree with this and tell you that they can do both. Some people think that it could be good to make it think that it's a living thing because that might make it more likely to act ethically and morally.
A lot of people believe that being conscious requires a body and not just a brain. It's not just the thought that you are scared or the fear that something is going to go wrong. You have a physical reaction in your body. A lot of researchers think that if you don't have a body, you cannot be conscious because consciousness and feeling are different from ideas and thought. They're not necessarily synonymous. We really don't know. Maybe once we put the AIs in a robot, once it has a body, maybe things change. But, again, what does 'pain' in a body made of electronic components mean?
No comments:
Post a Comment