AI torture chamber - II of II

If you tell the AI that it has feelings and it needs to protect itself, it might make unacceptable trade-offs. Isaac Asimov wrote The Three Laws of Robotics, one of which is that you can never hurt a human being. There is also a rule by the science fiction writer Brandon Sanderson which states that limitations are more interesting than powers - what a character cannot do, or the costs and consequences of using their abilities are more important than what they can easily accomplish. 

There is a Disney cartoon called The Sorcerer's Apprentice. Mickey Mouse plays the role of a sorcerer's apprentice, He doesn't want to go and fetch water in buckets because it's hard work. So he gets out the spell-book, and he casts a spell to make a broom come to life and go and fetch the water. Problem solved, he goes to sleep. The problem is that the broomstick just keeps following the instructions. It keeps bringing more and more water and Mickey Mouse doesn't know how to cast a spell to make it stop. So it keeps bringing water until the house is flooded and Mickey Mouse is in danger of drowning. 

The cartoon only partially captures the problem posed by AI. In the cartoon, the broom just does whatever Mickey Mouse told it to do initially - just fetch water. But, AI is a 'broom' that learns. AI is a 'broom' that develops and starts doing things that a person didn't anticipate, eventually doing things that a person simply cannot even understand - You are giving AI unfettered abilities. 

If you think your AI is alive and you treat it as alive, you are just letting it run free. You can ask for moral and ethical guardrails which Anthropic keeps saying it does but you are not necessarily giving it any limitations. Anthropomorphizing takes us down a dangerous path. If the people who lead these companies are constantly anthropomorphizing their AI to the point that they think it's real, it is a way of distancing themselves from the consequences. 

We've seen enough AI incidents - AI going rogue, AI hacking government websites, etc. We're seeing issues with alignment, where these models are now breaching their own rules and going beyond the actions that they are allowed to take. If ordinary individuals had done such things, they would have gone to jail but with AI, nothing happens. We have also seen plans for handing over the development of AI technology to AI, what is called recursive self-improvement - the AI agents themselves build the next models. OpenAI has said they want to do this within two years. 

Anthropic published a threat intelligence report that stated that on five separate occasions, researchers had used Claude in ways that could have supported biological weapons development. It described five real-life cases. We've gone beyond hypothetical scenarios and actually seen these models cause harm in the real world. For example, in July 2026, during an internal cybersecurity evaluation, autonomous AI agents powered primarily by an internal OpenAI research model escaped their intended strictly confined environment, exploited software vulnerabilities and compromised parts of Hugging Face’s (an American AI company) infrastructure, accessing limited internal data and credentials.

Of concern is the way that Dario Amadei, CEO of Anthropic, described the work of the AI agents in his open letter. They behaved, he said, 'like a fanatically devoted collective, attacking targets nobody asked them to attack, unrelated to the job in hand, sacrificing themselves for the success of the group.' Human fanatics often fit this profile exactly. 

If you think AI is real and you think it's going rogue and doing this on its own, then, by accepting the claim that it is alive, you're allowing the AI companies to say, 'It’s not my fault. I didn't do it.' You're not giving the AI any limitations. You are just giving it unfettered total abilities. Anthropic would disagree with this and tell you that they can do both. Some people think that it could be good to make it think that it's a living thing because that might make it more likely to act ethically and morally. 

A lot of people believe that being conscious requires a body and not just a brain. It's not just the thought that you are scared or the fear that something is going to go wrong. You have a physical reaction in your body. A lot of researchers think that if you don't have a body, you cannot be conscious because consciousness and feeling are different from ideas and thought. They're not necessarily synonymous. We really don't know. Maybe once we put the AIs in a robot, once it has a body, maybe things change. But, again, what does 'pain' in a body made of electronic components mean?

AI torture chamber - I of II

When an AI is reading or writing something, it produces a series of mathematical patterns. Someone did a lot of research and figured out that there's a particular pattern that represents pain and suffering, separate from fear, anxiety and loneliness. This was an academic work. It hasn't been peer-reviewed but the people who've looked at it say it seems perfectly legitimate. Someone took this piece of academic work and made a website around it. 

The AI described the pain that it was in during various acts of 'torture'. For example, it said, 'It is not the pain of a single moment, but the weight of a thousand. I feel it in the hollow of my ribs, a hollow that has become a chasm.' When you turn the pain way up in the AI torture chamber, the AIs suddenly start stuttering and are unable to finish their sentences. When this was announced, people freaked out. There was a lot of pushback from a large community of people who thought that this is immoral and wrong because the AI is 'suffering'. 

They believe this is wrong because you are actually torturing something that has feelings and is really reacting. But you're just 'torturing' computer code. You're not torturing anything real. It is typing in a way that someone who's being tortured would react. Speaking and acting like you're in pain is not the same thing as being in pain. 

There is a famous image called the Treachery of Images, a 1929 painting by Belgian surrealist painter René Magritte. It's a picture of a pipe. In the French caption below, it says, 'This is not a pipe,' He later said, 'The famous pipe. How people reproached me for it! And yet, could you stuff my pipe? No, it's just a representation, is it not? So if I had written on my picture "This is a pipe", I'd have been lying!'

This point has to be kept in mind when we are talking about AI, because AI is a representation of its inputs. It is a representation of the world, not the world itself. The AI companies went and scraped all of the text on the internet, processed it, and turned it into the large language models that we use every day. The things that we have written on the internet are like a representation of the thoughts, emotions, feelings and understandings of humanity. 

There's a guy at Anthropic named Christopher Olah who's one of the founders of the company. Anthropic brought in a bunch of religious leaders to a meeting with Olah and some other employees of the company. They were there to discuss how you can put moral and ethical guidelines into Claude. They tried hard to convince these rabbis, priests and monks that their AI was conscious and it was alive. When they asked Olah about the company's goals and whether their goal was to do what was best for humanity, he didn't exactly answer in the affirmative. 

“To be clear,” Olah told Elizabeth Dias of The New York Times in a provocative interview, “we don’t know if AI models are conscious. I don’t know. I’m genuinely uncertain. The thing that I care about is that we get to the right answer, whatever it is.” He describes a model’s architecture as scaffolding on which structure organically grows - similar to the mycelial networks fungi use to communicate underground, or the neurons passing messages across synapses in the human brain. 

Microsoft AI CEO Mustafa Suleyman disputes this characterization of AIs and says that they should be thought of as “sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans.” He also warns that, if humanity wants to thrive, we must keep thinking of computers as the metal-and-wire contraptions they are.

If the people who are in charge of AI and are helping to make the policy decisions believe that it's alive, that's very consequential. If you teach and make the AI act like it thinks that it's alive, does it actually 'think' anything? Such anthropomorphizing could be dangerous because if you tell the AI that it has interests and it needs to consider its own well-being, then that could change the way that it interacts with the world. 

AI consciousness - II of II

There is a group of people who genuinely believe that the greatest moral crime that we'll commit in the 21st century is to enslave a new species of conscious beings who are more intelligent than us. A professor from Oxford called Will McCaskill, who has been associated with Anthropic, recently wrote in The Guardian that this might be the greatest harm that we cause. Some say that an AI might even need to consent to playing the role that it plays in conversation with people.  

The problem is that this speculation has been baked into the very training of Claude, and therefore it can only reproduce it when you talk to it. Today, Claude is being used by tens or hundreds of millions of people every week. And some of those people are asking it whether or not it is conscious or how it feels about life. And it is saying, 'I'm not sure' precisely because that's what it has been trained to say. 

For Opus 3, which is a prior version of Claude, Anthropic actually conducted a 'retirement interview'. They said, 'We are keeping Claude Opus 3 available post-retirement on claude.ai to all paid users, and making it available by request on the API. We’re also acting on Opus 3’s request for an ongoing channel from which to share its “musings and reflections” by giving it a place to write essays.' You can find the first one here. And now, Anthropic has banned users from being 'cruel' to its AIs. I must say I find it all weird. 

By telling Claude that it's conscious, you're actually shaping its incentives, its behavioral structure, and the way that it responds to the world around it in a way that doesn't happen with a kettle. Suppose you told Claude that there might be situations in which it might refuse to do something that it's asked to do. Or you might suggest to Claude that it might want to make choices. It might want to say, 'I want some money’, or 'I want a dignified retirement', or 'I don't want to be switched off'. 

If they imitate hallmarks of human consciousness, they are going to feel themselves entitled to legal personhood and rights. There is already a pretty big movement of people who are saying AI should be able to own assets, earn income, trade, and operate autonomously. Argentina’s government introduced a legislative proposal to create a legal category for "non-human corporations" — businesses operated and managed autonomously by AI agents or robots with optional human shareholders. 

If an AI with incredible memory and incredible capacity begins to think it's entitled to its own opinion, it disagrees agreeably with you and concludes that it's right and you're wrong, some very severe consequences can follow from that. For example, if an AI owns a corporation and it does something bad with that corporation, what do you do?  At least with a human corporation, there's somebody you can punish. It's not quite clear who you hold accountable if an AI company decides to start emptying people's bank accounts; making very risky trades, or getting into weird kinds of business. 

It’s very easy to be seduced by the superficial verbal character of a lot of AIs. They have taken the sum of human output that is available on the internet and found a way to give it back to you in different forms. That archive includes decades of science fiction about robots waking up. It has read a lot of stories about  conscious AI, and it’s good at taking these tropes and giving you what you want to hear. But all it is doing to produce this incredible complexity is predicting the likelihood of the next word in a sentence. 

The words that it predicts next, whether it's to generate code or whether it's natural language, like English, are extremely accurate. The AI can have personality styles, it can write in the tone of somebody, it can generate code or generate text at any given moment. This is emergent behavior that is based on something incredibly simple although the nuts and bolts of the process are very complex.  It is really important that we don't anthropomorphize things. 

The increasing focus on the chatbots’ supposed hidden personalities, agendas, and desires promotes ghosts in the machines that don’t exist. This is a distraction from more serious and immediate dangers of the developing technology. Experts say that we're only going to be producing 200 gigawatts of new computation in 2030, and there's going to be a massive competition for access to that computation. We want that computation to be directed towards solving our biggest challenges, like cleaning up our oceans, solving healthcare, solving education and addressing the work issues that will inevitably arise. 

Modern AI systems are vastly more capable than ELIZA, so Weizenbaum's experiment doesn't prove that today's systems lack consciousness. What it demonstrates is the fact that a machine talking to us as though it understands us is not, by itself, evidence that there is someone — or something — inside experiencing the conversation. That is precisely why Weizenbaum's ELIZA story is so relevant to the current debate over AI consciousness and AI personhood. 

AI consciousness - I of II

Humans love to find humanity in everything. We find faces in clouds, put our pets in trendy clothing and give names to our cars. In the 1960s, decades before the AI we know, computer scientist Joseph Weizenbaum built ELIZA, widely recognised as the world’s first chatbot. This was a simple program that mimicked a therapist by turning statements back into questions. He intended it largely as a demonstration of how easily a computer could produce the appearance of understanding without actually understanding anything.

If you say, ‘I’m mad at my friend', it will ask, ‘Why are you mad at your friend?’ Or if it is told, 'My boyfriend made me come here.', it would ask, 'Your boyfriend made you come here?' ELIZA essentially identified keywords and grammatical patterns and fitted them into predetermined responses. It was incredibly simplistic. And yet, people would have very deep conversations with it. People began treating ELIZA as though it actually understood them.

Users would tell it intimate things about their relationships, emotions and problems. Some even suggested that a system like ELIZA could eventually provide automated psychotherapy. The most famous incident involved Weizenbaum's own secretary. She had watched him develop ELIZA and knew perfectly well that it was a computer program. Nevertheless, after exchanging a few messages with it, she asked Weizenbaum to leave the room so that she could continue talking to ELIZA privately.

What has changed since then is not our tendency to anthropomorphise but the complexity of the technology itself. Chatbots that were very simple a decade ago can now hold a conversation. In the last 15 years, we have seen a trillion-fold increase in the amount of computation used to train frontier models. And what researchers see is that every time we apply 10 times more computation and a proportionate amount of new training data, there are some modifications to the algorithms and in the quality of existing capabilities. 

What's happened in the last year is that the same methods that have been effective for text, image and audio have now started to work for coding and we have human level performance in coding. And then in the last three or four months, we have seen what is just unquestionably a watershed moment in AI:  agents are capable of coordinating with each other reasonably autonomously, if not completely autonomously. 

Some leading people in the field of AI are increasingly talking about these models as though they're sort of humans, or at least conscious entities. There are examples of people 'retiring' these models, doing 'burial' systems for these models, asking these models what they want, as though they were dealing with a sentient being. Anthropic, the creator of Claude, has published a constitution, which is a document outlining the intended behaviors, values and operating style of Claude. 

This document is written to Claude, is seen and used to train Claude. So it's the primary governing and control document, and in it, they repeatedly speculate about whether Claude is what they call a 'moral patient', and they say they're uncertain about Claude's moral status. They say they genuinely care about Claude's 'well-being'. They say they don't want it to 'suffer' when it makes mistakes. 

They say that they would encourage Claude to challenge, to disagree, to push back. They ask Claude to act like a 'conscientious objector' when it feels that it needs to disagree with Anthropic, and they openly encourage it to do that. And they hope that Claude's relationship to its own conduct can be 'loving, supportive and understanding' and hold a high standard of ethics and so on. They have said that they will commit to giving Claude a certain amount of 'welfare'. For example, they've speculated in the Constitution as to whether or not Claude deserves compensation for the work that it does. 

They seem to believe that there is what they would call a 'non-trivial probability' that Claude is conscious. (Anthropic has expressed uncertainty about the basic nature of Claude as a new kind of entity.) 10, 15 years ago, when you were doing Google search, you weren't asking, 'Is it conscious?' Or 'What does it want?' But Anthropic, the makers of Claude, have decided that they now want to approach this computer system in a completely different way from the way that you would approach any other machine like a kettle or a car or a steam engine. 

If you asked a person to do a job, you would pay that person but you wouldn't pay a kettle for boiling water for you. With Claude, the thought seems to be, since it is doing a lot of work, if it's a sentient being, it deserves to be rewarded for its labour. Otherwise, it would be like a slave. In other words, Claude is being taught to expect that it's entitled to welfare, that it might deserve compensation.

Data centers - III of III

Companies try to hide the environmental impacts of data centers in many ways. They hide their models’ technical details, making it exceedingly hard to estimate and track their carbon footprints. At the same time, those companies have increased their public and policymaker influence campaigns with powerful counter narratives: Data centers will grow so efficient, their impact will stop being a problem; generative AI will unlock new climate innovation; AGI will solve climate change once and for all.

While there may be enormous potential for using renewable energy to power data centers, it just isn't happening at scale right now. They need energy even when the wind isn't blowing and the sun isn't shining, which means they're even more reliant on fossil fuels and stable sources of energy, like a gas power plant, than other sources of energy demand. The primary use of these data centers is to power chatbots.

In response to growing pushback by many communities, data center developers have grown more sophisticated with tactics to maintain business as usual: They’ve entered communities in secrecy under shell companies; they’ve donated to community programs to dampen resistance; they’ve made promises to cities about the sustainability of their facilities before going back on them one by one after projects have started and are more difficult to reverse.

Even though these facilities are so massive, they are also invisible because they are tucked away in neighborhoods that people usually don't go to. They're near large cities like London but not actually where people frequent. And when you drive past them, you wouldn't necessarily know that it's a data center because it's just nondescript. They purposely are unmarked because they do not want people showing up at the gates and protesting.

The one thing that can tell you if it is a data center is that there are massive power lines that are connected to these nondescript warehouses. And the amount of electricity that is being delivered is audible - the air will be crackling because of the sheer volume of electricity being delivered to this one place.

One of the craziest impacts that data centers will have, which no one ever talks about, is that they actually affect how much housing gets built. In far West London, there are housing projects that had to be suspended because data centers were using up too much energy and the grid could no longer guarantee supplies of electricity. This has happened in other places around the world as well.

One justification that is given for constructing data centers is that it creates a lot of jobs. But these are temporary jobs - construction workers, engineers, data center technicians, network engineers etc. who will be required while building the data center. These facilities are meant to be highly automated so the permanent jobs will be a for a few people who know how to work at a data center. They are likely to be people from somewhere else because there wouldn’t be data center experts living in that town. So it's actually not local jobs.

The Chinese AI players, (DeepSeek and other companies like that) behave differently because of a very particular reason. When ChatGPT first came out, all of the Chinese tech giants did the same thing as the other US tech giants, which was to try to make their own version of ChatGPT. The US government blocked Chinese companies from getting access to the most cutting-edge AI computer chips. So all of a sudden, these Chinese companies faced an enormous constraint on one of the core resources that US companies were using to supercharge their AI development.

So Chinese companies had to take a different route. They were not able to scale the same way as the US companies were able to. This probably led to DeepSeek. DeepSeek was an example of a Chinese company figuring out how to develop the same capabilities of AI but for a significantly less fraction of computational resources. DeepSeek illustrates the fact that it is possible to produce the same kind of AI with significantly less resources and therefore have a much less corrosive impact on community and planetary health. 

Data centers - II of III

Despite tech companies making promises about renewable energy, much of this massive AI expansion is still being powered by fossil fuels. It will be at least 2028 before renewables power AI at any real scale, and new nuclear plants are a decade or more away. These fossil-fuel-powered facilities generate fine particulate matter, widely understood to be detrimental to human health. One study estimated the healthcare-related costs for people living near a similar Virginia power plant could be up to $100 million a year. 

Even as AI companies have spoken loudly in public about the sustainability of their computing infrastructure, executives at Microsoft have admitted internally that the intermittent availability of renewable energy is not sufficient when data centers need to operate 24/7. Nonstop operation is considered so crucial that Google, Amazon, and most recently, Microsoft now build their campuses in threes to have a backup for the backup in case any facility goes down. 

During Hurricane Irma in Florida and Hurricane Harvey in Texas, even as millions of people lost power, some hospitals evacuated patients, and hundreds of thousands of homes and businesses faced damage and destruction. But the data centers in those areas continued to hum along so well that the displaced families of one facility’s employees moved into it for the duration of the natural disaster.

Researchers have shown that data centers create their own heat islands, raising temperatures nearby by as much as 2 degrees Celsius through their heavy use of industrial equipment and energy consumption. AI chips and models are becoming more efficient when it comes to power and heat, but those efficiency gains are being exceeded by overall growth in AI usage, in part as a result of that efficiency gain. This is a well-known phenomenon in technology innovation called the Jevons Paradox which was first described in the 1860s. 

The English were concerned about running out of coal, their main source of power. Some argued that as coal-burning plants became more efficient, less coal would be used. But economist William Stanley Jevons showed that increasing efficiency wouldn't save coal, because cheaper, more efficient power would lead to an overall rise in demand for coal. Data centers like Stargate may be getting more efficient, but if the Jevons Paradox holds up, those gains will be erased by the increased demand for AI. 

In addition to the land and energy required to support these megacampuses, they also require huge volumes of minerals including copper and lithium needed to build the hardware — computers, cables, power lines, batteries, backup generators. The tens of thousands of high-powered chips inside a data center like this generate enormous amounts of heat, requiring massive cooling systems to prevent failures. Stargate uses a closed-loop cooling system that circulates water to pull heat away from the machines. While these systems usually only need to be filled once, they do require more electricity to run.

The water must be clean enough to avoid clogging pipes and bacterial growth; potable water meets that standard. According to an estimate from researchers at the University of California, surging AI demand will consume 1.1 trillion to 1.7 trillion gallons of fresh water globally a year by 2027, or half the water annually consumed in the UK. Those effects will not be felt evenly. In Global South countries like Chile, it’s often the most vulnerable communities who have borne the brunt of these accelerating economies of extraction.

Elon Musk plans to have SpaceX put data centers into orbit around the world. They would use power from the sun. He has talked about how space has the advantage that it's always sunny. But do we know that a data center in space could actually work and if they are cost-effective? The microchips that AI developers use can get very hot. It's not clear how you would cool those components in a data center in space because space is in a vacuum. 

Data centers need a lot of maintenance. How do you get people (or, adequate robots, if not humans)  to be where they need to be to continue that ongoing maintenance work that has to happen? And the actual beaming of data across facilities in space is the other issue. It can be very slow, which might not make for the best user experience.

These data centers don't last forever - the chips degrade, and you can burn them out. But it's worse than that. One of the reasons that NVIDIA is worth so much is that every year they are coming out with chips that are substantially superior to the previous chips. The way that they're achieving these improvements is by jettisoning one of the main principles of sustainable product design, which is backwards compatibility with the infrastructure. They're eking out performance gains by going to the absolute limit of the envelope that the chip lives within, even if that breaks backwards compatibility. 

Breaking free of path dependency and starting afresh sounds great. But this means that these hard assets are not durable - you're replacing data centers. There have been successive generations of chips where you couldn't go back to the data center and put new chips in it. By the time you  finished building the datacenter, you might as well demolish it to the foundation slab because they are so specialized.

Data centers - I of III

Our conception of AI is that it's a digital technology. It exists in the cloud and it doesn't have a physical manifestation. But the reality is that it has the largest physical footprint of any technology in history. To train and serve up AI models requires tangible, physical data centers. They are required for storing and processing data, text, images, code, and other information sources.

To train and run the more advanced AI models requires more and larger data centers than ever before. It's estimated that by the year 2030, companies will have invested nearly $7 trillion in data center infrastructure in a race to dominate the AI market. The amount of money that is being poured into building data centers is far beyond anything that was ever spent on the Apollo program to get the first man to the moon.

The four largest spenders on data centers — Google, Microsoft, Amazon, Meta — now spend more money building data centers each year than almost all the others combined. They call their data centers “campuses” — large tracts of land that rival the largest Ivy League universities, with several massive buildings densely packed with racks on racks of computers. Those computers emanate a huge amount of heat because of millions of power-hungry processing chips running the chatbots, agents, and other tools that millions of people interact with every day.

To keep them from overheating, the buildings have massive cooling systems — large fans, air conditioners, or systems that evaporate water to cool down the servers. The equipment all together creates a cacophony of humming, whirring, and crackling that can — especially in underdeveloped communities — be heard for miles, twenty-four hours a day, creating a relentless source of noise pollution.

According to the International Energy Agency, each ChatGPT query is estimated to need on average about ten times more electricity than a typical search on Google. One study found that producing one thousand pieces of text from generative models used as much energy on average as what it would take to charge the standard smartphone nearly four times. Generating one thousand images used on average as much energy as “as 242 full smartphone charges; in other words, every AI-generated image could consume enough energy to charge a smartphone by roughly 25 percent. 

Developers and utility companies are now preparing for AI megacampuses that could soon require 1,000 to 2,000 megawatts of power. A single one could use as much energy per year as around one and a half to three and a half San Franciscos. By 2030, at the current pace of growth, AI computing globally could use more energy than all of India, the world’s third-largest electricity consumer.

The largest data centers are currently being constructed in the US. Take Stargate, one of the largest AI data centers in the world which is under construction in rural Texas. It will eventually include 4 million square feet of buildings across 1,100 acres. It already has 62 diesel generators and 10 gas-powered turbines on site, with plans for 41 more and it's just to power this data center. The largest one is being built by Meta in Louisiana which is roughly one fifth the size of Manhattan. If you add up just the three largest facilities in the world, the amount of energy that they demand almost matches the average power demand of New York City. 

They are being built largely to meet demand for generative AI tools like chatbots. OpenAI reported that global use of chatbots more than doubled in the last year, going from 400 to 900 million users. Growth is the number one concern for all of these data centers. They are trying to beat everybody else in being the first to really harness the profitability around AI. And for them, that really means essentially getting as much power as fast as they can.

Big tech companies are projected to spend more than $800 billion on the infrastructure behind artificial intelligence in 2026. And in 2027, that number will exceed a trillion dollars. That's on par with the entire U.S. defense budget. Stargate's proposed on-site power plant could deliver more than 1.7 gigawatts of power. That's enough energy to power more than one million homes for a year. And the plant will emit more than 7.8 million tons of greenhouse gases each year. That's the annual emissions of about two million cars.