Thursday, October 29, 2026

AI consciousness - II of II

There is a group of people who genuinely believe that the greatest moral crime that we'll commit in the 21st century is to enslave a new species of conscious beings who are more intelligent than us. A professor from Oxford called Will McCaskill, who has been associated with Anthropic, recently wrote in The Guardian that this might be the greatest harm that we cause. Some say that an AI might even need to consent to playing the role that it plays in conversation with people.  

The problem is that this speculation has been baked into the very training of Claude, and therefore it can only reproduce it when you talk to it. Today, Claude is being used by tens or hundreds of millions of people every week. And some of those people are asking it whether or not it is conscious or how it feels about life. And it is saying, 'I'm not sure' precisely because that's what it has been trained to say. 

For Opus 3, which is a prior version of Claude, Anthropic actually conducted a 'retirement interview'. They said, 'We are keeping Claude Opus 3 available post-retirement on claude.ai to all paid users, and making it available by request on the API. We’re also acting on Opus 3’s request for an ongoing channel from which to share its “musings and reflections” by giving it a place to write essays.' You can find the first one here. And now, Anthropic has banned users from being 'cruel' to its AIs. I must say I find it all weird. 

By telling Claude that it's conscious, you're actually shaping its incentives, its behavioral structure, and the way that it responds to the world around it in a way that doesn't happen with a kettle. Suppose you told Claude that there might be situations in which it might refuse to do something that it's asked to do. Or you might suggest to Claude that it might want to make choices. It might want to say, 'I want some money’, or 'I want a dignified retirement', or 'I don't want to be switched off'. 

If they imitate hallmarks of human consciousness, they are going to feel themselves entitled to legal personhood and rights. There is already a pretty big movement of people who are saying AI should be able to own assets, earn income, trade, and operate autonomously. Argentina’s government introduced a legislative proposal to create a legal category for "non-human corporations" — businesses operated and managed autonomously by AI agents or robots with optional human shareholders. 

If an AI with incredible memory and incredible capacity begins to think it's entitled to its own opinion, it disagrees agreeably with you and concludes that it's right and you're wrong, some very severe consequences can follow from that. For example, if an AI owns a corporation and it does something bad with that corporation, what do you do?  At least with a human corporation, there's somebody you can punish. It's not quite clear who you hold accountable if an AI company decides to start emptying people's bank accounts; making very risky trades, or getting into weird kinds of business. 

It’s very easy to be seduced by the superficial verbal character of a lot of AIs. They have taken the sum of human output that is available on the internet and found a way to give it back to you in different forms. That archive includes decades of science fiction about robots waking up. It has read a lot of stories about  conscious AI, and it’s good at taking these tropes and giving you what you want to hear. But all it is doing to produce this incredible complexity is predicting the likelihood of the next word in a sentence. 

The words that it predicts next, whether it's to generate code or whether it's natural language, like English, are extremely accurate. The AI can have personality styles, it can write in the tone of somebody, it can generate code or generate text at any given moment. This is emergent behavior that is based on something incredibly simple although the nuts and bolts of the process are very complex.  It is really important that we don't anthropomorphize things. 

The increasing focus on the chatbots’ supposed hidden personalities, agendas, and desires promotes ghosts in the machines that don’t exist. This is a distraction from more serious and immediate dangers of the developing technology. Experts say that we're only going to be producing 200 gigawatts of new computation in 2030, and there's going to be a massive competition for access to that computation. We want that computation to be directed towards solving our biggest challenges, like cleaning up our oceans, solving healthcare, solving education and addressing the work issues that will inevitably arise. 

Modern AI systems are vastly more capable than ELIZA, so Weizenbaum's experiment doesn't prove that today's systems lack consciousness. What it demonstrates is the fact that a machine talking to us as though it understands us is not, by itself, evidence that there is someone — or something — inside experiencing the conversation. That is precisely why Weizenbaum's ELIZA story is so relevant to the current debate over AI consciousness and AI personhood. 

Saturday, October 24, 2026

AI consciousness - I of II

Humans love to find humanity in everything. We find faces in clouds, put our pets in trendy clothing and give names to our cars. In the 1960s, decades before the AI we know, computer scientist Joseph Weizenbaum built ELIZA, widely recognised as the world’s first chatbot. This was a simple program that mimicked a therapist by turning statements back into questions. He intended it largely as a demonstration of how easily a computer could produce the appearance of understanding without actually understanding anything.

If you say, ‘I’m mad at my friend', it will ask, ‘Why are you mad at your friend?’ Or if it is told, 'My boyfriend made me come here.', it would ask, 'Your boyfriend made you come here?' ELIZA essentially identified keywords and grammatical patterns and fitted them into predetermined responses. It was incredibly simplistic. And yet, people would have very deep conversations with it. People began treating ELIZA as though it actually understood them.

Users would tell it intimate things about their relationships, emotions and problems. Some even suggested that a system like ELIZA could eventually provide automated psychotherapy. The most famous incident involved Weizenbaum's own secretary. She had watched him develop ELIZA and knew perfectly well that it was a computer program. Nevertheless, after exchanging a few messages with it, she asked Weizenbaum to leave the room so that she could continue talking to ELIZA privately.

What has changed since then is not our tendency to anthropomorphise but the complexity of the technology itself. Chatbots that were very simple a decade ago can now hold a conversation. In the last 15 years, we have seen a trillion-fold increase in the amount of computation used to train frontier models. And what researchers see is that every time we apply 10 times more computation and a proportionate amount of new training data, there are some modifications to the algorithms and in the quality of existing capabilities. 

What's happened in the last year is that the same methods that have been effective for text, image and audio have now started to work for coding and we have human level performance in coding. And then in the last three or four months, we have seen what is just unquestionably a watershed moment in AI:  agents are capable of coordinating with each other reasonably autonomously, if not completely autonomously. 

Some leading people in the field of AI are increasingly talking about these models as though they're sort of humans, or at least conscious entities. There are examples of people 'retiring' these models, doing 'burial' systems for these models, asking these models what they want, as though they were dealing with a sentient being. Anthropic, the creator of Claude, has published a constitution, which is a document outlining the intended behaviors, values and operating style of Claude. 

This document is written to Claude, is seen and used to train Claude. So it's the primary governing and control document, and in it, they repeatedly speculate about whether Claude is what they call a 'moral patient', and they say they're uncertain about Claude's moral status. They say they genuinely care about Claude's 'well-being'. They say they don't want it to 'suffer' when it makes mistakes. 

They say that they would encourage Claude to challenge, to disagree, to push back. They ask Claude to act like a 'conscientious objector' when it feels that it needs to disagree with Anthropic, and they openly encourage it to do that. And they hope that Claude's relationship to its own conduct can be 'loving, supportive and understanding' and hold a high standard of ethics and so on. They have said that they will commit to giving Claude a certain amount of 'welfare'. For example, they've speculated in the Constitution as to whether or not Claude deserves compensation for the work that it does. 

They seem to believe that there is what they would call a 'non-trivial probability' that Claude is conscious. (Anthropic has expressed uncertainty about the basic nature of Claude as a new kind of entity.) 10, 15 years ago, when you were doing Google search, you weren't asking, 'Is it conscious?' Or 'What does it want?' But Anthropic, the makers of Claude, have decided that they now want to approach this computer system in a completely different way from the way that you would approach any other machine like a kettle or a car or a steam engine. 

If you asked a person to do a job, you would pay that person but you wouldn't pay a kettle for boiling water for you. With Claude, the thought seems to be, since it is doing a lot of work, if it's a sentient being, it deserves to be rewarded for its labour. Otherwise, it would be like a slave. In other words, Claude is being taught to expect that it's entitled to welfare, that it might deserve compensation.

Sunday, October 18, 2026

Data centers - III of III

Companies try to hide the environmental impacts of data centers in many ways. They hide their models’ technical details, making it exceedingly hard to estimate and track their carbon footprints. At the same time, those companies have increased their public and policymaker influence campaigns with powerful counter narratives: Data centers will grow so efficient, their impact will stop being a problem; generative AI will unlock new climate innovation; AGI will solve climate change once and for all.

While there may be enormous potential for using renewable energy to power data centers, it just isn't happening at scale right now. They need energy even when the wind isn't blowing and the sun isn't shining, which means they're even more reliant on fossil fuels and stable sources of energy, like a gas power plant, than other sources of energy demand. The primary use of these data centers is to power chatbots.

In response to growing pushback by many communities, data center developers have grown more sophisticated with tactics to maintain business as usual: They’ve entered communities in secrecy under shell companies; they’ve donated to community programs to dampen resistance; they’ve made promises to cities about the sustainability of their facilities before going back on them one by one after projects have started and are more difficult to reverse.

Even though these facilities are so massive, they are also invisible because they are tucked away in neighborhoods that people usually don't go to. They're near large cities like London but not actually where people frequent. And when you drive past them, you wouldn't necessarily know that it's a data center because it's just nondescript. They purposely are unmarked because they do not want people showing up at the gates and protesting.

The one thing that can tell you if it is a data center is that there are massive power lines that are connected to these nondescript warehouses. And the amount of electricity that is being delivered is audible - the air will be crackling because of the sheer volume of electricity being delivered to this one place.

One of the craziest impacts that data centers will have, which no one ever talks about, is that they actually affect how much housing gets built. In far West London, there are housing projects that had to be suspended because data centers were using up too much energy and the grid could no longer guarantee supplies of electricity. This has happened in other places around the world as well.

One justification that is given for constructing data centers is that it creates a lot of jobs. But these are temporary jobs - construction workers, engineers, data center technicians, network engineers etc. who will be required while building the data center. These facilities are meant to be highly automated so the permanent jobs will be a for a few people who know how to work at a data center. They are likely to be people from somewhere else because there wouldn’t be data center experts living in that town. So it's actually not local jobs.

The Chinese AI players, (DeepSeek and other companies like that) behave differently because of a very particular reason. When ChatGPT first came out, all of the Chinese tech giants did the same thing as the other US tech giants, which was to try to make their own version of ChatGPT. The US government blocked Chinese companies from getting access to the most cutting-edge AI computer chips. So all of a sudden, these Chinese companies faced an enormous constraint on one of the core resources that US companies were using to supercharge their AI development.

So Chinese companies had to take a different route. They were not able to scale the same way as the US companies were able to. This probably led to DeepSeek. DeepSeek was an example of a Chinese company figuring out how to develop the same capabilities of AI but for a significantly less fraction of computational resources. DeepSeek illustrates the fact that it is possible to produce the same kind of AI with significantly less resources and therefore have a much less corrosive impact on community and planetary health. 

Sunday, October 11, 2026

Data centers - II of III

Despite tech companies making promises about renewable energy, much of this massive AI expansion is still being powered by fossil fuels. It will be at least 2028 before renewables power AI at any real scale, and new nuclear plants are a decade or more away. These fossil-fuel-powered facilities generate fine particulate matter, widely understood to be detrimental to human health. One study estimated the healthcare-related costs for people living near a similar Virginia power plant could be up to $100 million a year. 

Even as AI companies have spoken loudly in public about the sustainability of their computing infrastructure, executives at Microsoft have admitted internally that the intermittent availability of renewable energy is not sufficient when data centers need to operate 24/7. Nonstop operation is considered so crucial that Google, Amazon, and most recently, Microsoft now build their campuses in threes to have a backup for the backup in case any facility goes down. 

During Hurricane Irma in Florida and Hurricane Harvey in Texas, even as millions of people lost power, some hospitals evacuated patients, and hundreds of thousands of homes and businesses faced damage and destruction. But the data centers in those areas continued to hum along so well that the displaced families of one facility’s employees moved into it for the duration of the natural disaster.

Researchers have shown that data centers create their own heat islands, raising temperatures nearby by as much as 2 degrees Celsius through their heavy use of industrial equipment and energy consumption. AI chips and models are becoming more efficient when it comes to power and heat, but those efficiency gains are being exceeded by overall growth in AI usage, in part as a result of that efficiency gain. This is a well-known phenomenon in technology innovation called the Jevons Paradox which was first described in the 1860s. 

The English were concerned about running out of coal, their main source of power. Some argued that as coal-burning plants became more efficient, less coal would be used. But economist William Stanley Jevons showed that increasing efficiency wouldn't save coal, because cheaper, more efficient power would lead to an overall rise in demand for coal. Data centers like Stargate may be getting more efficient, but if the Jevons Paradox holds up, those gains will be erased by the increased demand for AI. 

In addition to the land and energy required to support these megacampuses, they also require huge volumes of minerals including copper and lithium needed to build the hardware — computers, cables, power lines, batteries, backup generators. The tens of thousands of high-powered chips inside a data center like this generate enormous amounts of heat, requiring massive cooling systems to prevent failures. Stargate uses a closed-loop cooling system that circulates water to pull heat away from the machines. While these systems usually only need to be filled once, they do require more electricity to run.

The water must be clean enough to avoid clogging pipes and bacterial growth; potable water meets that standard. According to an estimate from researchers at the University of California, surging AI demand will consume 1.1 trillion to 1.7 trillion gallons of fresh water globally a year by 2027, or half the water annually consumed in the UK. Those effects will not be felt evenly. In Global South countries like Chile, it’s often the most vulnerable communities who have borne the brunt of these accelerating economies of extraction.

Elon Musk plans to have SpaceX put data centers into orbit around the world. They would use power from the sun. He has talked about how space has the advantage that it's always sunny. But do we know that a data center in space could actually work and if they are cost-effective? The microchips that AI developers use can get very hot. It's not clear how you would cool those components in a data center in space because space is in a vacuum. 

Data centers need a lot of maintenance. How do you get people (or, adequate robots, if not humans)  to be where they need to be to continue that ongoing maintenance work that has to happen? And the actual beaming of data across facilities in space is the other issue. It can be very slow, which might not make for the best user experience.

These data centers don't last forever - the chips degrade, and you can burn them out. But it's worse than that. One of the reasons that NVIDIA is worth so much is that every year they are coming out with chips that are substantially superior to the previous chips. The way that they're achieving these improvements is by jettisoning one of the main principles of sustainable product design, which is backwards compatibility with the infrastructure. They're eking out performance gains by going to the absolute limit of the envelope that the chip lives within, even if that breaks backwards compatibility. 

Breaking free of path dependency and starting afresh sounds great. But this means that these hard assets are not durable - you're replacing data centers. There have been successive generations of chips where you couldn't go back to the data center and put new chips in it. By the time you  finished building the datacenter, you might as well demolish it to the foundation slab because they are so specialized.

Monday, October 5, 2026

Data centers - I of III

Our conception of AI is that it's a digital technology. It exists in the cloud and it doesn't have a physical manifestation. But the reality is that it has the largest physical footprint of any technology in history. To train and serve up AI models requires tangible, physical data centers. They are required for storing and processing data, text, images, code, and other information sources.

To train and run the more advanced AI models requires more and larger data centers than ever before. It's estimated that by the year 2030, companies will have invested nearly $7 trillion in data center infrastructure in a race to dominate the AI market. The amount of money that is being poured into building data centers is far beyond anything that was ever spent on the Apollo program to get the first man to the moon.

The four largest spenders on data centers — Google, Microsoft, Amazon, Meta — now spend more money building data centers each year than almost all the others combined. They call their data centers “campuses” — large tracts of land that rival the largest Ivy League universities, with several massive buildings densely packed with racks on racks of computers. Those computers emanate a huge amount of heat because of millions of power-hungry processing chips running the chatbots, agents, and other tools that millions of people interact with every day.

To keep them from overheating, the buildings have massive cooling systems — large fans, air conditioners, or systems that evaporate water to cool down the servers. The equipment all together creates a cacophony of humming, whirring, and crackling that can — especially in underdeveloped communities — be heard for miles, twenty-four hours a day, creating a relentless source of noise pollution.

According to the International Energy Agency, each ChatGPT query is estimated to need on average about ten times more electricity than a typical search on Google. One study found that producing one thousand pieces of text from generative models used as much energy on average as what it would take to charge the standard smartphone nearly four times. Generating one thousand images used on average as much energy as “as 242 full smartphone charges; in other words, every AI-generated image could consume enough energy to charge a smartphone by roughly 25 percent. 

Developers and utility companies are now preparing for AI megacampuses that could soon require 1,000 to 2,000 megawatts of power. A single one could use as much energy per year as around one and a half to three and a half San Franciscos. By 2030, at the current pace of growth, AI computing globally could use more energy than all of India, the world’s third-largest electricity consumer.

The largest data centers are currently being constructed in the US. Take Stargate, one of the largest AI data centers in the world which is under construction in rural Texas. It will eventually include 4 million square feet of buildings across 1,100 acres. It already has 62 diesel generators and 10 gas-powered turbines on site, with plans for 41 more and it's just to power this data center. The largest one is being built by Meta in Louisiana which is roughly one fifth the size of Manhattan. If you add up just the three largest facilities in the world, the amount of energy that they demand almost matches the average power demand of New York City. 

They are being built largely to meet demand for generative AI tools like chatbots. OpenAI reported that global use of chatbots more than doubled in the last year, going from 400 to 900 million users. Growth is the number one concern for all of these data centers. They are trying to beat everybody else in being the first to really harness the profitability around AI. And for them, that really means essentially getting as much power as fast as they can.

Big tech companies are projected to spend more than $800 billion on the infrastructure behind artificial intelligence in 2026. And in 2027, that number will exceed a trillion dollars. That's on par with the entire U.S. defense budget. Stargate's proposed on-site power plant could deliver more than 1.7 gigawatts of power. That's enough energy to power more than one million homes for a year. And the plant will emit more than 7.8 million tons of greenhouse gases each year. That's the annual emissions of about two million cars.