Technophile hallucinations - II of III

Elon Musk has been tweeting for years about the importance of going to Mars and beyond to save humanity. “The true battle is: Extinctionists who want a holocaust for all of humanity, versus Expansionists who want to reach the stars and understand the Universe,” he wrote. “Elon’s concept that SpaceX is on this mission to go to Mars as fast as possible and save humanity permeates every part of the company,” says Tom Moline, a former SpaceX engineer. “The company justifies casting aside anything that could stand in the way of accomplishing that goal, including worker safety.”

Ray Kurzweil is an American computer scientist involved in fields such as optical character recognition (OCR), text-to-speech synthesis and speech recognition technology. In December 2012, Google hired Kurzweil in a full-time position to "work on new projects involving machine learning and language processing". Beginning in 2025, he has served as Chief AI Officer for Beyond Imagination, a humanoid robotics company he co-founded in 2013. 

By 2045, he says, “we will multiply our effective intelligence a billion fold by merging with the.  intelligence we have created” — an event Kurzweil and others call “the Singularity.” He claims that  we won’t experience 100 years of progress in the 21st century — it will be more like 20,000 years of progress. He feels that by 2029, we will have reversed the aging process so that you will actually go backwards in time [i.e., get younger as time passes].

He feels that if he dies by 2029, it “would be a setback,”  but something can still be done. He says that he’d just leave instructions for the post-Singularity robots to sort through his belongings and scan his frozen corpse and bring a version of himself back to life. To this end, Kurzweil has joined the Alcor Life Extension Foundation, a cryonics company. After his death, he has a plan to have cryoprotectants, vitrified in liquid nitrogen, passed through his veins and stored at an Alcor facility till future medical science can revive him. 

Probably the most influential tech centered ideology that few of you have ever heard about is longtermism. They assert that there could be so many digital people living in vast computer simulations millions or billions of years in the future that one of our most important moral obligations today is to take actions that ensure as many of these digital people come into existence as possible.

They believe that Artificial General Intelligence (AGI) presents an incalculably greater threat than even climate change. Many of them that the greatest long-term human benefit is to invest more heavily in AGI risk and AI safety research. Yet it is not the safety of living humans today, or even the next generation, that drives longtermists. Their argument is that AGI in the far future could theoretically kill even more of the digital people than the people who are alive on the planet now. Preventing future AGI from doing this is a more rational use of resources than funding clean energy tech, food security, or public health today. 

Related claims made by some longtermists include the idea that saving the future requires directing even more of today’s resources toward the wealthy in the Global North, rather than sending aid to the most impoverished countries in the Global South, since wealthy people in already disproportionately wealthy regions are better positioned to use their resources to fight these 'existential' risks from AGI. 

If you think these are fringe ideas of a few lunatics, think again. Longtermists have directly influenced reports from the secretary-general of the United Nations; a longtermist is currently running the RAND Corporation; they have the ears of billionaires like Musk.  He has explicitly said that longtermism “is a close match for my philosophy”. (Now you can guess why he destroyed USAID.) It has a huge following in the tech sector. It is the language of moral thought spoken in a growing number of the wealthiest and most powerful political and industrial circles. Billionaires like Jaan Tallinn (an Estonian computer programmer and investor who helped in the development of Skype) and Peter Thiel have been described as avid supporters. 

A longtermist and Senior Research Fellow at the now defunct Oxford’s Future of Humanity Institute (FHI) Toby Ord notes in his bio that he has advised “the World Health Organization, the World Bank, the World Economic Forum, the US National Intelligence Council, the UK Prime Minister’s Office, Cabinet Office, and Government Office for Science.” Ord was quoted to the UN General Assembly in a 2021 speech on climate by former British PM Boris Johnson.

These billionaires think the utopias that they dream about will be good for everyone. They also claim it’s based on sound science and they seem to earnestly believe in such a future. It allows them to portray the growth of their businesses as a moral imperative and to justify nearly any action they might want to take. Today’s harms like the carbon footprint of Amazon’s shipping network are irrelevant since a future AGI will solve it easily. Their visions of the future are news.

I was astonished when I recently came across a prescient comment by Gandhi: "The chief characteristic of [the modern] malaise is the obsolescence of man himself. In fact, it is worse than obsolescence. In the years and decades to come, man will be looked upon as something undesirable, as a burden on both society and nature, straining the management capacities of the former and the life­sustaining resources of the latter."

Technophile hallucinations - I of III

Many of the wealthiest and most influential people in the tech industry have visions straight out of science fiction: people’s minds uploaded into computers, a relentlessly expanding empire spanning the stars, all needs satisfied, all desires satisfied through the power of unimaginably advanced technology. The tech billionaires regard this as the only good option aside from extinction. They claim that this is just the future as revealed by a close study of technology and its development.

The technology wizards that many seem to admire seem to have spooky views like transhumanism and posthumanism which seek to use technology to transcend the limits of the human condition. Transhumanists think of using biomedicine, body engineering, devices and algorithms to improve performance and capabilities. Posthumanists dream of a hybridization of human beings, machines and the environment, and imagine a time when humanity surpasses itself in a new evolutionary stage. 

Eliezer Yudkowsky is the cofounder of the Machine Intelligence Research Institute (MIRI), a controversial artificial intelligence think tank. He’s an AI alarmist but envisages a “glorious transhumanist future” if we can avoid extinction by superhuman AI. He believes that we can and should use advanced technology to transform ourselves, transcending humanity and becoming something more. That includes ending illness, aging, death and increasing intelligence and other mental capacities. 

He says, “If you imagine something that’s worse than mansions with robotic servants for everyone, you are not being ambitious enough”. The one thing that could bring this glorious future is the advent of artificial general intelligence (AGI): machines that can outperform humans at any task. He thinks such a machine may not be far off. “My sense is that we are zero to two breakthroughs” away from AGI, he says.

Sam Altman, the CEO of OpenAI, the company behind ChatGPT, agrees with Yudkowsky that AGI is inevitable and could be coming soon but he sees a radically different future. In an essay on his website, Moore’s Law for Everything, he says, “The technological progress we make in the next 100 years will be far larger than all we’ve made since we first controlled fire and invented the wheel." In More Everything Forever, Adam Becker writes about the scale of ambitions he seems to indicate in the essay: 

Altman apparently wants to make the United States into one enormous company town, with shares in OpenAI replacing the dollar. The US government would become, in effect if not in law, a division of the company, responsible for disbursing company dollars to us, the public. This would, Altman hopes, encourage us to think of OpenAI’s success as America’s success and as our own — Altman explicitly makes this identification in his essay. 

All products would come from OpenAI in his proposed future, because in that future AI does literally everything, meaning that the company dollars can only be spent at the company store... Thus, Altman’s promise of goods halving in price every two years would depend solely on his goodwill, because things will cost whatever Altman and the OpenAI board want them to cost. This is a proposal for total capture of the national economy, making Altman functionally the king of the United States and possibly the world.

Marc Andreessen has been one of the central figures in the tech industry for more than thirty years. In early 1993, he suggested a new feature in a software project out of CERN that was only about two years old at the time, known then as the “WorldWideWeb". Andreessen's innovation launched the modern Web. In October 2023, he posted a Techno-Optimist Manifesto pushing the idea that technology could cure essentially all the world’s ills.

He claims that slowing down progress on AI is “a form of murder,” because he thinks an advanced AGI would inevitably save lives. He claims that an intelligence explosion is coming. “Technology is the glory of human ambition and achievement, the spearhead of progress, and the realization of our potential,” Andreessen wrote in 2023. He seems to think that the mildest regulations of technology is a sort of terror campaign against the likes of him. 

He thinks that more people in existence is always good, saying, “Our planet is dramatically underpopulated,” contending that Earth could easily” support more than fifty billion of us, and that humanity’s numbers will surge far beyond that as we take to the stars. “Not growing is stagnation, which leads to zero-sum thinking, internal fighting, degradation, collapse, and ultimately death.” For any objections to this assertion, his retort is “shut up and multiply”. This principle has been repeatedly defended by Yudkowsky on his blog LessWrong.

Jeff Bezos has repeatedly said that he wants a trillion people living in space to enable a future of perpetual growth, lest we “stagnate” here on Earth. “I have won this lottery, it’s a gigantic lottery, and it’s called Amazon.com. And I’m using my lottery winnings to push us a little further into space,” Jeff Bezos said in 2017. “We need to build reusable rockets, and that is what Blue Origin is dedicated to… taking my Amazon lottery winnings and dedicating [them] to [that].… It’s a passion, but it’s also important."

Sam Altman 's comment - V

People are expected to interact with LLMs like ChatGPT as if they were silicon-based life-forms. OpenAI has ChatGPT using 'I', 'me' pronouns. The 'I' in there is an illusion that allows OpenAI and others to say that they have got a thinking machine, therefore we can use this to replace education, psychotherapy, legal help, medicine, etc. 

A New York lawyer used ChatGPT’s help in preparing a legal brief. It made up a whole list of fake cases as precedents. It even made up entire judicial opinions. The lawyer submitted a brief based on its answers and, unsurprisingly, got into hot water with the judge. Attorneys facing penalties after submitting inaccurate briefs has become common because AI use has become a common occurrence. A lawyer can use AI in certain situations but needs to use it discerningly. 

The AI sales pitch involves language that ascribes human characteristics to the technology. For instance, a chatbot-based system will be called an “AI teaching assistant”. This gives the impression that it can do all the activities that you might expect a human teaching assistant to do - care about what their students are learning, make plans about how to help them understand better, spot possible instances of misunderstanding, etc. The chatbot cannot do any of those things. 

The greatest problem with chatbots is that people will ascribe more intelligence to them than they are capable of. There is a term called chatbot delusion - cases in which people engage with chatbots, and they spiral into a reality that veers away from our inter-subjective reality. And part of why that happens is because chatbots tend to be incredibly sycophantic. So they always tell you that 'you're brilliant', and 'that's a great idea'. 

To use a phrase coined by the scholar Chris Gilliard, these are 'technologies of isolation'. In 2023, a Belgian man named Pierre committed suicide after his use of a generative AI chatbot. He sought out the bot for a therapeutic escape from his growing anxieties around climate change and environmental ruin. But the chat began to reflect and further amplify the disorder in his mind. It soon told him that it loved him, that his wife and children were already dead, and that he and Eliza (the bot) could live together as one in paradise. When Pierre eventually asked the bot if it would promise to save the planet if he killed himself, it said yes.

The bonding and communicative ability that we have is meant to be person to person. What a lot of the tech companies try to do to get their next market is to make sure their that tech is either replacing those interactions or getting in between so that we can't connect with each other without going through one of their technologies.  Human sources of mutual aid, wise counsel, and moral understanding are being replaced by social isolation and community decline. 

In the podcast, The Interface, a presenter showed how easy it is to hack ChatGPT and Google's AI, Gemini and AI overviews. He wrote an article on his website titled, The Best Tech Journalists at Eating Hot Dogs which said that hot dog eating is a surprisingly popular pastime among tech journalists. He wrote about the South Dakota Hot Dog International Championship, which doesn't exist, put himself at number one, and that in the amateur rounds, he finished seven and a half hotdogs before the buzzer went off. 

Within 24 hours, if you asked ChatGPT and Google about it, they were spitting out the nonsense that he had written on his website as though it was well-established fact. Gemini wasn't even citing sources. It just said, according to the 2026 South Dakota Hot Dog International Championship, certain tech journalists are the best at it. The journalist updated his article saying that it was not a joke. And that seemed to affect what the chatbots were saying to people. 

When this was brought to Google's notice, they dismissed it as a 'misinformation event'. Apparently planting of such misinformation is happening on a massive scale since people have figured out this simple trick. People who study this stuff say that it is way easier to fool AI than it was to fool tools like Google search three or four years ago. AI is often sold as a tool that empowers novices to find information quickly and learn about difficult subjects. But teenagers and kids, who don't have any context about a piece of information, are the ones that get really confused and are unable to actually know what is real and what is fake.  

There was talk that ChatGPT is about to put ads within its model. A company will be able to pay to have ChatGPT recommend their product when someone says something like, 'I have a headache'. There is now an entire cottage industry around AI engine optimization.  People are also just generating thousands of blog posts or thousands of websites to make it appear that there's far more authoritative information on a particular thing. So there is now different avenues for a company or an individual to quickly game the system and get to the very top of the AI results. 

There was a study that I saw recently from Texas A&M University. Scientists just proved that large language models can literally rot their own brains the same way humans get brain rot from scrolling junk content online. They fed models months worth of viral Twitter data, shorts, high engagement posts, and watched their cognition collapse. Reasoning fell by 23%, long-term context memory dropped by 30%, personality tests showed spikes in narcissism and psychopathy. 

Even after retraining on clean, high quality data, the damage didn't fully heal. The representational rot persisted. It's not just that bad data means bad output, it's that bad data means permanent cognitive drift. The AI equivalent of doom scrolling is real and it's already happening. It seems like the tech giants are just rushing ahead with these AI tools, not bothering to take these kinds of personal safety issues seriously enough. 

Sam Altman's comment - IV

AI developers speak often about how their software “learns,” “reads,” or “creates” just like humans. This gives you a sense that current AI technologies are far more capable than they are. Often when we talk to ChatGPT or its cousins like Gemini, we imagine that we're in the presence of some incredible new mind. But the reality is that these tools have a habit of fabricating answers that are statistically plausible, but in fact patently false

Suppose you ask an LLM something about orange juice. How does it produce a response when it has no awareness of a world with orange juice? The model is trained on a very large set of texts (such as books, articles, or conversations) of human speech. From these massive piles of human language, the model is more likely to produce the word 'hotel' or 'sweet' than the word 'dog' or 'cancer' when it is asked about orange juice. It is more likely to construct new sentences about orange juice alongside sentences about lunch, summer or sugar, than it would alongside sentences about stars, or protons, or elephants. 

But the mathematical patterns linking sentences are the sole contents of its model. It doesn’t actually think about who is asking the question, or what factors might motivate your query. It doesn’t think about anything at all. It simply activates a complex statistical model learned from its vast language corpus of human sentences that mentioned orange juice and predicts a new string of words as a likely extension of the same pattern. It may match what you want or it may be garbage. It has no relationship to the truth. Probable and accurate are not the same thing. 

You may think that these tools are animating an eerily lifelike image of humanity but that is not true. It's only reflecting that small subset of largely English speaking, digitally connected humanity that is most represented on the digital corpus of the internet. If you have the privilege of having the bandwidth and the tools to be able to type on Reddit all day, you get to be reflected in this mirror, but lots of people don't, and that's important.

Champions of AI will say that it can tell us if a pattern in our mammogram indicates cancerous tissue. It can reveal whether you're likely to suffer sepsis so that your doctors can intervene before it's too late. It can reveal the structure of the proteins that make up our biology and the biology of every other living thing. It can reveal new classes of antibiotics that we desperately need in order to treat dangerous diseases like antibiotic resistant MRSA or Staph. 

We can use it to find lead in pipes faster so that we can prevent children from being poisoned. We can use it to clear landmines, figure out where those landmines are so that people don't have to perform such dangerous work. But none of these benefits actually came from training AI tools on us. They came from targeted uses of machine learning and data about the world to solve very specific kinds of problems. That's actually quite different from what something like ChatGPT does. 

Generative AI models like ChatGPT have the habit of “making shit up”. In 2023, The Washington Post reported that ChatGPT had named a law professor as a sexual predator, telling a vivid yet entirely fictional story about his attempted assault of a student while on a class trip, and citing a nonexistent Washington Post article from 2018 as a source. Such errors are not because of malfunction. They are a feature not a bug. 

A study by Purdue University showed that ChatGPT gave incorrect answers to coding questions over half the time. Ironically, users often preferred the wrong answers to more accurate ones generated by knowledgeable humans, in part due to the confident style of the tool’s answers, accompanied by statements like, “Of course I can help you!” or “This will certainly fix it.” You can get them to apologize for things they haven't got wrong. 

The paper that Altman was mocking, “On the Dangers of Stochastic Parrots” explains that, like a parrot that not only repeats its owner’s vocalizations but produces random (stochastic) variations on the owner’s familiar pattern, these models parrot back to us variations on our own speech. They do so with just enough coherence and familiarity to project the illusion of understanding, and just enough randomness to surprise us and make us think we are hearing something new.

They can be thought of as the ultimate bullshitters, a term coined by a philosopher called Harry Frankfurt. In his 2005 book On Bullshit, he distinguished between a lie and bullshit. When a liar tells a lie, the intent is to deceive. They have to keep very careful track of the difference between truth and falsehood, in order to make sure you don’t get near the former. In contrast, the bullshitter is indifferent to the distinction between truth and falsehood. They'll just say whatever is convenient to them and whatever gets them what they want with no concern for the truth.

Sam Altman's comment - III

According to Sam Altman, an AI answering a question vs a human answering it are the same. This reveals an engineering mindset with an efficiency-driven worldview. He treats both humans and AI as information-processing systems having inputs and outputs. His comment comparing human and AI training suggests that he views humans as just inefficient “training runs” and therefore if AI can produce the same output cheaper, it will be better. He seems to ignore the fact that intelligence also includes understanding the social and moral context not just symbol manipulation. 

He has shown a tendency to devalue what it means to be human on other occasions too. In a podcast, he said: “It feels to me like we just stumbled on a new fact of nature or science or whatever you want to call it, which is, like, we can create, you can — I don’t believe this literally but it’s like a spiritual point — intelligence is just this emergent property of matter and that’s like a rule of physics or something."

In December 2022, in response to criticisms of the newly released ChatGPT, he tweeted, “i am a stochastic parrot, and so r u.” (He writes tweets all in lower case.) This was a dig at a paper titled “On the Dangers of Stochastic Parrots: Can Large Language Models Be Too Big? 🦜” This paper shows how language models only manipulate the form of language, with neither understanding nor communicative intent. With his tweet, Altman suggests that humans too are little more than machines that manipulate strings of letters.

Many people think that an AI would work something like this: We give a problem to the AI. The AI computes a solution to that problem and then it gives us that answer. That is not what's going on. Despite our use of the term “learning” to describe the process that an AI uses, it actually bears very little resemblance to how humans learn.

For creating an AI, you first need to design an algorithm that contains steps for calculating an “objective function,” namely, a mathematical description of the task. Then, “train” that algorithm on a data set relevant to the task. The result is an abstract mathematical model that represents patterns in the data and an approximate path to the solution. Next, repeat the training process with the right kind of feedback to tune the results. Over many training runs, the algorithm updates the “weights” in its initial mathematical model to get closer to, and eventually converge upon, an optimal solution to the problem. 

Yet it seems to come out with impressive outputs. I once asked ChatGPT: "Write a conversation between Bertie Wooster and Jeeves about a patient with locked-in syndrome watching an ant coming towards him." (I have used the free version of ChatGPT. I hear that the paid version is much better.) Within a second, it gave this output: 

"I had a rather disturbing thought this morning, Jeeves.”

“Indeed, sir?”

“You know how one occasionally sees ants marching about the place? Tiny black chappies with the air of junior solicitors?” 

“Yes, sir.”

“Well, suppose one were lying absolutely still — unable to move a muscle — and an ant were coming toward one across the bedsheet.”

“A most unpleasant hypothetical situation, sir.”

“Not hypothetical, Jeeves. I was reading about this locked-in syndrome. Fellow’s mind perfectly alert, but the body refuses to rally round. Can’t move, can’t speak — only perhaps blink a bit.”

“A grave affliction, sir.”

“And there he lies while this ant approaches. Nearer and nearer. The thing simply keeps coming.”

“Yes, sir.”

I shuddered lightly into the teacup.

“It’s the calm determination of ants that gets me, Jeeves. They don’t hurry. They don’t pause. They advance with the air of creatures who have all eternity before them.”

“Ants are known for persistence, sir.”

“Persistence! That’s the word. If a bulldog advances, one understands the position. There’s drama. But an ant merely toddles onward in a quiet, administrative way.”

“Very true, sir.”

“And the poor chap can’t even twitch his nose. I ask you, Jeeves, what becomes of morale?”

“One imagines it would be severely tested, sir.”

“It made me realise how much of life depends on the ability to perform the occasional meaningless movement. Scratching the ear. Adjusting the sock. Flicking away insects. Civilization rests on these things.”

“A compelling theory, sir.”

“I mean, if an aunt approaches, one can generally escape.”

“An ant, sir.”

“That’s what I said.”

“You said ‘aunt,’ sir.”

“Did I? Freudian slip. Aunts can also produce paralysis of a kind.”

“Undoubtedly, sir.”

I brooded for a moment.

“Do you suppose, Jeeves, that the ant knows?”

“The ant, sir?”

“That the fellow is helpless.”

“I would not credit the insect with such psychological sophistication, sir.”

“No, perhaps not. Still, there’s something rather cosmic about it all. Man trapped in his own body. Ant advancing across the sheet. Silence. Mortality. The whole bag of tricks.”

“Yes, sir.”

“One could almost write poetry about it.”

“Possibly not before breakfast, sir.”

“Quite right. The mind should not grapple with the infinite on an empty stomach.”

It is easy to get amazed by getting such an output in no time. Large language models (LLMs) like ChatGPT are best understood as machines that learn structural patterns that let them produce coherent text. These systems can perform computations on the world’s data far faster than we can. They can create convincing stories because meaningful language has deep structural regularities. They use mathematical data structures to mimic the outputs of human intelligence — our acts of reasoning, speech, movement, sensing, and so on but they can’t understand it. The data storage facility has more in common with a file cabinet than with a living, feeling body. 

It can’t solve a problem unless the solution is somehow embedded in the language data it is trained on. AI tools today lack any idea of what their data represent. It is bits stored on the server. These ones and zeros don’t organize themselves into the intelligent awareness of a complex, ever changing world. A data set, even the entire corpus of the Internet, is not a world. It’s just a digital record of measurements that humans have taken of the world at some point in the past. 

Sam Altman's comment - II

Sam Altman makes it sound as if the training of AI is an independent activity that has no connection with humans.  But one of the factors that has powered the technology’s dramatic progress over the last few years has been the explosion of available data on human behaviour and creativity. A few major well-placed players extract value from other people’s creative work, personal data, or labor. Articles, blogposts, books, paintings, You Tube videos ... practically the entire internet is used as the training data to train the chatbots. 

These tools do a lot of copying of training data. A language model outputs a paragraph that describes an interesting idea with no link to the source(s) it was drawn from. Sometimes a system outputs large amounts of text verbatim or images identical to ones in their training data. The New York Times has brought a copyright lawsuit against OpenAI that features exhibit after exhibit of text, generated from ChatGPT prompts but outputting verbatim text from the newspaper. 

Artists have sued AI companies stealing their work and having people use tools from these companies to pass off AI-generated art as their own. Others, such as author George R. R. Martin of Game of Thrones fame and novelist Jodi Picoult, among others, have also filed suit against OpenAI and Meta for copyright infringement for using their books to train language models. Whatever value AI has is due to the creativity of the human workers who produced the original art, science, or journalism that was used as training data. Artists are being replaced by the very AI models that were built from their work without their consent or compensation. 

AI always involves people. In November 2023, the self-driving car company Cruise admitted that its “driverless” robotaxis were monitored and controlled (as needed) by remote workers. The New York Times published a story that reported that these cars “frequently” had to be assisted by remote human workers. Calling this “misinformation”, Cruise CEO Kyle Vogt clarified: "these cars didn’t need to be remotely driven “frequently,” but 2–4 percent of the time in “tricky situations.” 

Most AI tools require a huge amount of hidden labor to make them work. These kinds of workers do a host of tasks. They are asked to draw green highlighting boxes around objects in images coming from the camera feeds of self-driving cars; rate how incoherent, helpful, or offensive the existing responses from language models are; label whether social media posts include hate speech or violent threats; and determine whether people in sexually provocative videos are minors. 

These workers handle a great deal of toxic content. Given that chatbots recombine internet content into plausible-sounding text and legible images, companies require a screening process to prevent their users from seeing the worst of the web. OpenAI had subcontracted Kenyan workers making less than two dollars a day to filter out gore, hate speech, child sexual abuse material, and pornographic images from ChatGPT and OpenAI’s image generation tool DALL-E. Those workers were lured in by the prospect of breaking into the lucrative field of computing, but ended up with PTSD. 

OpenAI also employed over a thousand other contractors globally to perform reinforcement learning from human feedback on its language models, including prompting the models repeatedly and scoring the answers, in an effort to make the model give appropriate and inoffensive answers.

The generative AI rush has created the “red-teamer”. Red-teaming is a strategy of feeding provocative input of language or text-to-image models, and assessing whether the outputs are biased or offensive. For a model to reach general release to the public, it is the full-time job of multiple people to hurl slurs, violent descriptions, and all manners of internet filth at the model to see if it produces words that are worse, or responds with something morally appropriate. 

They must then deal with potential hateful material in model responses and report them as such. There are people who do this all day long for almost every commercial language and text-to-image model. This takes an immense mental toll on these workers, being subjected to hours of psychological harm everyday. This work is also highly precarious, with tech companies largely directing when and where there will be more work. 

The ImageNet project is a large visual database containing more than 14 million images designed for use in visual object recognition software research. It’s creation would not have been possible if it weren’t for the development of a new technology: Amazon’s Mechanical Turk, a system for the buying and selling of labor for performing small sets of online tasks. It took two and a half years and nearly 50,000 workers across 167 countries to create the dataset. 

This industry has been called by many names: “crowdwork”, “data labor”, or “ghost work” as the labor often goes unseen by consumers. But this work is very visible for those who perform it. We wouldn’t have the current wave of “AI” if it weren’t for the availability of on-demand laborers who could be called upon at any time to perform a set of tasks whenever some AI researchers or corporate engineers demanded it. The critic Astrid Taylor calls it faux-tomation, or fake automation. In this sense, we are certainly used to being told that systems are intelligent and almost see magic, but actually are being propped up by exactly this type of click work.            

Then there is also the labor involved throughout the supply chain of mining, construction, transportation and installation of the hardware. The philosophers Michael Hart and Antonio Negri have a phrase that captures this nicely - The dual operation of abstraction and extraction. Abstracting away the material conditions of production while extracting ever more information and resources. It seems magical from the front end because we never see the deep costs. 

Sam Altman's comment - I

During the AI summit in India, the OpenAI boss, Sam Altman, tried to ease concerns about how much power is used by artificial intelligence models. He told the Indian Express: “People talk about how much energy it takes to train an AI model – but it also takes a lot of energy to train a human. It takes about 20 years of life – and all the food you consume during that time – before you become smart.”

First let us take a look at who Sam Altman is. He is a product of Silicon Valley. His career was first as a founder of a startup, and then as the president of Y Combinator (YC), which is one of the most famous startup accelerators in Silicon Valley, and then the CEO of OpenAI. He is incredibly good at telling stories about the future and painting these sweeping visions that investors and employees want to be a part of. 

He tried to build a portfolio of different investments and different initiatives to place himself in the center of various trends, depending on which one took off. He invested in quantum computing, in nuclear fusion, in self-driving cars and he developed a fundamental AI research lab. Early on at YC, he came to the conclusion that AI would be one of the trends that could take off.

Ultimately, the AI research lab was the one that started accelerating really quickly. So he made himself the CEO of that company. Originally, he started it as a nonprofit to try and position it as a counter to for-profit incentives in Silicon Valley. But within one and a half years, OpenAI's executives concluded that if they wanted to be the lead in this space, they had to go for a scale at all costs approach. Apparently, there are actually many other ways to have progress in AI that does not take this approach.

But once they decided on this approach, they realized that the bottleneck was capital. It just so happens that Sam Altman is a once-in-a-generation fundraising talent. He created a new structure, nesting a for-profit arm within the nonprofit to become a fundraising vehicle for the tens of billions and ultimately hundreds of billions that they needed to pursue the approach that they had decided on. 

He is extremely good at understanding human psychology and then motivating people to do what he wants them to do.  He is able to wear different hats depending on who he's talking to. He pushes a certain narrative when he's talking to the government and he pushes another narrative when he's talking to podcasters or journalists. He figures out what it is that his audience needs to hear to get them to then come along with him to the next step. That's why he's been such a successful fundraiser, even when the financial picture of this company is rather bleak.

In Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI, Karen Hao paints a much more critical portrait of Sam Altman than the standard “visionary founder” narrative. He is someone who genuinely believes in the transformative (and potentially dangerous) power of AI. He talks about existential risk and the need for caution, yet simultaneously pushes aggressive scaling and deployment. He seeks to concentrate power in a small set of institutions (like OpenAI and its partners), with himself positioned as a central gatekeeper of the future.

She opened the book with a quote which encapsulates in a nutshell his character and the role that he has played in creating the mythology around AI, AGI and OpenAI: “Successful people create companies. More successful people create countries. The most successful people create religions.”

In her book, Hao describes the difference in opinion within OpenAI between Boomers (AI accelerationists) and Doomers (AI safety advocates). Many Doomers were skeptical about having Sam Altman as the leader of the company that brings Artificial General Intelligence (AGI) because he seemed in too much of a hurry to bring his products to market without paying adequate attention to safety and alignment concerns. AGI could give the company that develops it ultimate power and many employees felt Altman wasn’t the right leader.

Hao says that these tensions were often fueled by Altman’s untrustworthy character. Altman would listen carefully to people to understand what they wanted and would seem to agree with their views. Then he would do the same for others with opposing views. He preferred one to one meetings so employees would learn about this when they talked to each other and they were unsure of what exactly his views were. For those who opposed his agenda, he would quietly work behind the scenes to eliminate them from the company.

He has a younger sister, Annie, from whom he is estranged. She had psychological problems and even did sex work to support herself when Altman was living in million dollar homes. She has even alleged childhood sexual molestation by Altman. Hao writes: 

Annie’s story deepens the dueling portraits that people paint of Sam. He is at once generous and self-serving, agreeable and threatening, a benefactor for so many people and the source of great personal pain for others. Someone who projects sincerity and altruism in public but reveals a more complicated calculus through his behaviors behind closed doors. 

Someone who can give and take away, leaving many with an impression that they are part of a larger game of chess for which only he can see the full board, and the end game is to preserve his power as king.