Silicon Valley is cheerleading the prospect of human–AI hybrids — we should be worried

The Singularity Is Nearer: When We Merge With AI Ray Kurzweil Viking (2024)
The US computer scientist, inventor and futurist Ray Kurzweil is a prophet of transhumanism, the movement that advocates for the possibility, even the moral imperative, of technologically tinkering with the human condition to enhance our cognition and eventually transcend our biology.
In his book The Singularity is Nearer — a sequel to The Singularity is Near (2005) — Kurzweil says that humanity is about to enter a new epoch in the history of intelligence: the merging of humans with artificial intelligence (AI). He borrows the ‘singularity’ metaphor from the physics of black holes: as we fall into the abyss of ever-improving computing power, what it means to be human seems to be on the verge of breaking down.

Consciousness: what it is, where it comes from — and whether machines can have it

Kurzweil reappraises his earlier predictions and insists that such an event is imminent and welcome. His forecasts are based largely on the exponential growth in the number of computations per second that can be performed per inflation-adjusted dollar, a trend that has been observed since 1939. He expects the Turing test to be passed before 2030 — computers will be able to exhibit intelligent behaviour indistinguishable from humans, and will go on to achieve “superhuman ability” in a wide range of areas. The book’s urgency thus comes from the nature of the exponential change itself: nearer is a different kind of near.
In Kurzweil’s techno-utopian world view, humanity’s future can be unpacked by merely following the evolution of information technology. By the end of this decade, Kurzweil says, most clothes and other common goods will be 3D-printed; brain–computer interfaces will be able to feed simulated sensory data directly into people’s brains. And in the long run, a species of human–AI hybrid will discover ways to turn all ordinary matter in the Universe into “computronium”, a hypothesized programmable substrate for ultra-high-efficiency computation.
To predict the future is difficult indeed. To articulate technocratic fictions is easier, especially when footloose metaphors are taken literally. Juggling basic evolutionary neuroscience with the new possibilities unleashed by large language models, Kurzweil postulates that computers will be able to simulate human brains “in all the ways we might care about within the next two decades or so”, adding, “at last we will have access to our own source code”.
Kurzweil repeatedly muddles computation with intelligence and consciousness. He flirts with materialism, dualism and panpsychism, contending that consciousness is “much like a fundamental force of the Universe”. Kurzweil then states that “it is the kind of information-processing complexity found in the brain that ‘awakens’ that force into the kind of subjective experience we recognize”. The words ‘complexity’ and ‘emergence’ are too often used in contexts in which ‘abracadabra’ might do as well.
The futurist imagines a technology capable of replacing each bit of the brain with a digital copy. Kurzweil does not entertain the role of uncertainty-ridden quantum processes, I suspect, because non-computational mechanisms would upend his paradigm and agenda. He also foretells that nanobots will soon dive into people’s brains and copy all their memories and personalities, and store them “in the cloud”.
But would such a “complete computerized replica” still be you? As a functionalist, Kurzweil thinks that doing trumps being. So, if an AI chatbot acts exactly like you, then it is you. The copy matches the original and then supersedes it. I strongly disagree. Reality is not a simulacrum, nor is consciousness a computer function.
Kurzweil’s hyperbolic technological fetishism does not stop in ‘the cloud’. Apparently, the soul is digital and the body is mechanical. And so, the litany of fiction science, as I call it, goes on: the hype is squared as AI meets nanoengineering, in a revolution that “will enable us to redesign and rebuild — molecule by molecule — our bodies and brains and the worlds with which we interact”. He also argues that diligent people will achieve “longevity escape velocity”, living for much longer than we do now, by 2030. I can only hope that we would have reached bullshit escape velocity by then, too.

If AI becomes conscious: here’s how researchers will know

Kurzweil’s stance towards death is key to understanding his entire quest. He yearns to bring his father Fredric back to life, at least in the form of a virtual avatar. To preserve a person’s identity, he first gathers everything ever written or said by them and stores them digitally. The next step is to bequeath these virtual replicas with physical bodies, choosing from several robotic avatars, eventually “grown from the DNA of the original person”, so that one could “continue a relationship with that person, even a physical one, including sex”. If the singularity is near, so is coitus with your counterfeit resurrected lover. This is less parenthetical than it sounds, because Kurzweil strives for a kind of transmuted intercourse between humanity and technology; ‘when we merge with AI’ is the subtitle of the book. Or, perhaps, as the Canadian philosopher Marshall McLuhan said in the book Understanding Media (1964), humans are fated to become “the sex organs of the machine world”.
In one particularly revealing passage, Kurzweil’s existential vulnerability and will to power is on full display. One could feel a sense of disappointment, mistrust and even revulsion towards Mother Nature and her limits. He laments that his brain evolved such that he is unable to learn faster, remember better, or reprogram his fears, traumas and doubts.

Consciousness theory slammed as ‘pseudoscience’ — sparking uproar

But the singularity will free us all. Kurzweil is convinced that we can copy life, delete death and edit ourselves free of biological frailties. Here, we find his gospel of salvation in a nutshell: the doctrine of liberation from the flesh and the promise of a post-human immortal species. Even the book’s title echoes the New Testament: “The kingdom of heaven has come near” (Matthew 4:17). As the UK philosopher John Gray puts it, such “unwitting theologians” engage in “a continuation of monotheism by other means”, with surrogates of God expressed in the rhetoric of scientific progress (Seven Types of Atheism; 2018).
Every road to utopia, however, meets dystopia halfway. Kurzweil does discuss the main technologies that threaten all of humanity, from nuclear weapons to biotechnology and, indeed, AI. Yet, under the spell of technolatry — the idol worship of technology — the cost–benefit analysis is invariably and infallibly positive: what creates the problems will take care of them. Thus, he seems to suggest that we should unthinkingly embrace technological change first and articulate the social contracts surrounding its use later. This sells humanity’s future short. Progress can give us non-stick frying pans and facilitate human flourishing, but it can also wipe us out or make our lives miserable.
In sum, Kurzweil’s motto is ‘merge thyself’ (with AI) — we can, we will and we must. Techno-utopian pseudo-religions of this kind can easily get out of hand, but he confidently reassures us that “nothing is being taken away. There will be a great deal added”. If I may be forgiven, it seems to me that Kurzweil’s understanding of humanity and nature is so bizarrely deficient that his augury is more of a curse than a blessing. His spiritual quest is devoid of spirit. The singularity betrays what is singularly human.