AI and the fort of consciousness

Scientific advances "explain away" what is otherwise magical and metaphysical. In this article we explore how scientific advances in general and AI advances in particular are reducing the set of attributes we regard as uniquely human.

Stars were thought by some ancient cultures to be different worlds, or manifestations of gods. Today we think of them as burning gases, like our Sun.

There is immense variety of life on earth - plants and animals and bacteria and fishes etc. The best explanation earlier was that God created them all. But now we understand that all life evolved from single celled organisms. All the complexity that we see today is a result of evolution. At each step of reproduction, offsprings are slightly different (as a result of genetic mutation), and the offsprings carrying the mutations most suitable for survival perform the next step of reproduction. All the intricate morphology and behaviors of the organisms can be explained. Maybe this is what prompted Nietzsche to say God is dead.

For a long time it was thought that organic compounds - the ones which living things (plants and animals) are made up of - can only be made by living things. But then in 1828, Friedrich Wöhler synthesized urea in lab and since then many organic compounds have been synthesized in lab, leading to loss of magic in organic compounds.

Notable here is the celebrated Miller-Urey experiment where Miller and Urey tried to create the environment of early earth and were able to create several amino acids, which are the building blocks of living cells. They could not create living cells though.

This arc of scientific progress of explaining various attributes we think are unique to humans or to living things collides with our tendency to think that "We are special" - Indians are very proud of their ancient culture and Americans talk about American Exceptionalism. There is Chinese Exceptionalism called - 中国例外主义 too. In terms of religion - of course, everyone's religion is the one true right path and it is silly not to follow that path. Some people are chosen people, and God sends messangers to select group of people rather than using his omnipotence to put everyone on the right path.

As living beings and as humans, these beliefs are even stronger. There is a strong belief that living things are special. They feel pain and pleasure and non living things don't. There is the theory of Panpsychism though which says that mind like aspect is ubiqutuous feature of reality, not confined to living beings. But it is a fringe theory. And then among living beings humans are particularly superior - they alone have the power to think and do long term planning. Their consciousness is the most evolved (if at all we grant that non humans have consciousness).

However, bit by bit, our special ness is slipping away. Let's see how it is slipping away in the area of mind.

First, the machines could calculate faster than us. Well that was a mechanical task. In principle we could also calculate anything, it is just a matter of speed - and silicon can calculate faster than organic neurons.

Then machines could play chess better than us. As long as they were doing Alpha beta pruning we could rest assured that they are just using their computing prowess, but don't have any intuition of the game. Look - this approach was able to defeat best chess player (See here), but never the best go player. This is partly because chess has fewer positions than go - $10^{46}$ vs $10^{180}$ and chess positions are mostly easier to evaluate e.g. based on material. So we could say there is something called "intuition" which is more than computation.

Then things started to move fast. Once Geoffrey Hinton broke the Imagenet benchmark in 2012 using deep neural networks, there is no looking back for neural networks. First the images got recognized for dogs and cats and then came good description of images, and then came LLMs that answered simple questions. We humans could still hang on to our superiority since they acted only like "glorified auto complete"

But then reinforcement learning introduced another way of improving the models. Alphago combined deep learning with reinforcement learning and was able to defeat the best human player in Go. LLMs enhance deep learning with reinforcement learning and can solve IMO level Maths problems and solve competitive programming problems. While in strict sense-

It is just brute force computation - we know exactly what computation steps are there.
It is just memorization and interpolation / extrapolation - learning phase is a kind of memorization

But it is clear that this memorization is little bit of generalized memorization, just the kind that can be used to solve the problems that were not seen by the model in the trainig set.

Furthermore, just by (a) Training LLMs for next token prediction (b) Supplementing the training with reinforcement learning, some human like qualities are emerging in LLMs.

Chain of thought reasoning When we prompt large model using chain of thoughts, then model solves problem using reasoning. An example from here is below. Chain of though reasoning capabilities emerges only in very large models (say 100B+ parameter models) - so bigger models are in some ways getting closer to human capabilities.

Deception capabilities Larger models also have better deception capabilities - as show here, in game where there is a secret unknown killer amongst all the participants, and everyone discusses amongst themselves to determine who the killer is, bigger model is better able to convince smaller models that it cannot be the killer. Or else, as noted here a model may say that "As an AI assistant I don't have subjective opinion on this matter" whereas actually it would have opinion on that matter. All the deceptive behaviors emerge from the fact that model efficiently learns the behavior that reduce its training loss.

So, it appears that as you have bigger and bigger models, more and more human like qualities can emerge. While human brain has around 100 trillion conections between the neurons, the largest DNNs have currently 1-2 trillion parameters. So, as the the number of parameters in DNNs increase it is possible for new human like traits emerge in DNNs.

That brings me back to what we started with - we are running out of qualities that humans (or animals, or even living things) uniquely possess. Long term planning is unique to use today, but as DNNs get even larger, we losing our edge more in future is more likely that not. So, may be unique possession is "consciousness", or perhaps the ability to feel pleasure and pain. That might the last and the most promising holdout for humanity under attack from its own invention.

I view the uniquely human capabilties like a fort - doing maths, playing chess, doing science, discovering things, wondering about the world, being aware of the self and experiencing pleasure and pain. The fort is being breached. Perhaps only experiencing pleasure and pain will survive the onslaught of machines. But then, feeling pleasure and pain is a very different capability. Unlike other capabilities, it is not problem solving or accomplishing tasks. Is that capability the only capability that is uniquely human? Why does this capability even exist? Does that give us purpose? May be, because at least in Hinduism and Buddhism, the aim of life is to get Moksha which is absence of pain. May be it is pain and pleasure which give us purpose. And in absence of this, AI will be very good problem solver, but remain purposeless and derive purpose from us.

But then while other abilities are verifiable, experiencing pain and pleasure is subjective and hence not verifiable. What then is the guaratnee that it even exists? It might just be an illusion after all. Our saints has been saying Brahma-Satyam Jagan-Mithyā Jivo Brahmaiva Nāparaḥ (Only Brahma is True, the world is an illusion and invidual is not separate from Brahma). Perhaps it is time to reinterpreat the Upanishads in light of the latest technological advances.

At any rate, it will be very interesting to see how all this unfolds in coming 20-30 years. There are phases in human history when there is rapid progress in some areas - There was the period of 1900 - 1930 when so much of progress happened in Physics - Einstein, Planck, Bohr, Heisenberg, Schrodinger - all were contemporaries. There is a 1927 most intelligent photo ever taken where so many of these scientists in one frame. This is time of similarly rapid progress in the field of computing, even Nobel prizes are being won by people in AI - and we will look back at this time as pivotal in human history.