AI versus the human brain
It's a complete walkover
Back in the early 90s I studied neural networks at university. Neural networks are the foundational technology behind ChatGPT and all the other AI Large Language Models (LLMs) that are currently turning the world upside down. A comparison of their complexity and efficiency compared to real brains leads to some interesting conclusions about AI progress.
Neural networks are based on how real brains work. They are simplified models of how neurons in our skulls (and spines) connect to one another. In an artificial neural network the neurons are called nodes, and they are arranged in layers. A node is connected to nodes in the layer next to it.
A neural network works by taking an input and converting it into an output. That doesn’t sound like much, but the output could represent something very significant. For example, the input could be the picture of a face and the output could be the name of the person in the picture.
Training is an integral part of how neural networks work. During training, the network is fed inputs for which the correct outputs are known. For example, pictures of faces and names. The system compares the actual output to the correct answer, then goes back and weights connections within the network to amplify what is correct and diminish what is wrong.
The process is then repeated many, many times. The training is complete when the network takes inputs and turns them into the correct outputs most of the time. The system is ready to be unleashed on real-world data.
For example, we feed 100 images of 10 people into the network, asking for their names. Every time the system spits out the wrong name, we adjust the network to become incrementally more accurate. Eventually the network’s multitude of connections reflect the differences in the faces of 10 people based on the training images. The beauty of a trained neural network is that it contains generalised knowledge, not yes/no rules: if we feed the system a new picture of one of the 10 people, it will be able to identify them.
Brains versus AIs
Modern neural networks are often sized based on how many “parameters” they have. A parameter is a number: either the “weight” of a connection or the “bias” on a node. In the case of our little network, there are 12 weights and 5 biases, for a total of 17 parameters.
Compare this to a modern LLM like ChatGPT 4. It reportedly has more than a trillion parameters spread over 120 layers. The computational power required to calculate and update all those numbers through hundreds of thousands of cycles is immense.
We can speculate from these numbers, working backwards, that the GPT4 network has at least 11 million individual nodes. Note that this number is a reasonable guess based on the information we have - OpenAI hasn’t disclosed any numbers and it’s all dependent on architecture.
That sounds like a big network until you compare it to the human brain, which has around 86 billion neurons. On these numbers, our brain is around 8000 times more complex than a state-of-the-art neural network.
The connection test
Another way of comparing the complexity of LLMs and brains is to look at the number of connection points in each: parameters in the LLM, or synapses in brains. Synapses are the points where neurons communicate, and there are an estimated 100 trillion of them in any human brain. On a simple comparison, that puts GPT 4’s trillion parameters at 1 percent of the complexity of human brain.
On these numbers, it seems our best efforts with AI may be at least approaching a fraction of the power of brains. But that’s misleading. The brain is a dynamic system, constantly adjusting its own network (learning) and also reliant on non-neuronal systems like brain chemicals and a maintenance system of helper cells and blood supply. You will often hear people talk about brain chemicals like dopamine, a neurotransmitter involved in the brain’s pleasure, reward, motivation, and motor control systems. The regulation of dopamine is vital to brain function and just this one neurotransmitter increases the complexity of the system. There are many such multipliers in real brains, enough to completely dwarf any current AI parameters.
What if our brains were an LLM?
We can turn this thing around, and ask what an neural network of comparable complexity to a human brain might look like.
A BeastGPT with 86 billion nodes, arranged in 120 layers, would contain around 60 quadrillion parameters. That’s a 6 with 16 zeros behind it. More than all the grains of sand on several thousand beaches.
The amount of power that it would take to train such an AI is mind boggling.
Experts estimate that training GPT4 consumed 10,000 MWh of electricity, the consumption of 9000 homes in a month.
Our BeastGPT would gobble up hundreds of thousands times more than that. In fact, it would take 10 percent of America’s entire electricity output for a year. It couldn’t be trained in Australia because there’s not nearly enough power, even if everything else was switched off.
Compare that to the modest energy requirements of the human brain. It takes just 20 watts of energy to keep our brains operational, sufficient to power a light globe. Over a “training period” of 18 years, the brain consumes 3155 kWh of energy, 139 billion times less than BeastGPT’s training power requirements.
We are only just beginning with AI
AI systems and human brains are so different that most computer scientists and neuroscientists would dismiss the comparisons I’m making. But I think they tell us one important thing. Our best current AIs are nothing compared to the human brain.
I can give you also give you another comparison that is easier to immediately grasp: GPT4 has less neurons than a frog.
I’m not here to play down AI and the generative revolution. My point is the opposite. I think it’s extraordinary, exhilarating and scary how much we have been able to achieve with what are comparatively rudimentary networks.
We are only taking our first steps. Pride in a trillion parameter systems will probably seem naive in a few years. The energy and data requirements of the AI training phase will come down, and eventually the training phase will be incorporated into the operational phase, just as it is in the real neural networks that allow animals to function.
Postscript: A curious thing about neural networks
The weighting process within a neural network is governed by statistical algorithms that run automatically. The network is finding differences in the inputs (in the case of our example network above, pictures of faces) and learning to categorise them in ways that are unknown to the network’s human creators.
So our face ID network may be distinguishing faces based on obvious things like hair length, eye colour, or skin tone. Or it may be using identifiers that are less obvious, like facial proportions. All the training does is reward the network for getting it right, without any idea how it arrived at the answer.
This can lead to unexpected problems. One well-known anecdote within the AI community involves a neural network trained to identify cancerous tumours in x-rays but failing when tested on real patient data. It is discovered that in the training data, all the images with tumours have a millimetre scale present, while those without tumours do not. The network had inadvertently learned to detect the scale rather than the tumours themselves. While this specific case is anecdotal, spurious correlation is a well-known pitfall in neural networks.
Some caveats
The total number of neurons in a brain is not directly related to intelligence. There are a handful of species with more neurons than humans, for example elephants and sperm whales. These big animals need big brains for controlling their movement, and the area of their brain dedicated to motor skills - called the cerebellum - is packed with neurons. The same is true of humans, the cerebellum, or “little brain”, sits at the back of our skulls, underneath the two main brain hemispheres. It contains about 80% of all our neurons. The “little brain” contains the majority of neurons in most mammals.
A better correlate to intelligence is the number of neurons in the frontal cortex, the area associated with higher functions like conscious decision-making. Humans have around 16 billion neurons in this area, more than any other animal.
Have a great week,
Hal