The photo shown here is a typical CPU (Central Processing Unit) silicon chip used in desktop and laptop computers.

Specifically this is an AMD Ryzen 3 2200, a so-called “entry-level” chip for people building their own desktop gaming computers. Just a few years ago the power of such chips was still talked of in terms of the number of transistors they held, with one of the classics, the Intel 8086 of the 1980’s, having 29,000.

That AMD chip has 5 billion transistors.

As such people nowadays usually talk about other measurements of power such as clock speed and “cores”. What’s a core? Well it means a processing unit, a computer in itself. If the original silicon chips were said to be “a computer on a chip” then a 4-core chip like the Ryzen 3 2200 has four computers on a chip, and that’s pretty ordinary now. There are retailed chips with 64 cores.

Why do this? Why not just keep making a single core ever larger? Well there are scaling problems, not just in the hardware but in using a single processor to do a job. Instead, use is made of something called parallel computing, where one job is split into many, all run at the same time. It started off as something only used with the supercomputers simulating things like nuclear explosions. Parallel processing makes it possible to perform ever larger data processing jobs in human time.

The thing is that the human brain is also, basically, a parallel processor (Minsky), and a pretty massive one at that, with 100 trillion synapses (brain cells), each of which is like a transistor but with hundreds of connections to other synapses, which multiplies that 100 trillion in terms of data storage and delivers processing power beyond what should be possible given how much slower nerve impulses are compared to electronics.

The SF author Arthur C Clarke, in the book version of 2001: A Space Odyssey, actually makes reference to Minsky’s research on neural networks in explaining how the infamous computer HAL 9000 was developed, which shows you the sort of background study Clarke did for his stories. Minsky was an advisor to the movie.

For decades, most of these neural networks amounted to creating artificial “neurons” in the software, which was basically a clunking simulation, ultimately limited by the hardware it ran on. You could do interesting things, just slowly.

Which brings me to this news story, World’s Largest Chip Unlocks Brain-Sized AI Models With Cerebras CS-2.

Cores? It has 850,000 of them!

Cerebras Systems today announced that it has created what it bills as the first brain-scale AI solution – a single system that can support 120-trillion parameter AI models, beating out the 100 trillion synapses present in the human brain. In contrast, clusters of GPUs, the most commonly-used device for AI workloads, typically top out at 1 trillion parameters. Cerebras can accomplish this industry-first with a single 850,000-core system, but it can also spread workloads over up to 192 CS-2 systems with 162 million AI-optimized cores to unlock even more performance. 

Are you scared?

I am, and I’ve been in this world for most of my life.

Of course it’s going to take some time to develop the software to truly use this baby, but if we get to the point where the software-hardware configuration can start truly learning in growing itself, well…