AI 101: Intelligence Processing Unit (IPU) and other alternatives to GPU/TPU/CPU

Even a child now knows what a GPU (Graphics Processing Unit) is – thanks to AI and to Nvidia, which keeps pushing its chips relentlessly. Of course, hardware is both the stumbling block and the engine that powers models and runs their tech stack. But why is the conversation so fixated on GPUs alone? Are there other contenders that could shape the future of AI hardware? CPUs and TPUs, sure – but is that all?

Today, let’s break out of the GPU bubble and look beyond the familiar trio of GPUs, CPUs, and TPUs. Developers around the world have been working on alternative designs, each promising new efficiencies and fresh approaches.

We wanted to create a full guide on AI hardware for you, so let’s start with the three main giants and move to something less popular but not less interesting from the inside: custom-built ASICs like Cerebras WSE and AWS hardware; APUs, NPUs, IPUs, RPUs, and FPGAs. We’ll make sense of all of these terms, so you can capture the full picture of AI hardware. It’s going to be a very insightful read!

In today’s episode, we will cover:

• CPU, GPU, TPU – three core hardware designs

  • Central Processing Unit (CPU)

  • Graphics Processing Unit (GPU)

  • Tensor Processing Unit (TPU)

• Application-Specific Integrated Circuits (ASICs)

  • Cerebras Wafer-Scale Engine (WSE)

  • AWS Trainium and AWS Inferentia

• Accelerated Processing Unit (APU)

• Neural Processing Unit (NPU)

• Other promising alternative architectures

  • Intelligence Processing Unit (IPU)

  • Resistive Processing Unit (RPU)

  • Field-Programmable Gate Arrays (FPGAs)

• Emerging Architectures

• Conclusion

• Sources and further reading

CPU, GPU, TPU – three core hardware designs

Before we move to the alternatives lets break down what these notorious CPU, GPU and TPU are.

These three giants are Processing Units (PUs) – specialized electronic circuits that execute instructions from software programs for computation. Many call them the "brain" of a computer system. PUs perform different arithmetic, logic, controlling, and input/output operations to process data, transforming it into useful information.

The main PUs types are optimized for different types of workloads →

Central Processing Unit (CPU)

Central Processing Unit (CPU) is developed for general-purpose computing and sequential task execution.

CPU is simply the oldest. The story of CPU’s predecessor begins in 1945 with ENIAC, Electronic Numerical Integrator and Computer, introduced by John Mauchly and J. Presper Eckert Jr. It was the first ever programmable, electronic, general-purpose digital computer, that could solve a wide range of numerical problems through reprogramming, using 18,000 vacuum tubes.

The same year came John von Neumann’s "First Draft of a Report on the EDVAC", proposing storing both data and instructions in the same memory. This stored-program model became the template for modern CPUs.

In mid-1950s, vacuum tubes were replaced by transistors. From that moment, processors were built out of many transistor-based components spread across circuit boards, making computers smaller, faster and less power-hungry.

In 1960s integrated circuits (ICs) appeared, packing multiple transistors onto a single silicon chip. And finally, in 1971, Intel released the 4004, the world’s first commercial microprocessor, a 4-bit CPU on a single chip. This was the true birth of the modern CPU.

Intel 8086 is the ancestor of today’s x86 CPUs, and up-to-date solution for modern efficiency is multi-core processors – multiple CPUs on one chip.

So what is inside up-to-date CPUs and how do they work?

At the heart of a CPU is the control unit with complex circuits that control the computer by sending out electrical signals, direct data and instructions to the right places. The arithmetic logic unit (ALU) handles math and logical operations, while registers and cache provide tiny but super-fast storage areas for data the processor needs constantly.

The CPU also contains cores – processing units in the CPU itself, each of which can independently process instructions, and threads, which allow a core to juggle multiple instruction streams. All of this runs to the beat of the clock, which provides the rhythm that keeps everything in sync. Supporting components like buses (for data transfer) and the instruction register and pointer (to track what’s next) tie the system together so instructions can move smoothly from one step to the next.

The CPU operates on a simple but powerful cycle: fetch → decode → execute.

• It fetches data or instructions from memory,

• Decodes them into signals the hardware understands,

• Executes the required operations (e.g. calculation solving, comparing values, or sending data somewhere else).

This happens billions of times per second in modern processors, with multiple cores and threads working in parallel to boost performance, making CPU like a highly organized team of components. Fewer cores (1-2) in CPU leads to efficiency, while more cores are used to power high-performance tasks.

Today’s CPUs mainly come from:

• Intel, which makes Core (consumer), Xeon (server/workstation), Pentium and Celeron (budget) chips;

• AMD, that offers Ryzen (consumer/high-performance) and EPYC (server) processors along with APUs (Accelerated Processing Unit) that combine CPU and GPU on one chip (we will look at them later below).

The main problem with CPUs for AI is that they’re optimized for sequential, general-purpose tasks rather than massively parallel matrix operations, making them far slower and less efficient than GPUs or specialized chips.

So let’s move to the second chip on our timeline – the famous GPUs.

Graphics Processing Unit (GPU)

Graphics Processing Unit (GPU) is optimized for massively parallel data processing with high throughput. Originally GPUs were invented to accelerate computer graphics in images and videos, but later they turned out to be useful in non-graphic calculations. Now they are broadly used in parallelizable workloads like handling data-intensive tasks and training of AI models.

Today GPUs are the main driver of AI performance and a key benchmark of computational capacity in AI.

The term Graphics Processing Unit (GPU) was officially introduced by NVIDIA in 1999 with the release of the GeForce 256. NVIDIA called it the world’s first GPU, and the official definition is “a single-chip processor with integrated transform, lighting, triangle setup/clipping, and rendering engines.”

So how does the legendary GPU work? →