Graphcore: From Unicorn to SoftBank Acquisition – What Happened?

Graphcore had plans for $120 million ‘brain-scale’ supercomputers in 2024. In February 2024, PitchBook estimated a 97% probability that Graphcore would go public. Instead, the company was sold to SoftBank in July 2024, reportedly for $500 million – less than the total investments it received over the years. Once a unicorn with a $2.8 billion valuation and ambitions to overshadow NVIDIA, Graphcore found itself scrambling to pay salaries. As we continue to investigate AI infrastructure unicorns, particularly AI chip companies, let’s explore what happened to this once-promising semiconductor company, why it proved almost impossible to beat NVIDIA, what’s happening with the “brain-scale” computer, what SoftBank has in mind with this acquisition, and what the Japanese god of creation has to do with it. Read on!

In today’s episode:

Nigel Toon and Simon Knowles sold their company, Icera, to NVIDIA in 2011 for $435 million. Icera specialized in creating advanced 3G cellular modem chips, which played a crucial role in mobile communications technology. Following this successful sale, Toon and Knowles believed they could develop something that NVIDIA hadn’t yet achieved. In 2012, they started to discuss what they could build to beat NVIDIA. Knowles believed that the main obstacle in AI development was the inefficiency of computer chips, like CPUs and GPUs, which aren't designed to mimic human intuition. Instead of efficiently processing information, these chips analyze massive amounts of data, consuming vast amounts of energy. The two partners decided to design chips – Intelligence Processing Unit (IPU) –  that think more like human brains to improve AI efficiency and reduce energy usage. At that time, they argued that overcoming the hardware limitations was more crucial for advancing AI than just focusing on complex software.

“We wanted to build a very high-performance computer that manipulates numbers very imprecisely,” says Knowles. The founders assert that they recognized in 2012 that AI was going to be a huge thing. They understood it would need new types of processors. 

They believed that processors specifically designed for AI could outperform more general-purpose chips across a variety of ML tasks. 

“NVIDIA was still pretty small at the time. We thought we could compete with them,” says Toon. Toon explains that unlike traditional programming where machines follow step-by-step instructions, Graphcore's chips enable machines to learn autonomously. He likens this shift to the revolutionary emergence of microprocessors in the 1970s, suggesting that Graphcore is reinventing the approach to computing, much like Intel did back then.

Herman Hauser (co-founder of Acorn Computers that later became ARM) believed in Toon and Knowles from the very beginning, hoping that they would unleash the third wave of computing (CPU in the 1970s being the first, GPU in the 1990s being the second). 

After such endorsement, it was hard not to start the company. In 2013, the Graphcore project began in stealth mode, with an official launch in 2016 in Bristol, UK. This place is sometimes called a “Deep Tech Powerhouse” and is part of the Silicon Gorge region. Companies innovating in Bristol traditionally receive financial support from the British government. 

If you’re not interested in AI infrastructure unicorns, check out these practical courses from our partners →

But not in this case. The pre-launch of the company was tough. The founders started to test the waters with investors in 2015, but back then many VCs didn't yet see the potential of AI, nor the need for specialized chips. Often, they found an ally within VC firms, only to be dismissed at partner meetings where the question "What’s AI?" was still common. Investors were wary of the high capital intensity of chip development compared to software, which could be tested and scaled incrementally. Building a chip meant committing all resources upfront, making it a daunting proposition for many.

In 2016 – boom! – Intel acquired the AI startup Nervana for $350 million, and Google announced its own AI chip development. After that, investor interest surged, and Graphcore suddenly received a windfall.

Sequoia's partner, Matt Miller, was so convinced of Graphcore's potential that he pursued the investment despite the company not actively raising funds at the time. In 2019, Toon was recalling with a smile how Matt Miller from Sequoia, at his first board meeting, humorously warned other investors against talking about selling the company, emphasizing Sequoia's focus on building big, public companies. Back then Toon confirmed that Graphcore's goal is to go public, stating, "That’s the path we’re shooting for, absolutely."

This marked a significant endorsement and a turning point for Graphcore. The company's valuation soared, reaching $1.7 billion after a $200 million Series D round in December 2018. 

In 2019, CEO Nigel Toon said that the company would make $1 billion in revenue by 2024. 

The company valuation reached $2.8 billion after a $222 million Series E in 2020. 

In total, according to the rounds announced on the company website, Graphcore raised $684 million over 6 rounds. But according to another page on the their company website, they raised over $710 million.

Only to be sold just above $600 million (according to Financial Times) in July 2024. 

We talked to an AI researcher who has collaborated extensively with Graphcore (he asked not to quote him by his name). Here’s what he said:

“I think Graphcore is a great company, and they have an interesting hardware architecture – IPU – that, in certain cases, is superior to NVIDIA. The main advantage of NVIDIA is all the software tools. It's much easier to develop software for NVIDIA GPUs than for Graphcore IPUs. NVIDIA is a juggernaut, and because they have been in this space for many years and captured most of the market, it's nearly impossible to compete with them.”

According to former employees, as reported by Sifted, Graphcore’s issues stem from a mix of bad luck and poor strategy. Executives made poor decisions regarding commercial and tech strategies, leading to low morale and talent loss. Targeting big clients like Microsoft initially backfired, as the company found more success with startups but realized this too late. Confusion around market focus and inadequate software support further hampered progress. The software issues, particularly with usability and maintenance, led to the collapse of the Microsoft deal, further exacerbating the company’s struggles.

Remember, from the beginning, the founders were saying that overcoming hardware limitations was more crucial? In 2024, they changed the narrative, stating that Graphcore is one of the companies still competing with NVIDIA “because it's not just about the hardware. It's actually also very much about the software as well.” 

Narrative is one thing; what you have in practice is another.

First, let’s start with describing what an Intelligence Processing Unit (IPU) is.

IPUs leverage a parallel processing architecture that supports multiple instruction streams and multiple data streams, among other specialized features tailored for AI and machine learning.

In 2020, Graphcore introduced Colossus™ MK2 GC200, with massively parallel, MIMD (Multiple Instruction, Multiple Data) architecture*: 

*MIMD is a type of parallel computing architecture in which multiple processors operate independently, executing different instructions on different sets of data simultaneously. This contrasts with other architectures like SIMD (Single Instruction, Multiple Data), where multiple processors execute the same instruction on different data sets at the same time. With 900 MB of in-processor memory (IPM) and up to 256 GB of streaming memory (external DRAM): Up to 256 GB.

When it comes to performance, the MK2 GC200 delivers 560 TFLOPS FP8, 280 TFLOPS FP16, and 70 TFLOPS of FP32 performance at 185W. To put the numbers into context, NVIDIA's A100 delivers 312 FP16 TFLOPS without sparsity as well as 19.5 FP32 TFLOPS, whereas NVIDIA's H100 card offers 3,341 FP8 TFLOPS.

But seriously, 2020 is more than a few centuries ago in AI chips years. Comparatively, NVIDIA's latest The NVIDIA GB200 Grace Blackwell Superchip, introduced in 2024, offers 1.4 exaflops of AI performance and includes 30TB of fast memory.

Graphcore’s latest innovation was from 2022: Bow IPU.

Bow IPU is based on a wafer-on-wafer design with TSMC's 7nm process, includes 1,472 independent cores and 900MB of on-chip memory with a 65TB/s access speed. This architecture allows the Bow IPU to achieve 350 peak teraflops of mixed-precision AI compute and 87.5 peak single-precision teraflops. It provides a 40% performance improvement and 16% better power efficiency compared to its predecessor. The IPU also features 10 IPU links, delivering a total inter-chip bandwidth of 320 GB/s. The Bow IPU integrates into Bow-2000 IPU Machines, supporting configurations that scale up to the Bow Pod1024, which offers 89.6 petaflops FP32 compute.

To support all this hardware, Graphcore developed Poplar SDK – the software development kit for programming and optimizing IPU applications, supporting popular machine learning frameworks like TensorFlow and PyTorch.

Poplar SDK is quite good within its niche, but it is not as widely used or as popular as CUDA (specialized software platform for GPUs). Poplar SDK’s adoption is largely tied to the use of Graphcore’s IPUs, which are less common than NVIDIA GPUs.

In 2022, Graphcore promised to introduce the "Good Computer" by 2024, aiming to deliver 10^19 calculations per second, making it 100 million times faster than an average laptop. Named after codebreaker Jack Good, it was supposed to feature 4 PB of memory and cost $120 million. Utilizing 3D wafer stacking technology, it would accommodate up to 500 trillion parameters, positioning itself as a significant advancement in AI and high-performance computing. 

It was supposed to dwarf leading models like OpenAI’s GPT-3, which had 175 billion parameters at that time. Little did they know about the rapid advancements in AI in 2023 and 2024.

After Graphcore’s GC200 launch in 2020, questions were raised about the lack of major customer announcements, and therefore, about market adoption. Despite impressive technical specs, the absence of endorsements from key investors like Microsoft and Dell EMC, and the limited deployment statements from listed customers, suggest potential issues in securing large-scale commitments. This is particularly surprising given their significant funding and high-profile backers.

After losing a key deal with Microsoft in 2022, Graphcore has faced significant financial challenges. In June 2024, Molten Ventures reduced its stake's value by 45%, marking the second consecutive year of such a decrease. Baillie Gifford cut its stake by half, and Schroders reduced its valuation by 25%. 

Sequoia has written off its investment completely. 

Though founded in Bristol, the town didn’t bring Graphcore any luck. Despite the UK government's significant investment in AI supercomputing, it did not support Graphcore's ambitious Good AI supercomputer project, leaving the company with limited options. Graphcore’s plans for showcasing their advanced IPUs were stymied by the lack of financial backing. Facing financial difficulties and investor reluctance, Graphcore started to look for a buyer for its ambitious projects.

Masayoshi Son, the CEO of SoftBank, has laid out an ambitious plan that positions AI and AGI (Artificial General Intelligence) at the center of his vision for the future. He envisions a world where AI systems far surpass human intelligence. His vision includes AGI transforming industries such as manufacturing, finance, and logistics, and he believes that companies that embrace this technology will lead the future of humanity.

Beyond AGI, Son is also focused on the development of Artificial Super Intelligence (ASI), which he predicts could emerge by 2030. ASI, according to Son, would be vastly superior to AGI, potentially becoming up to 10,000 times smarter than humans by 2035. He actually said “he was put on Earth to create ASI”, just saying.

The question is: where will the compute for that come from? 

Too much for NVIDIA to have it all! In 2016, SoftBank acquired ARM for $32 billion. In February 2024, it became known that SoftBank is working on a project called “Izanagi,” a $100 billion AI chip venture intended to compete with industry giants. This project, named after the Japanese god of creation, speaks to the seriousness of Son’s ambitions. Son’s drive towards AI is not just business-focused; he sees it as a personal mission, believing that his purpose is to bring ASI to reality.

Now, they’ve pocketed Graphcore and its IPUs. Graphcore’s intelligence processing units (IPUs), designed for AI workloads, could complement Arm’s architecture, possibly leading to advancements in AI chip design, particularly for edge computing and data centers. The details of how Graphcore will fit into SoftBank’s portfolio are still unfolding, but this move underscores SoftBank’s strategic intent to remain competitive – or to win! – in the AI compute race.

Initially aiming to outshine NVIDIA with its innovative IPUs, Graphcore struggled due to strategic missteps and market challenges. While they had the foresight to envision the AI boom, they were not quick enough to execute on that knowledge. What a miss to not pay enough attention to the software stack! Failed strategic partnerships that could have pushed them forward also played a role. Despite receiving substantial investment and having powerful VCs on their side, the company ultimately failed to deliver. Their ambitious goals, such as the development of the 'Good Computer,' were overshadowed by these challenges, as they were simply too slow.

The acquisition by SoftBank marks the end of one chapter and the beginning of another, as Graphcore's technology may now play a role in SoftBank's broader AI and AGI ambitions. Hopefully, there will be people with a strong idea/speed of execution ratio to help them.

Thank you for reading, please feel free to share with your friends and colleagues. 🤍