CoreWeave: The Underdog Powering Generative AI's Explosion

Welcome to our new series! AI Infrastructure Unicorns. They provide the hardware, software, and services necessary for Generative AI companies but even if GenAI will someday become extinct, these infrastructure builders won’t stay without the job as they serve a much bigger industry of AI/ML models in general.

From its humble beginnings as a cryptocurrency mining operation, CoreWeave has emerged as a leading player in the world of cloud computing for AI. Last December, their valuation climbed from $2 billion to $7 billion after a minority investment of $642 million led by Fidelity Management and Research Co.

Their strategic pivot and early access to NVIDIA's cutting-edge GPUs made this happen. CoreWeave's ability to provide powerful and cost-effective GPU resources is fueling the current generative AI revolution. But they don’t forget their roots and previous partners: on March 7, 2024, CoreWeave has entered into a multi-year contract worth up to $100 million to lease 16MW of data center space from Core Scientific, a Bitcoin mining and digital infrastructure provider.

Let's explore CoreWeave's pivotal history, their strategies, its claims of being picks and shovels for all AI applications, its vast technical infrastructure offerings, dependencies challenges, and what it means to be dancing between the feet of elephants. 

In 2016, three commodity traders – Michael Intrator, Brian Venturo, and Brannin McBee – in a Manhattan office started a little hobby: they purchased one humble GPU to mine some Ethereum, hoping to “make an extra $1,000” here and there. Bitcoin was the leader in mining, but in 2015 a new cryptocurrency caught the miners' attention: Ethereum. According to the NYT, in March 2016, “Ethereum has soared in value, climbing 1,000 percent over the last three months.” It was worth giving it a shot. Many miners understood that teaming up in these entities called "pools" offered a better chance of success. But there was another novel concept: cloud mining. Cloud mining involves renting computational power from a provider to mine cryptocurrencies remotely, without managing physical hardware. In the world of AI and for those hungry for GPU power, that sounds familiar, right?

Riding the wave of cloud mining hype – and traders are very perceptive to hype waves in finance – the three colleagues kept buying GPUs. Soon, they found themselves in a sweltering office full of whirring GPUs. As Michael Intrator puts it: "All of a sudden, it went from a GPU to a bunch of GPUs to the pool table being covered in GPUs… We’re going to burn down this skyscraper!"

In 2017, they moved their hot GPUs to a garage in New Jersey and started CoreWeave (originally named Atlantic Crypto). They were now in the cloud mining business, with their first data center.

In 2018-2019, crypto suddenly and magnificently crashed. It could have been the end, but for CoreWeave, it was a pivotal moment. With cheap, distressed GPUs on the market, they made a strategic move. Intrator, Venturo, and Brannin McBee – their Chief Strategy Officer – started focusing on media & entertainment, life sciences, high-performance computing, machine learning (ML), and AI – all industries that needed the massive power GPUs offered.

Quick reminder, why GPUs are so crucial for ML, AI, and HPC: they accelerate it due to their parallel processing capabilities, efficiently handling large datasets and complex computations, leading to faster training and inference times in deep learning models. And indeed, AI loves to crunch data on those very GPUs.

In 2020, Forbes reported that the global machine learning market is projected to grow from $7.3B in 2020 to $30.6B in 2024. ML investments got higher and higher. With more ML companies, the search for GPU power was getting stronger as well. Traditional cloud companies (AWS, Microsoft Azure, Google) were still figuring this stuff out, charging a lot for access to this new AI-powered tech. 

But also, CoreWeave saw the gap – they had the GPUs, they just needed to build the system around them. 

They made another smart strategic move: when you own so many GPUs, you go to the maker of them to strike a partnership. And it's a genius of NVIDIA to build a relationship network with clients, service providers, influencers – everyone who can make their position wider and stronger.

Then we all know what happened in November 2022: OpenAI launched ChatGPT, igniting the generative AI revolution. Companies scrambled for the chips that make it all possible. Venturo realized, 'Everything we’ve thought from a scale perspective may be totally wrong... these people don’t need 5,000 GPUs. They need five million.' CoreWeave found themselves perfectly positioned to be in high demand. From a relatively unknown company with a few attention-worthy news items, they propelled into the unicorn league with all the media attention they could have imagined. Suddenly, everyone needed what they had.

What AI needed was a solid infrastructure. "It's similar to electricity: Do you think of the power plant when you flip a light switch?" said Brannin McBee. "What we're doing right now is building the electricity grid for the AI market. If this stuff doesn't get built, then AI will not be able to scale."

CoreWeave and NVIDIA are tightly intertwined. NVIDIA needs outlets for their chips and partners that are not competitors. CoreWeave needs the chips, credibility, and support to fulfill its ambitious plans. Though it does provide access to other chips (as detailed in the Technical part), the majority of its offerings consist of NVIDIA products. NVIDIA benefits from having a competent partner like CoreWeave, showcasing what their GPUs can do. CoreWeave's platform makes it easy for businesses and researchers to tap into the latest NVIDIA technology, often without needing their own expensive in-house hardware. This helps NVIDIA reach a wider audience. CoreWeave gets the benefit of early access to cutting-edge GPU tech from NVIDIA, plus the clout of being closely associated with the industry leader. It's like being part of the 'in crowd' in the GPU world.

According to CBInsights, NVIDIA is giving priority access to its GPUs to smaller specialty cloud providers CoreWeave and Lambda Labs, which are offering cloud GPU access at lower costs to enterprises.

There are a few key reasons:

Important Note: While CoreWeave often gets early access to new NVIDIA GPUs, it's not always guaranteed to be the absolute first. Some very large enterprise clients with direct contracts with NVIDIA might occasionally have equal or slightly earlier access based on specific use cases and needs.

This specialization lets them really get the most out of the hardware. They fine-tune their infrastructure, software, and networking specifically for graphics-intensive and computationally heavy workloads.

While CoreWeave doesn’t compete with NVIDIA, it does face off against the 'big 3' (AWS, Microsoft Azure, Google Cloud), which have been in the market for some time. One might wonder if there's a real need for a GPU-first cloud platform when GPU resources are accessible via established cloud computing platforms. In an interview with TheSequence, Brian Venturo noted that the 'big 3' suffer from 1) a limited variety of compute options, 2) the painful difficulty of scaling on-demand, and 3) excessive costs. CoreWeave aims to outperform the competition in these areas. If valuation is any indicator, it appears to be succeeding (refer to the Investment section for more).

Regarding smaller competitors like Lambda Labs, CoreWeave opts to attract their key personnel. In September 2023, CoreWeave welcomed Mike Mattacola, previously the Chief Operating Officer at Lambda Labs, as its new Chief Business Officer.

On March 12, they announced another big hire: Nitin Agrawal, who served as Vice President of Finance for Google Cloud, joins the company as Chief Financial Officer. There are other changes in the leadership: co-founder, Brannin McBee, will now serve as Chief Development Officer; co-founder, Brian Venturo has been appointed Chief Strategy Officer. Peter Salanki, current Vice President of Engineering will be elevated to Chief Technology Officer.

In just a few months, CoreWeave goes from obscurity to "whoa, where did they come from?" thanks to generative AI. They raise millions, secure billions in financing, and build a reputation as THE go-to GPU cloud resource.

Nvidia threw its weight behind the rapidly growing startup, and the investments paid off. CoreWeave went from being an unknown player to raising hundreds of millions in funding and building some of the world's most powerful supercomputers. "It’s similar to electricity," McBee said about the AI infrastructure they're building, "Do you think of the power plant when you flip a light switch? What we’re doing right now is building the electricity grid for the AI market."

According to Crunchbase, CoreWeave has undergone a series of financial rounds totaling $3.5 billion across eight rounds, as of March 8, 2024.

They quickly expanded, building giant data centers – a far cry from the humble beginnings in that New Jersey garage. The scale became astounding. "The ChatGPT moment," says Venturo, "was when I was like, ‘Everything we’ve thought from a scale perspective may be totally wrong. These people don’t need 5,000 GPUs. They need five million."

Partnerships (from CoreWeave’s website):

Microsoft also signed a multi-billion dollar deal with GPU cloud provider CoreWeave to meet AI needs.

Revenue

McBee said to VentureBeat that ‘CoreWeave did $30 million in revenue last year, will score $500 million this year, and has nearly $2 billion already contracted for next year.’

What part of it is from AI partnership and what from cryptocurrencies is unknown. From the recent news, we know that they maintain connections with their crypto partners. On March 7, 2024, CoreWeave entered into a multi-year contract worth up to $100 million to lease 16MW of data center space from Core Scientific, a Bitcoin mining and digital infrastructure provider. That’s certainly a nice diversification of revenue.

In a recent email, CoreWeave suggested some important statistics. In the past year alone: 

They call themselves "picks and shovels behind all of the AI applications." But even supported by all these numbers, they are still a relatively small company that doesn't actually produce any picks or shovels but rather buys or loans them. Here are a few challenges that might be crucial for CoreWeave's future.

“It feels like we are dancing between the feet of elephants,” said McBee to the Wall Street Journal.

CoreWeave targets a specific need for cloud computing resources within compute-intensive workloads that benefit from GPUs, unlike major cloud providers like Google Cloud Platform, Amazon Web Services, or Microsoft Azure, which offer a wide range of services. This focused approach allows CoreWeave to optimize its infrastructure for GPU-intensive workloads, delivering superior performance and cost-efficiency compared to general-purpose cloud solutions.

Just based on the example of the recent GPU shortage, you can understand the critical role of powerful compute resources for AI applications. Without proper computing resources, no machine learning application could exist not to speak about the popular large language models that require a tremendous amount of computing to train and run. So this is where the match happens between CoreWeave and enterprise AI applications which are becoming more and more popular raising the demand for the company’s services. CoreWeave also targets industries that work with visual effects, animation, and pixel streaming.

CoreWeave offers access to the separate components of its infrastructure including:

Cloud architecture

Cloud architecture is the high-level design that defines how cloud resources and infrastructure are organized, deployed, and managed. It defines how the various components of a cloud system will work together.

CoreWeave Cloud Architecture promises its clients “compute solutions that are up to 35x faster and 80% less expensive than legacy cloud providers.” It is responsible for the scalability and elasticity of the resources allowing them to be easily scaled up or down to meet changing demands.

Bare metal or a direct usage of computing

CoreWeave leverages "bare metal" technology, meaning user workloads run directly on the server's physical hardware (nodes) instead of a virtual machine created by software (hypervisor). This eliminates the hypervisor layer typically used to manage multiple virtual machines on a single server.

As a result, clients have direct access to the full resources of the server and benefit from more efficient resource allocation. Since the physical server is dedicated to their workload, clients experience predictable performance and granular control over hardware specifications – perfect for tasks requiring significant processing power or specific hardware needs.

Broadest range of NVIDIA GPUs

Traditional cloud platforms often force businesses to choose between affordable, low-powered GPUs and expensive high-performance options. This can limit the development of AI-powered products and applications.

CoreWeave breaks this limitation by offering a wide range of over 10 NVIDIA GPUs. This extensive selection allows companies to perfectly match their workload complexity with the appropriate processing power. This results in optimized performance and a cost-effective solution for businesses building cutting-edge AI applications.

Serverless + Kubernetes

Bare metal comes with the responsibility of managing the server by the client but not in the case of CoreWeave.

CoreWeave combines the benefits of the serverless approach and Kubernetes. Serverless computing eliminates the need to manage infrastructure. Kubernetes provides a powerful orchestration platform for containerized applications.

With Serverless Kubernetes, clients can focus on their applications' core functionality without worrying about managing servers or complex infrastructure. It can automatically scale resources up or down based on the client’s needs. This includes scaling across hundreds of GPUs for computationally intensive tasks. It also allows for more efficient resource usage and saves the client’s money. Serverless Kubernetes makes CoreWeave perfect for situations with fluctuating workloads or a need for on-demand GPU resources.

Storage

Traditionally, loading large PyTorch models can take a significant amount of time, impacting application performance. CoreWeave tackles this issue with two solutions:

CoreWeave's story is an ongoing saga. The crypto world that birthed them may feel like ancient history, but it instilled a scrappy, adaptive mindset vital in today's volatile tech landscape. They stand as proof that innovation isn't always born in sleek Silicon Valley offices; sometimes, it starts with a sweltering room of GPUs and a willingness to gamble on the future. It’s also quite an unusual story when traders build an ML infrastructure company.

Whatever waits ahead, CoreWeave's strategic vision so far hasn’t let them down.

Thank you for reading, please feel free to share with your friends and colleagues. 🤍