Snowflake's Mission: Demolishing Data Limitations in the Era of Enterprise AI

Welcome to the AI Infrastructure Unicorns series. They provide the hardware, software, and services necessary for Generative AI companies but even if GenAI will someday become extinct, these infrastructure builders won’t stay without the job as they serve a much bigger industry of AI/ML models in general.

A few days ago, Snowflake made headlines with their Arctic LLM, an enterprise-grade model that is ready to rival DBRX, an LLM from one of their main competitors, Databricks. Snowflake Arctic offers top-tier intelligence at low training costs under $2 million, utilizing a Dense-MoE Hybrid architecture with 17B active parameters out of 480B total. It’s available under an Apache 2.0 license, promoting open access and collaboration. We’ve recently published a profile on Databricks (read here) as part of our AI Infra series and though Snowflake can’t be called a unicorn anymore since their IPO in 2022, we decided to tell their story as well. With Arctic, they demonstrated that they play high stakes and want to join the generative AI narrative, delivering its potential to enterprises. 

Let's explore Snowflake’s history, its struggles after the IPO, the CEO-for-the-task strategy, the tech specs for Arctic, and what makes it special, as well as the future prospects for Snowflake.

Today we will cover:

Snowflake was born in 2012 out of a shared frustration with traditional data warehousing solutions and a vision to redefine data analytics. The goal was to create a new data warehouse that could unite all users, data, and workloads in a single cloud service. Benoit Dageville, Thierry Cruanes, and Marcin Żukowski brought significant expertise in data warehousing to the table. Dageville and Cruanes had been architects at Oracle for over a decade. And only Żukowski had startup experience as a founder, having sold Vectorwise to Ingres Corp. in December 2010. Unusually, the founders decided not to appoint any of themselves as CEO. Instead, after raising seed and Series A rounds from Sutter Hill and working in stealth mode for two years, they launched in 2014 with a product, a Series B round, and their first appointed CEO, former Microsoft and Juniper Networks executive Bob Muglia. This ‘CEO-for-the-task’ approach would become a hallmark of Snowflake.

Their first product, the Elastic Data Warehouse launched on Amazon Web Services (AWS), set the stage for rapid evolution and expansion. By 2015, Snowflake was gaining traction, evidenced by a $45 million Series C funding round and a first-place win at the Strata + Hadoop World Startup Showcase.

The momentum continued with the introduction of new features like Snowpipe for continuous data ingestion and innovative data sharing capabilities, bolstering customer confidence and investment. Snowflake’s platform soon expanded beyond AWS to include Microsoft Azure, further broadening its market reach.

In 2019, with an IPO in mind, the founders decided to invite Frank Slootman as CEO, given his experience leading two companies through successful IPOs: Data Domain and ServiceNow. Under Slootman’s leadership, Snowflake broadened its product offerings, venturing into Google Cloud and launching the Snowflake Data Marketplace and Snowgrid.

He didn't let them down during the company's IPO in 2020. It was a landmark event, raising $3.4 billion and valuing Snowflake (SNOW) at $33 billion, making it the largest-ever IPO for a software company at the time. The stock finished its first day of trading above $250 per share (from $120 initial set). 

Starting March 2022, the price per share was negative for the early investors. Initially, the stock was irrationally high, which has contributed to its downward trend. 

There is skepticism about Snowflake's ability to meet its long-term revenue targets, especially after the company removed references to its fiscal 2029 goals from recent presentations, which may suggest challenges in sustaining its earlier rapid growth pace. What to do?

In May 2023, Snowflake rather quietly acquired – or more likely, acqui-hired – Neeva, a company attempting to disrupt the search industry with AI. It was announced that Snowflake planned to enhance its Data Cloud search capabilities with Neeva's technologies, aiming to integrate generative AI to transform how businesses interact with data, enabling more intuitive and conversational data query experiences. Some of the brightest minds in search have joined the Snowflake team, including those who worked on Google's search advertising and YouTube monetization. Neeva's founder, ex-Googler Sridhar Ramaswamy, has joined Snowflake as Senior Vice President of AI.

For the first 9 months, his LinkedIn profile said “Learning Snowflake”. Soon enough, in February 2024, Sridhar Ramaswamy was appointed as the CEO of Snowflake. Interestingly, February 2024 is also the last month on Marcin Żukowski's LinkedIn profile where he was listed as a co-founder of Snowflake. The company's website and the video titled 'How It All Started' present it as if the company was started by only two people. This change was done quietly. 

However, the appointment of a new CEO was noticed and did not please the market. When Frank Slootman, who remains Chairman of the board, stepped down, Snowflake's stock price plunged more than 20% in after-hours trading upon the news. Changes in management, however, often require time to positively impact the company. It has become clear that Snowflake’s strategic focus is on Enterprise AI.

Following this strategy, they introduced the Cortex platform (still in private preview), which Ramaswamy has led since joining Snowflake. Cortex is designed to facilitate the use of serverless functions for ML and GenAI applications, including LLMs, without the reliance on expensive GPU infrastructure. The platform incorporates features like GenAI Copilot and Document AI, which enhance data navigation and processing capabilities.

And now, two months since his appointment, Ramaswamy announced Snowflake’s own LLM, created in just a few months at a cost of $3 million (compare to Databricks' $10 million spent on their recent enterprise LLM DBRX). Data cloud companies aim to reduce LLM costs to enable wider business use with their data-science tools and databases, their main revenue sources. Both companies claim their open-source LLMs are cheaper than Meta’s. Sridhar Ramaswamy believes their new model, Arctic, could lead to significant long-term revenue by showing businesses the cost-efficiency of developing AI-powered applications.

With open-source being the new trend, Arctic is called “the most open” large language model. What’s also trendy is that it uses the Mixture-of-Experts (MoE) architecture which was around for a while, read from 1991, but revitalized and popularized thanks to Mistral AI’s bold release of Mixtral 8x7B.

We captured the trend at the end of 2023 and now we see it’s unfolding:

So, Snowflake’s Arctic steps in this line. But what’s special about it and how it’s built? Let’s look together.

According to Snowflake, Arctic outperforms other open-source models like DBRX, Yi and LLaMa on what Snowflake calls “enterprise intelligence” at the same time requires far less resources to train. For example, Arctic activates roughly 50% less parameters than DBRX, and 75% less than Llama 3 70B during inference or training.

Enterprise intelligence is a metric derived by averaging scores from specific evaluations: coding ability (assessed through HumanEval+ and MBPP+), SQL query generation (measured by Spider), and the capacity to follow instructions (evaluated by IFEval).

Arctic stands out because of the unprecedented speed it was created. Snowflake’s engineers and researchers took less than three months to build and train the model with only 12.5% of the training costs of similar models. Specifically, a training compute budget for Arctic is roughly under $2 million (less than 3K GPU weeks).

Arctic is a model with 480 billion total and 17 billion active parameters. It combines a traditional transformer with 10 billion parameters and 128×3.66 billion mixture-of-experts multi-layer perceptron (MoE MLP). In other words, Arctic uses a Dense-MoE Hybrid transformer architecture.

Based on the practical evidence, Snowflake researchers saw that the model quality depends primarily on:

That’s why Arctic uses 128 experts, more than other recent models, and a big number of parameters, 480 billion parameters. Unlike typical models that utilize all parameters continuously, Arctic optimizes resource use through a top-2 gating mechanism. This mechanism selectively activates only about 17 billion parameters, specifically choosing the best two experts for any given task. This boosts the model's performance and enhances efficiency by conserving computational resources.

The second insight from Snowflake researchers is to integrate a dense transformer with a residual MoE component to overcome the high communication overhead of training a standard MoE architecture with numerous experts.

Thirdly, the training data was divided into three parts to train the model within three stages focusing on generic skills like common sense in the first phase (1T Tokens), and enterprise-focused skills like coding, math, and SQL in the latter two phases (1.5T and 1T tokens). This allows for optimized resource use and targeted skill development tailored to enterprise needs.

The high training efficiency of Arctic also means its users can create cost-efficient custom models at scale. You can find more technical details and performance evaluations here.

Snowflake is releasing Arctic’s weights under an Apache 2.0 license that permits ungated personal, research, and commercial use. Additionally, they provide code templates, flexible inference, and training options to deploy and customize Arctic using a preferred framework.

Arctic LLM is part of the Snowflake Arctic model family, including the Arctic Embed for retrieval use cases. The family of five models are available on Hugging Face. Here is their Github repo for inference and fine-tuning recipe. Soon the model will be available on AWS, Lamini, Microsoft Azure, NVIDIA API catalog, Perplexity, Together AI, and more. Arctic is available for serverless inference in Snowflake Cortex, Snowflake’s fully managed service that offers machine learning and AI solutions in the Data Cloud.

There is no official mission statement on the company’s website, but at his opening remarks on Snowflake’s May 2023 earnings call, Frank Slootman said that the Snowflake mission is to steadily demolish any and all limits to data users, workloads, applications, and new forms of intelligence. “Our goal is for all the world’s data to find its way to Snowflake and not encounter any limitations in terms of use and purpose. From our perspective, machine learning, data science, and AI are workloads that we enable with increased capability, continuous performance, and efficiency improvements. Data has gravitational pull, and given the vast universe of data Snowflake already manages, it’s no surprise that interest in these capabilities is escalating while their uses are still evolving.”

“The era of enterprise AI is here,” said Sridhar Ramaswamy introducing the Arctic model.

Snowflake has yet to turn a profit since its IPO, as it focuses on growth.

The company reported a product revenue increase to $2.7 billion for the year ending January 31. Despite a robust net revenue retention rate of 131%, there has been a consistent decline over the past seven quarters, highlighting potential challenges in maintaining its revenue base. As for the Q1 of fiscal 2025, the financial forecasts revealed concerns about slowing growth, with revenue projections significantly lower than previous quarters. Snowflake anticipates first-quarter revenue growth of only 26% to 27% year-over-year and an annual product revenue increase of just 22%, signaling a deceleration in its growth trajectory.

There are a few important things to consider:

To offset potential losses and capture a share of the global tech spend, Snowflake and similar companies are broadening their revenue streams. They are incorporating additional services like advanced analytics, AI integrations, and tailored cloud solutions. The main question for Snowflake is whether its new leadership can tame the AI hype and seize the opportunity.

Open-sourcing Arctic, makes it possible for the company to catch up with its rivals. By leveraging its strong data infrastructure, Snowflake can effectively compete if it persuades users to retain their data within its data lake for training custom models. “If Snowflake’s new model, Arctic, can show businesses that they don’t need to spend a lot of money to develop new AI-powered applications or features themselves, it could unlock more revenue for enterprise software providers like Snowflake in the long run,” said CEO Sridhar Ramaswamy.

Thank you for reading, please feel free to share with your friends and colleagues. 🤍