∇ Guide: DeepSeek Opened Its AI Stack. How Much Can You Actually Use?

TL;DR: DeepSeek has opened much more than a model: weights, an inference engine, an agent harness, and the libraries underneath them. Together, they reveal how an agent system can be built. But how much of that system can we actually use – and what will it take to run it?

We explore DeepSeek’s ecosystem under the hood and the gap between open components and a working service.

When we talk about AI progress, we usually start with what a model can do. But when we use a capable model as an agent, other questions come up: Who can run it? What does it cost? How do we connect it to tools? We often explore questions like these in our Attention Span episodes on YouTube.

And here is DeepSeek again, this time with more to its story than leading reasoning models. It has opened an inference engine, an agent harness, and libraries that reach all the way down to how data moves between chips. Together, they let us see a large part of the machinery around the model.

But seeing the machinery and running it raise different questions. How much of this stack can we actually use to build an agent? And what still depends on DeepSeek’s hosted services or private infrastructure?

Here, we follow the pieces from the model to the agent and the infrastructure around it – to see what DeepSeek has put in our hands, and what it opens up. Let’s dive in →

In today’s episode:

If you set out to build an end-to-end coding agent using strictly what DeepSeek has open-sourced, the model is the obvious starting point.

And it’s an amazing model. The newest DeepSeek-V4.1-Flash came with multimodal reasoning, coding, and agent workflows on September 10, 2026. DeepSeek reports that the base model performs comparably to V4-Pro-Base across a range of benchmarks, with stronger results on some tasks and lower scores on others. October 17 brought the V4.1-Flash-1017 update, and now the updated weights are available under an MIT license.

One result helps explain why it matters for agents: the model’s global KV cache – stored information that helps it reuse previously processed context – takes 890 bytes per token, around one-quarter of the corresponding V4 Flash footprint. A smaller cache can make it easier to carry long contexts through a session. This measures global KV state, not the memory footprint of the whole running model.

Architecturally, V4.1 Flash is a multimodal mixture-of-experts model. Its 552 billion backbone parameters are paired with 196 billion Engram parameters – a sparse, look-up memory bank learned during training. Engram gives the model conditional access to externalized knowledge weights without bloating the active KV cache, and is distinct from the user's prompt or conversation history.

It is very interesting how this model treats input. Its 40 layers are split into a 20-layer causal encoder and a 20-layer decoder. The decoder’s global KV state is projected from the encoder’s final states. Due to this, most prompt tokens can bypass full decoder computation. It uses 8 billion activated parameters per token during prefill, and 16 billion during decoding. But local sliding-window attention still needs additional work.

V4.1 Flash also uses bounded replay: it approximately rebuilds recent sliding-window state from a short suffix. This exchanges some recomputation for lower persistent-storage requirements. But note that approximation and sparse-selection errors may cause problems in cases not covered by its evaluations.

Compressed Sparse Attention 2 addresses another source of duplication. Some layers create global KV state and select relevant entries, while others reuse the state, or both the state and the selection. Combined with low-precision FP4 caching, this is how the model reduces that global KV footprint.

For agents, these choices are designed to cut the compounding cost of long context. Whether that actually pays off depends on how you build the serving stack – the infrastructure – around the model.

Once you have the model, the next step is to find out how to run it efficiently, and in the case of DeepSeek, the solution is DeepInfer. DeepSeek describes it as the inference framework behind its production services. The public release currently supports offline inference.

So what’s inside?

DeepInfer gives developers quite a few pieces of the puzzle. First of all, it includes DSpark speculative decoding: a draft model proposes several tokens, and confidence estimates help decide how many are worth verifying. The speedup depends partly on how many draft tokens are accepted. Meanwhile, asynchronous scheduling keeps the accelerator busy while the CPU prepares the next batch, reducing the gaps between computation steps. Its supported hardware is NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs.

But then comes the reality when several users start using it at once and more uncertainties appear. Where should each request go? Can it reuse context already processed? The public release leaves shared KV cache storage, separate vision/prefill/decode serving, and load balancing for developers to assemble. The engine handles its own internal KV state. Sharing and reusing it across a service takes more infrastructure.

DeepSeek’s deployment guide gives a sense of the intended scale: 64 GPUs for prefill plus 64 for decoding is its smallest recommended efficient serving setup. The guide also says one prefill instance and one decode instance are deployable.

So DeepInfer lets us inspect and adapt the engine. But running a coding agent for users takes a serving system around it. There is another question, too: how does the agent work with tools once the model is running?

Now we reach the most interesting part →

Let’s explore the DeepSeek ecosystem from the engine down to the hardware kernels, and break down what it actually takes to run it in production. Plus a bonus: a convenient PDF with DeepSeek Ecosystem

Join Premium members from top companies like Microsoft, NVIDIA, Google, HF, OpenAI, a16z, plus AI labs such as Ai2, MIT, Berkeley, .gov, and thousands of others to really understand what’s going on in AI. 

Not interested in the topic? Check our latest about the new model that learns to read without a tokenizer

DeepSeek V4.1 Flash is a multimodal mixture-of-experts model that accepts text and images. Its architecture reduces the cost of carrying long contexts through an agent session using compressed KV state and different computation paths for prefill and decoding. The October 17 update provides model weights under an MIT license.

DeepInfer runs the model; DeepSeek Harness manages the agent loop around it. The engine handles inference, scheduling, speculative decoding, and internal KV state. Harness assembles context, calls a model, runs permitted tools, and records session history. Its replaceable plugins let developers change those components without rewriting the whole loop.

Not on their own. DeepInfer’s public release supports offline inference but leaves out service-level cache management, separate vision/prefill/decode serving, and request balancing. Developers must connect those pieces and test their own workloads. Harness also remains an unaudited developer preview, so its permission controls are not a production security guarantee.

The public materials reviewed here describe DSec’s architecture but do not provide a deployable platform. The paper explains how DeepSeek manages isolated agent environments at scale. Teams can use other sandbox systems; reproducing DeepSeek’s training and evaluation infrastructure is a separate project from running its agent harness.

API access does not provide a self-hostable DeepSeek search engine. Search returns web results, while Fetch retrieves page content as Markdown. Both run as hosted services. A team that wants to operate independently would need to build or choose alternatives rather than treating these APIs as released infrastructure.