10 Small Language Models to Know in 2026

Updated July 18, 2026

Small language models used to mean one thing: compromise. You accepted weaker performance because the model was cheaper, faster, or small enough to run on a local device.

That trade-off still exists, but it has become much less obvious. The latest small language models can reason, call tools, understand images and audio, process long documents, and run parts of an agentic workflow. Some fit on laptops or phones. Others use mixture-of-experts architectures, carrying tens of billions of parameters while activating only a small fraction for each token.

So yes, small models are getting bigger. AI naming remains a calm and orderly science.

Small language models, or SLMs, are compact AI models designed to perform language, reasoning, coding, retrieval, or multimodal tasks with lower computational and memory requirements than frontier-scale models.

There is no universally accepted parameter limit. Some SLMs have fewer than one billion parameters, while others reach 12B or 15B.

/ For a broader comparison of SLMs, LLMs, multimodal models, mixture-of-experts models, and other architectures, see our guide to 13 foundational types of AI models

Mixture-of-experts models may contain 30B total parameters but activate only 3B during inference. Proprietary providers also use labels such as “mini” and “nano” without publishing parameter counts.

What makes a model small is increasingly its deployment profile rather than one specific number. SLMs generally offer some combination of:

In 2026, small models are used for summarization, classification, extraction, retrieval-augmented generation, coding assistance, tool calling, document processing, local assistants, and specialized agents. They will not replace frontier models for every task, but they can be the better choice when latency, privacy, control, or cost matters more than maximum general capability.

/We explored these trade-offs in more detail in Large vs Small in AI: The Language Model Size Dilemma

TL;DR: The strongest small language models in 2026 are no longer limited to simple text tasks. GPT-5.4 mini, Gemma 4, Ministral 3, Nemotron 3 Nano, Phi-4, Tiny Aya, Granite 4.1, Qwen3, SmolLM3, and North Mini Code cover everything from local multimodal AI to multilingual and coding agents.

Context-window numbers do not guarantee that a model will use every part of a long prompt equally well. Actual memory use and speed also depend on precision, quantization, runtime, hardware, batch size, and how many reasoning tokens the model generates.

OpenAI released GPT-5.4 mini and nano in March 2026 as smaller versions of GPT-5.4 optimized for coding, tool use, multimodal reasoning, and high-volume workloads.

GPT-5.4 mini accepts text and image inputs and supports function calling, web search, file search, computer use, and skills through the API. It has a 400K-token context window and is also available in Codex and ChatGPT.

GPT-5.4 nano is available through the API and is designed for simpler, repetitive tasks such as classification, extraction, ranking, routing, and lightweight subagents. OpenAI does not disclose either model’s parameter count.

GPT-5.4 mini is a strong choice when you need a fast hosted model for coding, document analysis, tool calling, visual interface understanding, or agents working inside a larger system.

Nano is better suited to high-volume tasks where the output is narrow and easy to evaluate. There is little reason to send every classification or extraction request to a frontier model if a smaller one can do the job reliably.

Both models also remove the infrastructure work required to deploy, optimize, and maintain an open-weight model.

These are proprietary models. You cannot download the weights, inspect the training data, or run them completely offline.

“Mini” also does not automatically mean inexpensive. Long prompts, extensive reasoning, computer use, and repeated tool calls can still create significant costs at scale.

Google introduced the first Gemma 4 models in April 2026, releasing the family in Effective 2B, Effective 4B, 26B mixture-of-experts, and 31B dense versions. A 12B multimodal model followed in June.

The family is built for reasoning and agentic workflows rather than simple chat alone. Larger Gemma 4 models support context windows of up to 256K tokens, while Google has also released quantization-aware training checkpoints and multi-token prediction drafters to improve local memory use and inference speed.

Gemma 4 12B is particularly interesting. It processes text, images, audio, and video through a unified architecture without relying on separate large multimodal encoders. Google says it can run locally on laptops with 16GB of VRAM or unified memory.

/ For a closer look at the model’s architecture and its role in local agent systems, read our deep dive on Gemma 4 with OpenClaw.

Gemma 4 gives developers several deployment options inside one family. The smaller models are intended for edge and local use, while the 12B and larger variants support more demanding multimodal and reasoning workloads.

The 12B model is especially relevant for private desktop assistants, meeting analysis, accessibility tools, media search, local coding agents, and applications that need to process sensitive images or audio without sending everything to an external API.

The capabilities differ significantly across the family. Do not assume every Gemma 4 checkpoint supports the same modalities, context length, or hardware profile.

A 12B or 31B model may be small relative to frontier systems, but it is not tiny. Comfortable local deployment still requires suitable hardware, quantization, and an optimized inference runtime.

Mistral AI released the Ministral 3 family in December 2025 in 3B, 8B, and 14B sizes. Each size is available in base, instruction-tuned, and reasoning variants.

All Ministral 3 models support text and image inputs, multilingual use, and context windows of up to 256K tokens. Mistral released the family under the Apache 2.0 license and provides compressed checkpoints for local and edge deployment.

Ministral 3 offers a practical range of model sizes. The 3B version is appropriate for constrained hardware and narrow tasks, while the 8B and 14B versions provide stronger reasoning, language, and visual capabilities.

The permissive license and separate instruct and reasoning variants also make the family attractive for companies that want to customize or privately deploy a model instead of relying entirely on external APIs.

Reasoning variants can generate considerably more tokens than standard instruction-tuned models. That may improve difficult answers, but it also increases latency and compute use.

Developers should test all three sizes on their actual workload. The largest available model is not automatically the best economic choice.

→ Explore the Ministral 3 model documentation

NVIDIA released Nemotron 3 Nano in December 2025 as the smallest member of its Nemotron 3 model family.

The text model uses a hybrid Mamba-Transformer mixture-of-experts architecture with 30B total parameters and approximately 3B active during inference. It supports context windows of up to one million tokens and is designed for coding, reasoning, instruction following, tool calling, retrieval, and agent workflows.

In April 2026, NVIDIA followed with Nemotron 3 Nano Omni. It extends the architecture to images, video, speech, and text, allowing multimodal information to remain inside one context across an agent loop.

Nemotron 3 Nano is useful for workloads that need long context and stronger reasoning without activating a conventional dense 30B model for every token.

The Omni version is particularly relevant for multimodal agents that need to interpret documents, screens, video, or spoken input without combining several separate perception and language models.

Both models are designed to work closely with NVIDIA’s inference and agent infrastructure.

“3B active parameters” does not mean the model has the memory footprint of a dense 3B model. The complete expert weights still need to be stored, so deployment can require substantially more memory.

NVIDIA’s strongest efficiency claims are also measured on its own optimized hardware and software stack. Performance may differ on other hardware or through less mature runtimes.

Microsoft’s Phi-4 family has expanded from compact text and mathematical reasoning models into multimodal systems.

Phi-4-mini-reasoning is a 3.8B-parameter model designed for mathematics and structured reasoning. Phi-4-multimodal supports text, audio, and vision, including speech recognition, translation, OCR, chart understanding, and image analysis.

In March 2026, Microsoft released Phi-4-reasoning-vision-15B, a compact open-weight model built for visual and structured reasoning. It is designed for tasks such as interpreting documents, solving math and science problems from images, understanding interfaces, and grounding elements on computer or mobile screens.

Phi models are strongest when used as compact specialists rather than generic replacements for every larger model.

Phi-4-mini-reasoning is relevant for math tutoring, structured problem solving, and educational tools. Phi-4-reasoning-vision is more appropriate for document understanding, visual agents, interface navigation, and systems that combine perception with multi-step reasoning.

The Phi-4 family contains several models with very different capabilities. A model optimized for mathematics may not be the best choice for creative writing, multilingual communication, or broad factual questions.

Reasoning models can also generate long internal and final outputs. This may increase latency enough to cancel some of the expected benefits of a compact model.

→ Read about Phi-4-reasoning-vision

Cohere Labs released Tiny Aya in February 2026 as a 3.35B-parameter multilingual model family designed to run locally, including on mobile devices.

Tiny Aya supports more than 70 languages and is available as a globally balanced model and as region-specialized variants. Tiny Aya Earth focuses on West Asian and African languages, while other versions are designed around different language regions.

The models have an 8K context window and are released under the CC-BY-NC-4.0 license.

Tiny Aya is one of the most interesting small models for multilingual applications. Most model benchmarks still concentrate heavily on English, while Tiny Aya is explicitly designed around real-world language diversity.

It is suitable for local translation, multilingual assistants, educational tools, communication applications, and services for regions that are poorly represented in mainstream model development.

The license restricts commercial use. Companies should not treat open weights as permission to use the model in any product without reviewing the license.

Its 8K context window is also much shorter than those of many competing models. Tiny Aya is designed for compact multilingual interactions, not extremely long document analysis.

→ Explore the Tiny Aya models

IBM released Granite 4.1 in April 2026 as a family of dense language models in 3B, 8B, and 30B sizes. Each is available in base and instruction-tuned versions, with optional FP8 quantization.

The new family improves on Granite 4.0 in instruction following, tool calling, coding, and mathematical reasoning. IBM positions Granite as an enterprise model family, with additional Granite 4.1 models for vision, speech, embeddings, and safety.

One notable design goal is predictable inference. IBM emphasizes strong instruction and tool-calling performance without depending on extremely long reasoning traces for every answer.

Granite 4.1 is well suited to enterprise assistants, internal tools, document processing, coding workflows, structured extraction, and systems that need reliable tool use.

The availability of related vision, speech, embedding, and Guardian models also makes Granite useful for teams that want to build a broader enterprise AI stack around one model family.

Granite is optimized around enterprise workloads, so benchmark strength on tool calling or structured tasks does not automatically mean it will be the best general conversational model.

The family also contains many specialized components. Teams should choose the exact language, vision, speech, embedding, or safety model they need rather than referring to “Granite” as if it were one model.

→ Explore the Granite 4.1 documentation

Alibaba’s Qwen3 family includes dense models with 0.6B, 1.7B, 4B, and 8B parameters, alongside much larger dense and mixture-of-experts models.

The original Qwen3 release supports 119 languages and dialects. The models can switch between thinking and non-thinking modes, giving developers control over the trade-off between deeper reasoning and faster responses.

The small models are released under the Apache 2.0 license and are widely supported across local inference tools such as Ollama, llama.cpp, LM Studio, vLLM, and SGLang.

Qwen3 offers one of the broadest ranges of compact model sizes. The 0.6B and 1.7B models can handle lightweight classification, extraction, rewriting, and local assistant tasks, while the 4B and 8B versions provide stronger reasoning and instruction following.

Its multilingual coverage also makes it relevant for applications that need to operate across more than a handful of major languages.

Thinking mode can significantly increase output length, latency, and compute use. It should be enabled because a task requires it, not because “thinking” sounds reassuring.

Performance also varies by language and use case. Teams working in sensitive or regulated environments should independently test safety behavior, factual reliability, and regional compliance.

→ Explore the official Qwen3 repository

Hugging Face released SmolLM3 as a fully open 3B-parameter model trained on 11 trillion tokens.

It supports reasoning and non-reasoning modes, six languages, and long-context tasks. The model was trained with a 64K context window and can support up to 128K tokens through YARN extrapolation.

Hugging Face published the weights along with extensive information about the public data mixture, architecture, training configuration, and post-training process.

/ For more on how Hugging Face develops compact models through curated datasets and multi-stage training, read Inside the Family of Smol Models.

SmolLM3 is especially useful for researchers and engineering teams that want transparency and control.

It can be inspected, fine-tuned, deployed locally, and used as a foundation for narrow domain models. Its 3B size also makes experimentation more accessible than with larger open-weight models.

SmolLM3 officially focuses on English, French, Spanish, German, Italian, and Portuguese. Performance outside those languages may be inconsistent.

Its 128K context support relies on extrapolation beyond the 64K training context. That does not mean every long-context task will work equally well across the full window.

→ Read the SmolLM3 technical overview

Cohere released North Mini Code in June 2026 as its first open agentic coding model.

North Mini Code uses a mixture-of-experts architecture with 30B total parameters and 3B active during inference. It is designed for repository-level code changes, terminal-based agents, software engineering workflows, scientific coding, and local development.

The model is released under the Apache 2.0 license and is intended for private or sovereign deployment.

North Mini Code is a strong option for teams that want to run coding agents inside their own infrastructure.

Its relatively small active footprint makes it more practical for local agentic coding than a dense 30B model. It is designed to work inside harnesses that can inspect repositories, edit files, run commands, test changes, and iterate over failures.

A 30B MoE model still needs to store its full weights. The 3B active count describes inference computation, not the complete hardware requirement.

It is also a specialist coding model. Its inclusion makes sense for software agents, but it should not be treated as a general-purpose assistant simply because it can communicate in natural language.

→ Explore the North Mini Code documentation

Meta’s Llama 3.2 1B and 3B models remain useful for mobile assistants, summarization, rewriting, and lightweight local applications.

But they were released in September 2024. They no longer belong in a list focused on new or leading small models in 2026, especially when newer families offer stronger reasoning, multilingual support, tool use, or multimodal capabilities.

Llama 3.2 is now better understood as an established deployment option rather than a new model to watch.

→ Read Meta’s Llama 3.2 announcement

Before selecting a model, test:

A smaller model is not automatically the cheaper model. A compact reasoning model that generates thousands of tokens may consume more compute than a larger model answering directly.

/ For a closer look at caching, quantization, speculative decoding, routing, and the economics behind model serving, read AI Inference Cost: How Teams Cut Model Spend in 2026

The best SLM is the smallest model that can reliably complete your task within your quality, latency, privacy, and cost requirements.

They are certainly part of it.

The future is unlikely to be one small model replacing one large model. More likely, AI systems will route different tasks to models with different sizes and capabilities.

A frontier model may plan a workflow, while smaller models handle retrieval, classification, extraction, coding edits, safety checks, or local interactions. On-device models can protect sensitive information and continue working without a network connection. Enterprise systems can send repetitive requests to efficient models instead of paying frontier-model prices for every step.

Small language models are becoming less like limited versions of larger models and more like specialized components of advanced AI systems. That is a much more useful role.

🎙️ Deep dive on Phi-3 and synthetic data: Small Language Models: The Future of AI?

A small language model, or SLM, is an AI model designed to perform language, reasoning, coding, retrieval, or multimodal tasks with lower compute and memory requirements than frontier-scale models. There is no fixed parameter limit: SLMs can range from under one billion parameters to larger mixture-of-experts models that activate only a small portion of their total parameters during inference.

Examples of small language models include Microsoft Phi-4, Google Gemma 4, Mistral’s Ministral 3, Hugging Face SmolLM3, Alibaba’s smaller Qwen3 models, and NVIDIA Nemotron 3 Nano. Proprietary models such as GPT-5.4 mini and nano also belong to the broader small-model category, although OpenAI does not disclose their parameter counts.

The main difference between an SLM and an LLM is the trade-off between capability and efficiency. Large language models generally perform better on complex, open-ended tasks, while small language models are faster, cheaper to run, easier to customize, and more practical for local or edge deployment. An SLM can outperform a larger model on a narrow task if it has been trained or fine-tuned specifically for that domain.

Small language models are used for classification, summarization, extraction, translation, retrieval-augmented generation, coding assistance, document processing, tool calling, and on-device assistants. They are also increasingly used as specialized components inside agentic systems, where different models handle routing, retrieval, safety checks, or narrow domain tasks.

Yes. Many small language models can run locally on laptops, smartphones, workstations, or edge devices, depending on the model size, quantization level, inference runtime, and available memory. Local deployment can reduce latency, protect sensitive data, lower cloud costs, and allow applications to work without a constant internet connection.

The best small language model depends on the task. GPT-5.4 mini is a strong managed option for coding and tool use. Gemma 4, Ministral 3, and Nemotron 3 Nano support local multimodal or agentic applications. Phi-4 is strong in mathematics and visual reasoning, Tiny Aya focuses on multilingual use, Granite 4.1 targets enterprise workflows, and Qwen3 and SmolLM3 are flexible choices for open local deployment.

The phrase “four models of AI” usually refers to four types of artificial intelligence based on capability and behavior: reactive machines, limited-memory AI, theory-of-mind AI, and self-aware AI. Most current AI systems, including language models, fall into the limited-memory category. Theory-of-mind and self-aware AI remain theoretical rather than established technologies.

ChatGPT is based on deep learning, which is a subset of machine learning. It uses transformer neural networks trained on large amounts of data to predict and generate text. The broader system may also use reinforcement learning, human feedback, retrieval, tools, and other techniques, but its core language-model technology is deep learning.

Small language models are not universally better, but they can be better for specific workloads. They usually offer lower latency, lower inference costs, easier local deployment, and greater privacy. Large language models remain stronger for difficult reasoning, broad knowledge, and complex open-ended tasks. Many production systems use both, routing each request to the smallest model capable of completing it reliably.