Token 1.7: What Are Chain-of-Verification, Chain of Density, and Self-Refine?

You’ve likely heard about Chain-of-Thought (CoT) prompting methods in Large Language Models (LLMs), which have ignited an entirely new area of study. This, in return, has led to a myriad of "chain" spin-offs and related research in "‘chain reasoning.’ We diligently covered the multifaceted world of CoT (do check it out!).

But what about other concepts that are related to machine learning, have 'chain' in their name but hail from a different universe?

In this Token, we aim to name and clarify these other “chain” concepts to complete your vocabulary and make your comprehension crystal clear. We will:

Hope this adds some clarity!

Update 2026

Since this article was published, chaining has evolved far beyond prompt engineering. It now underpins AI agents, coding assistants, retrieval pipelines, multi-agent systems, and autonomous workflows. Modern frameworks such as LangGraph, OpenAI Responses API workflows, Google’s ADK, Microsoft’s AutoGen, CrewAI, and many enterprise agent platforms all rely on chaining as a core architectural pattern.

You've likely come across LangChain, the new framework for building LLM chains and related applications. Curious about the origins of this chaining concept and why it's important? Let’s dive in.

In Token 1.5, we've explored the concept of chain-of-thought prompting, designed to elicit complex reasoning from LLMs. While revolutionary, it wasn't without precedent. A pivotal paper from 2021 titled "AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts" made a seminal contribution to this arena. The paper's authors demonstrated that "Chaining," a methodology that connects multiple tasks within LLMs, significantly enhances user interaction in several ways:

By employing Chaining, the research suggests that LLM’s limitations can be mitigated, thereby improving models’ utility and reliability. Two case studies further explored how this method could be applied in real-world scenarios, indicating a promising avenue for future LLM applications.

This paper follows up on the work we’ve just discussed. While the previous paper defined the concept of LLM Chains, the process of authoring these chains remained a challenge. It involved not just individual LLM prompts but also a nuanced understanding of how to decompose the overarching task.

We’ve explained the chaining mechanism behind such frameworks as Langchain. We will cover this and other frameworks in more detail in a separate Token.

Learning from how humans refine their written text, researchers introduced a novel algorithm, Self-refine, that improves the quality of the output of LLMs, particularly in tasks requiring intricate reasoning or multifaceted objectives. Unlike existing models that require human annotations or extensive training sets for refinement, the authors of self-refine suggested relying solely on the LLM's capability for self-feedback!

The self-refine algorithm consists of four steps:

The paper demonstrates the extensibility of the self-refine algorithm across a range of tasks, from natural language to code generation. It showed a significant performance boost of base models, exemplified by an 8.7% absolute improvement in Code Optimization for GPT-4. Self-refine operates in either a zero-shot* or few-shot* manner, depending on the specific task.

Another approach to improve reasoning abilities in LLMs and fight hallucinations is Chain-of-Verification (CoVe). In the paper, ‘Chain-of-Verification Reduces Hallucination in Large Language Models’, researchers from Meta AI, showed how CoVe first generates a draft response and then produces a list of verification questions to self-examine that draft. It proceeds to answer these questions, cross-references them with the initial response for inconsistencies, and finally produces a refined output.

This self-verification process improves the model's performance by reducing hallucinations. While the baseline LLMs are not completely reliable, they can be directed through prompts to execute this four-step verification process effectively.

The CoVe methodology consists of four key steps:

The research explores variations in the third step, leading to more effective verification mechanisms. These variations can either use single, dual, or independent prompts for each question, with the latter showing the most promise in improving model output.

The CoVe approach relies on the base LLM’s ability to follow instructions in a few-shot or zero-shot manner. It leverages the LLM’s inherent capabilities to generate and execute a self-verification plan. Though CoVe doesn’t remove hallucinations completely from generation, it significantly enhances the model’s performance and might be further improved with such tools as RAG.

One of the major vulnerabilities of Chain-of-Thought prompting is its sensitivity to errors. Even minor mistakes can trigger a cascade of inaccuracies in subsequent reasoning steps. This issue is particularly pronounced in multi-step tasks requiring precise logic or calculations.

The paper ‘Large Language Models are Better Reasoners with Self-Verification’ addresses these challenges by introducing a dual-stage reasoning strategy: Forward Reasoning and Backward Verification.

This approach allows LLMs to verify their reasoning. The paper claims that this self-verification process negates the need for manual annotations, enabling the technique to be easily adapted to various domains and tasks. Self-verification method excels particularly in arithmetic reasoning tasks. These tasks saw higher average performance improvement (1.67%/2.84% ↑) compared to other reasoning tasks (0.62%/0.78% ↑) Another interesting conclusion is that even limited computational resources can yield performance gains.

Chain of Density (CoD) is a prompting technique proposed in the paper "From Sparse to Dense: GPT-4 Summarization with Chain of Density Prompting" to generate increasingly dense entity-based summaries using GPT-4 while controlling for length.

The CoD prompt has GPT-4 iteratively identify and fuse missing entities into an initial sparse summary without increasing the overall length. This forces abstraction, compression, and fusion to incorporate more details.

The methodology involves:

Analysis shows CoD summaries become more abstractive, exhibit increased fusion, and reduce lead bias compared to vanilla GPT-4 summaries. Human evaluation indicates intermediate CoD steps with higher density (~0.15 entities/token) are preferred over sparser or denser summaries.

The paper shows how CoD prompting effectively balances coherence and informativeness in fixed-length news summaries, with potential applications in other domains beyond news.

The researchers also release 500 annotated and 5000 unannotated CoD summaries on Hugging Face to promote further research into controlled density summarization.

That’s all with Chains for today (and for this series!). If you want to follow the development of the papers we mentioned above, you might be interested to check the following papers.

Thank you for reading, please feel free to share with your friends and colleagues. In the next couple of weeks, we are announcing our referral program 🤍

Previously in the FM/LLM series:

Self-Refine is a prompting technique in which an LLM improves its own output through multiple iterations. The model first generates an answer, critiques it, and then revises it based on its own feedback. This process can be repeated several times to improve quality without additional model training.

Yes, to a degree. Prompting techniques such as Chain-of-Verification (CoVe) and Self-Verification encourage LLMs to check their own reasoning before producing a final answer. While these methods can reduce hallucinations and improve reliability, they cannot guarantee correctness because the model may still verify incorrect assumptions.

AI model chaining is the practice of connecting multiple LLM calls or AI components into a workflow, where each step performs a specific task before passing the result to the next. Chaining is widely used in AI agents, retrieval-augmented generation (RAG), coding assistants, planning systems, and multi-agent applications to solve more complex problems than a single prompt can handle.

AI chaining executes a predefined sequence of model calls or tools, while AI agents can dynamically decide which actions, tools, or chains to use based on the current task. In practice, most modern AI agents are built from multiple interconnected chains.

No. LLM chaining is a general architectural concept of connecting multiple AI steps into a workflow. LangChain is one framework that implements this idea, alongside newer frameworks such as LangGraph, AutoGen, CrewAI, Google's ADK, and OpenAI's workflow tools.