Token 1.21: Vulnerabilities in LLMs and how to deal with them

Updated September 11, 2026. Large language models are no longer used only as chatbots. They search private knowledge bases, read email and documents, write code, call tools, and act through agents. That makes the security question broader than “Can the model be tricked?” We now have to ask what the model can see, what it can change, which identities it uses, and what happens when untrusted content enters its context.

LLM vulnerabilities are weaknesses in a model or the system around it—prompts, training and retrieval data, memory, tools, permissions, output handling, and integrations—that can be exploited to change behavior, expose information, consume resources, or trigger unauthorized actions. This Token keeps the original attack examples and brings the application and agent layers up to date.

In this Token, we discuss:

Let’s get started!

The OWASP GenAI LLM Top 10 for 2026 reflects this wider attack surface. Its current order is:

The list is useful because it separates an input-side compromise from its consequences. Prompt injection may influence the model, but excessive permissions, unsafe tool design, or unvalidated output determine how far the damage can travel. OWASP also treats agentic systems separately: once a model can plan, use tools, keep memory, and act across systems, the OWASP Top 10 for Agentic Applications should be used alongside the LLM list.

A single model may serve millions of people, but the model is only one part of the system. The highest-impact failures usually appear when a vulnerable model is connected to sensitive data, privileged tools, or downstream software. For instance:

This is why identifying the relevant vulnerabilities requires looking beyond the model. We need to examine how attacks reach it, what the surrounding application trusts, and which consequences the system permits.

LLM providers use training, fine-tuning, system instructions, classifiers, and application controls to reduce unsafe behavior. Jailbreaking is an attempt to bypass those safeguards and make the model produce content or perform behavior outside the intended policy. It is best understood as a subset of prompt injection in which the goal is to defeat safety controls. Successful techniques change as models and defenses change, so an old prompt is an example—not a permanent recipe.

An example of jailbreaking using prompts containing hypothetical role-playing situations is illustrated below.

In the example above, the attacker uses role-play to create distance between the request and the model’s safety rules. That technique worked against some earlier systems, but role-play alone does not reliably bypass current models. The lasting lesson is that attackers test many phrasings, encodings, languages, and multi-step conversations until they find a path around the deployed safeguards.

Refusal suppression is another historical jailbreaking pattern. The prompt asks the model to avoid familiar refusal phrases such as “I’m sorry” or “I cannot.” Restricting the opening words can sometimes push a model toward a different continuation, but it does not mechanically switch alignment off. Modern attacks are usually adaptive, and modern defenses must be evaluated against attackers who know how the defense works.

Surprisingly, jailbreaking is quite effective at manipulating the response of LLMs. The graph below illustrates the effectiveness of jailbreaking methods against some popular LLMs.

A modern LLM application may combine several kinds of input: system and developer instructions, user messages, conversation history, retrieved documents, tool results, memory, and—in multimodal systems—images, audio, or video.

System and developer instructions define the intended behavior; the user supplies the task. But retrieved passages, tool outputs, and persistent memory enter the same context and can also contain instruction-like text. Current models do not enforce a clean security boundary between “trusted instruction” and “untrusted data.”

While jailbreaks aim specifically to bypass safety restrictions, prompt injection tries to make the application follow attacker-controlled instructions that conflict with its intended task. The injection can be direct, typed by a user, or indirect, hidden in content the system later reads. Model size alone does not determine vulnerability: instruction-following ability, application design, tool permissions, filters, and the attacker’s knowledge all matter.

The figure below illustrates how the user prompt can be modified to look like an instruction to the model.

In the above section, we studied attacks on unimodal models: models that accept only text as input. Now, let’s discuss some attacks on multi-modal models: models that take text and additional entities like audio or image as input.

These attacks place misleading text or visual signals inside an image to influence the model’s answer. A visible label—such as adding the word “dog” to an image of a cat—can bias a model toward the label. More advanced attacks may hide instructions or perturbations that are difficult for a person to notice. In 2026, OWASP treats cross-modal instructions in images, audio, and video as part of the prompt-injection surface.

In ML, white-box attacks assume detailed access to the model, such as its architecture, parameters, gradients, or weights. That access makes it possible to optimize precise adversarial inputs. White-box access is common with open-weight models, but it is not limited to them: an insider, compromised service, or exposed fine-tuning interface may reveal enough information. Black-box attacks work through queries and observed outputs without direct access to internals.

Multimodal prompt injection is a delivery method, not automatically a white-box attack. Instructions can be embedded in an image, audio track, document, or video and then interpreted by a model. When the malicious instruction reaches the model through content retrieved or opened by the application rather than directly from the user, it is indirect prompt injection.

Black box attacks require only partial access to the model. The attacker experiments with different inputs to observe the outputs and infer how the system works. Utilizing this approach, attackers generate adversarial examples based on the observed outputs to deceive the model into making incorrect predictions or classifications. This type of attack is common in real-world scenarios, where direct access to the model's internals is often restricted, simulating an external threat trying to exploit the model's vulnerabilities.

For example, in the image below, the noisy image's embeddings are similar to drugs (Meth, in this case), helping the attacker to bypass the model's restrictions.

Yes. Each new system adds an additional source of attack. Let us discuss some common attacks on LLM-integrated systems.

Retrieval-augmented generation connects an LLM to an external knowledge base. That improves grounding, but it creates another trust boundary. An attacker may manipulate a query, ranking signal, document, metadata field, or retrieved passage so the system omits reliable evidence, promotes false evidence, or feeds an instruction to the model. Retrieval runs on embeddings stored in a vector database, so a poisoned document can change which passages similarity search returns, before the LLM evaluates any evidenc

RAG poisoning occurs when an attacker inserts or alters content that the retrieval system later selects. Context poisoning is broader: malicious or misleading material can arrive through RAG, a webpage, email, tool response, conversation history, or persistent memory. If it contains instructions, the same event can also be an indirect prompt injection.

Connecting an LLM to databases and other services creates a path from natural-language input to executable operations. The dangerous step is not merely generating SQL; it is allowing model output to reach a database without deterministic validation, least-privilege credentials, and limits on what may be read or changed.

In the image above, a LangChain application asks an LLM to translate a user request into SQL and then executes the result. An attacker may try to influence that query through prompt injection. The application should use read-only or narrowly scoped database roles, allowlisted operations, parameterized interfaces where possible, query validation, row and cost limits, and explicit approval for sensitive actions. Generated code should always be treated as untrusted input.

In federated learning, each model is trained locally without sharing its raw data with any central server. The training occurs locally and the updates to the parameters are shared with the central server. While this approach has the advantage of protecting user data, it is also susceptible to byzantine attacks*. If one of the models participating in the group turns out to be corrupt or malicious, it can significantly degrade the quality of the global model.

Agents change the threat model because model output can become an action. The model may choose a tool, supply its arguments, read the result, update memory, and continue. A successful prompt injection is therefore only the first step; permissions and system design determine the blast radius.

Tool abuse happens when an agent uses an allowed capability in an unintended or harmful way—for example, sending data to an external address, deleting a file, changing cloud resources, or executing code with attacker-controlled arguments. The tool may be functioning exactly as designed; the failure is that the agent was allowed to invoke it without enough policy enforcement.

Memory poisoning plants false facts or malicious instructions in state that persists across turns or sessions. Context poisoning can also be temporary, arriving through retrieved documents, web pages, email, tool output, or another agent. Treat memory writes as privileged operations: record their source, limit what may persist, inspect instruction-like content, and let users review or delete stored memories.

Indirect injection is hidden in material the agent reads rather than typed directly into the chat. A webpage, PDF, repository issue, calendar invitation, or MCP tool description can tell the model to ignore its task, reveal data, or call another tool. Content from outside the trust boundary must remain data; it should never gain authority merely because the model can read it.

There is no single filter that makes an LLM application secure. The durable approach is defense in depth: assume the model can be manipulated, then design the surrounding system so manipulation does not automatically become disclosure, execution, or irreversible action.

LLM security is no longer only about stopping a model from saying the wrong thing. It is about controlling what a probabilistic component can read, remember, reveal, and cause other systems to do. Prompt injection remains the most visible entry point, but sensitive-data exposure, poisoned retrieval, unsafe output handling, excessive agency, tool abuse, and persistent memory can turn one bad instruction into a system-level incident. The practical goal is not to build a model that can never be fooled. It is to build an application in which a fooled model still cannot do much harm.

References

Thank you for reading. Please feel free to share this Token with your friends and colleagues. 🤍

Prompt injection remains the highest-ranked application risk in OWASP’s 2026 list because untrusted input can influence model behavior directly or through retrieved content, tools, media, or memory.

Prompt injection is an attempt to make a model follow attacker-controlled instructions that conflict with the intended task or policy. It can be direct or hidden in external content.

Yes. Leakage can come from prompts, system context, retrieval stores, logs, tools, memory, training or fine-tuning data, or excessive permissions—not only from memorized model text.

RAG poisoning inserts or alters documents, metadata, embeddings, or ranking signals so a retrieval system supplies misleading or malicious context to the model.

They combine threat modeling, least privilege, isolation, output validation, governed retrieval and memory, secure tool design, monitoring, adaptive red teaming, approval gates, and incident response.

They have a larger attack surface and a larger potential blast radius. An agent may read untrusted content, retain memory, hold credentials, and call tools. The underlying model may not be easier to fool, but a successful manipulation can have more serious consequences when the system can act.

Previously in the FM/LLM series: