Quick answer
Improper output handling is when an application passes a language model's output to a browser, database, shell, API or other component without validating it first. Because attackers can influence what the model says, unfiltered output turns into classic attacks such as cross-site scripting, SQL injection and remote code execution. It is ranked fifth in the OWASP Top 10 for LLM Applications (2025).
On this page
How It Works
Developers often treat model output as trusted because it comes from their own system. In reality it is shaped by user input, retrieved documents and web content, any of which can carry a prompt injection. If the chat interface renders model output as HTML, injected script becomes cross-site scripting. If the model writes SQL that is executed directly, it becomes SQL injection. If its output is used as a URL the server fetches, it becomes server-side request forgery. If it generates code that is run, or file paths and shell commands that are used as given, the result can be full code execution on the server. Markdown rendering is a common case: an injected image tag can make the user's browser send conversation data to an attacker's server.
Why It Matters
This is where AI risk turns into conventional, well-understood impact. A text-to-SQL feature, an agent that runs code, or a support bot whose answers are inserted into emails and web pages all pass model output across a trust boundary. The vulnerability is in the application rather than the model, so it is fixable with proven controls, but only if the model is treated as an untrusted user.
How to Test for It
- Trace every place model output goes: rendered in a browser, stored, executed, used in queries, URLs, file paths, emails or calls to other systems.
- Use direct and indirect prompt injection to make the model emit payloads for each destination: script tags, SQL fragments, internal URLs, path traversal sequences, shell metacharacters.
- Check whether markdown images and links render automatically and whether they can carry data to external domains.
- Test code execution features for sandbox escape, network access and access to secrets.
- Confirm that structured outputs, such as JSON for tool calls, are validated against a schema before use.
How to Reduce the Risk
Treat model output exactly like untrusted user input. Apply context-aware output encoding before rendering, use parameterized queries and read-only database roles for text-to-SQL, validate structured output against strict schemas, allowlist URLs and domains for any fetch, run generated code in an isolated sandbox with no secrets and limited network access, apply a strict content security policy, and block automatic loading of external images in chat interfaces.
Regulatory and Framework Context
In the US, CISA's Secure by Design principles call for eliminating whole classes of vulnerability such as cross-site scripting and SQL injection, which applies equally when the input comes from a model. Testing provides evidence for the Measure function of the NIST AI RMF.
Latest intel on this topic
- Video: What is Tabnabbing?Embrace The Red · October 10, 2021
- Video: Web Application Security FundamentalsEmbrace The Red · September 6, 2021
- Web Application Security Principles RevisitedEmbrace The Red · February 13, 2020
Updated hourly from the daily briefing.
Common questions
Is improper output handling the same as prompt injection?
No. Prompt injection controls what the model says. Improper output handling is the application trusting what the model says. The two are usually chained together.
Do output guardrails solve it?
Content filters help, but the reliable fix is the same as for any untrusted input: encoding, parameterized queries, schemas and sandboxing at each destination.
Change log
- November 13, 2026First published.