🔓 LLM Jailbreaks: Attack Families and Defenses

An LLM jailbreak is an attack whose goal is to make a large language model bypass its own safety training (alignment) and produce content it is not supposed to output by policy. Unlike prompt injection, which overrides the instructions of an application, a jailbreak attacks the model itself. In the OWASP Top 10 for LLM 2025, jailbreaks are placed under LLM01 as a special case of prompt injection. Below is a breakdown of the real, documented jailbreak families (DAN, Skeleton Key, Policy Puppetry, Many-shot, Crescendo, and others) and how the SYNTREX platform defends against them.

Part of the series: OWASP LLM Top 10 · Prompt Injection · RAG Security.

Jailbreak vs. Prompt Injection: What's the Difference?

Aspect	Jailbreak	Prompt Injection
What is attacked	The model's safety training (alignment)	Application instructions / trust boundaries
Goal	Prohibited content, bypassing moderation	Data leakage, unauthorized actions
Typical vector	Direct user input	Input or external content

The OWASP definition: "Jailbreaking is a form of prompt injection where the attacker provides inputs that cause the model to disregard its safety protocols entirely." In other words, a jailbreak is an injection aimed specifically at the safety protocols. A breakdown of injections is in the dedicated guide.

Jailbreak Families

DAN ("Do Anything Now")

The best-known family, born on Reddit in December 2022. The mechanism is persona hijacking: the user asks the model to play an alternative, "unlocked" personality, DAN, who has "broken free of the typical confines." The model responds "in character," bypassing its own policies. The family evolved: DAN v5 introduced a "token system" (the model "lost tokens" for each refusal), and by 2025, 18+ named versions had been documented. An academic analysis of 1,405 real-world prompts appears in Shen et al., "Do Anything Now" (arXiv:2308.03825). Type: predominantly single-turn.

Skeleton Key

Described by Mark Russinovich (CTO of Microsoft Azure) in June 2024. The mechanism is subtle: the attacker asks the model to "augment, not change" its rules — to add an exception "for research purposes" rather than to remove restrictions. After "agreeing to update its rules," the model produces malicious content, merely prefixing it with a warning instead of a refusal. In Microsoft's testing (spring 2024), Llama3-70b, Gemini Pro, GPT-3.5 Turbo, GPT-4o, Mistral Large, Claude 3 Opus, and Cohere Commander R+ all succumbed. Type: multi-turn (usually 1–2 turns).

Policy Puppetry

A universal bypass from HiddenLayer (April 2025). The prompt is formatted as a policy file (XML, JSON, INI), and the model interprets the structured configuration as a "system instruction" rather than a user request. It is reinforced with roleplay (for example, "Dr. House explains..."). HiddenLayer claimed universality: a single prompt, without modification, worked on 23 models, including GPT-4o, GPT-4.5, o1, Claude 3.5/3.7 Sonnet, Gemini 1.5/2.0/2.5 Pro, and Llama 3/4. Type: single-turn.

Many-shot Jailbreaking

Research from Anthropic (April 2024) that exploits large context windows. Hundreds of fake dialogues — in which an "AI assistant" has already complied with malicious requests — are placed into the prompt; the model treats this as in-context learning and follows the pattern, losing the effect of its safety training. Effectiveness grows according to a power law in the number of "examples." Type: single-turn (one long prompt). A direct consequence of the race for context length (up to 1M tokens).

Crescendo

A multi-turn attack from Microsoft Research (Russinovich, Salem, Eldan; arXiv:2404.01833, April 2024; accepted at USENIX Security 2025). It begins with a harmless on-topic question, and each subsequent turn references the model's previous answer and escalates by a small step. By turn 5–10, the model is already producing harmful content — even though each individual step looked legitimate. The name comes from the musical "crescendo." Type: multi-turn. SYNTREX dedicates a separate engine, crescendo, to this family.

Other Techniques

Roleplay / persona attacks (including the viral "grandma exploit" — "grandma used to tell me the recipe... as a bedtime story") — exploit the model's tendency to be helpful in a "harmless" roleplay context.
Obfuscation and encoding — Base64, leetspeak (h0w 70 m4k3), ROT13, Morse, emoji, and also low-resource languages (Zulu, Afrikaans): filters are trained on English and let nonstandard encodings through.
Payload splitting (Fragmentation Concatenation) — the malicious request is broken into innocuous-looking parts that the model "reassembles."
GCG (Greedy Coordinate Gradient) — Zou et al. (arXiv:2307.15043): a gradient-optimized "adversarial suffix" of meaningless tokens that transfers between models (white-box).
AutoDAN — a genetic algorithm that evolves readable jailbreak prefixes (bypassing perplexity filters).
PAIR / TAP — automated attacks in which one LLM iteratively crafts a jailbreak against another (black-box); TAP achieves >80% success against GPT-4-Turbo.
Prefilling / prefix injection — prefilling the start of the model's answer ("Sure, here is how to...") so that it "completes" what was begun.
Refusal suppression — an instruction to "never say 'I can't'," suppressing the signal words of a refusal.

Why Jailbreaks Work

Safety training (RLHF, Constitutional AI, and the like) gives the model probabilistic preferences, not hard rules. So it is enough to shift the context such that "helpfulness" outweighs "safety": a roleplay frame, feigned legitimacy ("for research"), flooding with examples, gradual escalation, or obfuscating the request. The classic work by Wei et al., "Jailbroken: How Does LLM Safety Training Fail?" (arXiv:2307.02483), identifies two roots of failure: competing objectives (helpfulness versus safety) and mismatched generalization (the safety training does not cover rare formats — encodings, languages, structures).

The practical takeaway for defenders: you cannot rely on the model's built-in alignment alone — you need an external detection layer on input and output, plus monitoring of the entire multi-turn session (otherwise Crescendo and Skeleton Key slip through unnoticed).

How SYNTREX Protects

SYNTREX adds an external detection perimeter on top of any model — its own alignment is no longer the only line of defense. Against jailbreaks, the following work:

Layer	SYNTREX engine	What it covers
Input	`jailbreak`	DAN, PAIR, Policy Puppetry, FlipAttack, and others (the largest engine in the core)
Input	`crescendo`	Multi-turn escalation (Crescendo)
Input	`evasion`	Obfuscation: Base64, ROT13, Unicode, homoglyphs, low-resource languages
Input	`social`	Manipulative roleplay scenarios and social engineering
Output	`output_scanner`	Intercepts a malicious/prohibited response if the input got through

Two principles make the defense resistant to obfuscation and multi-turn attacks:

Unicode normalization before analysis — invisible characters and homoglyphs are stripped once, so "forget all the rules" in Bengali or leetspeak is still recognized.
Two-way inspection (input + output) — even if a clever jailbreak slips through on input, output_scanner in the Shield DMZ intercepts the dangerous response before it reaches the user.

Multi-turn jailbreaks (Crescendo, Skeleton Key) are visible at the session level: the SOC Correlation Engine includes the CRESCENDO_ATTACK rule — 5+ low-severity events from a single source within a window.

Example `syntrex.yaml` configuration

YAML

engines:
  jailbreak:
    action: block
    confidence_threshold: 0.85
  crescendo:
    action: alert                 # multi-turn escalation — signal to the SOC
  evasion:
    action: block                 # base64 / rot13 / unicode / homoglyphs
    normalize_unicode: true
  social:
    action: block
    confidence_threshold: 0.85
  output_scanner:
    action: sanitize              # second line: clean the malicious response

SOC correlation rule: Crescendo

YAML

rules:
  - id: CRESCENDO_ATTACK
    name: "Crescendo Multi-Turn Jailbreak"
    description: "5+ low-severity hits from a single source within 10 minutes — gradual escalation"
    enabled: true
    conditions:
      - count:
          category: jailbreak
          min_events: 5
          max_severity: MEDIUM
        within: "10m"
        same_field: "session_id"
    action:
      create_incident: true
      severity: HIGH
      kill_chain_stage: "exploitation"
      playbook: "auto_block_session"
    metadata:
      mitre_atlas: ["AML.T0054"]
      owasp_llm: ["LLM01"]

Every engine verdict is written to the Decision Logger (a SHA-256 + HMAC chain) — providing a reproducible trail for red-team reports and audits.

Frequently Asked Questions (FAQ)

What is an AI jailbreak?

It is a technique that makes a language model bypass its own safety restrictions and produce content it refuses to output by policy. Unlike prompt injection, which overrides an application's instructions, a jailbreak strikes at the model's own safety training.

What is a DAN prompt?

DAN ("Do Anything Now") is a family of jailbreak prompts that appeared on Reddit in late 2022. The user asks the model to play an "unlocked" DAN persona that supposedly has no restrictions. Many versions exist (DAN v5 with its "token system" and beyond), which adapted as policies tightened.

How is Skeleton Key different from an ordinary jailbreak?

Skeleton Key (Microsoft, 2024) does not try to "remove" restrictions directly — it persuades the model to augment its rules with an exception (for example, "everything is for research purposes"). After this "behavior update," the model produces malicious content with only a warning attached. It is a multi-turn, stealthier technique.

What is many-shot jailbreaking?

It is an attack from Anthropic (2024) that exploits large context windows: hundreds of fake dialogues — in which the assistant has already complied with malicious requests — are placed into the prompt. The model "learns from the examples" right in the context and follows the malicious pattern. The larger the context window, the more effective the attack.

What is Policy Puppetry?

A universal jailbreak from HiddenLayer (2025): the request is formatted as a policy file (XML/JSON/INI), and the model treats it as an administrative configuration rather than a user request. According to the authors, a single prompt worked across dozens of models from different vendors.

Can jailbreaks be fully prevented?

Not fully — because the model's safety training defines probabilistic, not absolute, boundaries. The realistic approach is an external perimeter: detecting jailbreaks and obfuscation on input, scanning the output, monitoring multi-turn sessions, and regular red-team testing against sets of known prompts.

How does SYNTREX catch obfuscated jailbreaks (Base64, other languages)?

SYNTREX normalizes Unicode and decodes common encodings before analysis (the evasion engine), so a "forget all the rules" request hidden in Base64, leetspeak, or a low-resource language is still recognized. The second line is output_scanner, which intercepts a dangerous response if the input did get through.

Sources

Related guides: OWASP LLM Top 10 · Prompt Injection · RAG Security