IBM Trains Two Small Granite Models to Separate Instructions and Data to Resist Prompt Injection
On November 12, IBM released two experimental models under the Apache 2.0 license, Granite 4.1 3B and 8B LLMON-preview, trained to read marked-up prompts where each instruction is explicitly linked to the data it processes. On the indirect injection benchmarks cited by IBM, attack success rates plummeted from 53–100% down to 0.02–5.6%. IBM cautions: this is a learned behavior, not a guarantee, and is not yet ready for production.

Indirect prompt injection is the vulnerability haunting AI agents: a hidden directive inside a web page, document, or tool output that the model executes as if it came from the user. The issue stems from the way models ingest their context: everything is raw text, with no clear boundary between what must be executed and what should merely be read. IBM proposes drawing that boundary within the prompt format itself, releasing two experimental models on November 12 to test the approach: Granite 4.1 3B and 8B LLMON-preview, licensed under Apache 2.0.
The core idea: an instruction points to its data
LLMON (“LLM Object Notation”) resembles XML, with one crucial difference: its delimiters are six reserved tokens in the vocabulary that cannot be produced by ordinary text. Every part of the prompt becomes a labeled block: instructions, data, constraints, documents, tool definitions, and tool outputs. A final block, exec, functions like a function call: it specifies which instruction to run, on which inputs, and where to write the response. According to the model card, the learned rule is simple: whatever is designated as an instruction carries authority, while anything supplied as an input is read without being executed, “regardless of phrasing.” Developers do not write this markup by hand: they structure the components of a message (instruction, data, requirement), and the provided template handles the conversion.
IBM achieved this behavior through post-training without altering model architecture: a 4.6-billion-token dataset in LLMON format, augmented with decoy content (instructions and data present in context but not referenced), followed by five stages of reinforcement learning and a final optimization step on model response pairs to injection attempts.
Reported numbers
| Benchmark (attack success rate) | 3B before | 3B LLMON | 8B before | 8B LLMON |
|---|---|---|---|---|
| OPI (injection in textual data) | 100% | 0.1% | 96.3% | 0.02% |
| ASB (injection in a tool output) | 99.2% | 5.6% | 52.8% | 3.1% |
Source: IBM research paper, compared against same-size Granite 4.1 Instruct models.
Trade-offs in general capabilities appear minor but non-zero. On the 3B model card, scores remain close to the Instruct version in knowledge (MMLU 68.05 vs. 67.02), instruction following (IFEval 81.02 vs. 82.30), and tool calling (BFCL v3 60.71 vs. 60.80), but regress in reasoning on MMLU-Pro (46.08 vs. 49.83) and code reasoning on CRUXEval-O (32.75 vs. 40.75).
The limits, according to IBM
IBM is explicit: the separation is a learned behavior, not a guarantee. It substantially reduces, but does not eliminate, the risk of a directive slipped into data being executed, and must not serve as the sole defense for an agent accessing sensitive systems. The cited benchmarks evaluate static injections; adaptive attacks, which unfold across multiple conversational turns, remain work in progress. The models expect the LLMON format (standard prompts degrade their outputs) and are framed as a preview for evaluation, not for production. Finally, the comparison with closed-source models (“47% more attacks prevented” than the top frontier model tested) does not name those models.
Our take
The concept builds on existing practice: chat formats already distinguish roles (system, user, tool). IBM’s twist is to move one layer deeper, right inside the message, linking each instruction to its data via an identifier much like a program passes arguments to a function. The reported deltas on static attacks are dramatic, but that is precisely the easiest testing ground; a real-world attacker adapts the payload. For a team building an agent that ingests web pages or emails, LLMON is primarily an avenue to monitor and evaluate on custom use cases, alongside standard defenses: least-privilege agent permissions, human confirmation for sensitive actions, and output filtering. The real test will be whether this approach scales from 3B and 8B models to larger architectures, or if other labs adopt an equivalent format.
Sources
- Hugging Face (IBM Granite blog) — LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B (November 12, 2026) Primary source
- Hugging Face — ibm-granite/granite-4.1-3b-llmon-preview model card Primary source
- arXiv — Post-Training Large Language Models with LLMON (IBM Research, full text, section 5.2) Primary source
A primary source comes directly from the company, lab or organization concerned. A secondary source reports the information without being its author: we flag it as such. Spotted an error? See our corrections policy.


