Granite 4.1 LLMON Preview

// Overview

Granite 4.1 LLMON Preview is a pair of experimental open-weight instruct models that IBM's Granite team published on Hugging Face on 12 November 2026, in 3B and 8B sizes, under the Apache 2.0 license. Both are fine-tuned from the Granite 4.1 base checkpoints rather than from the released instruct models, and both read prompts written in LLMON (LLM Object Notation, pronounced "lemon"), which IBM describes as an LLM-native markup language for explicitly communicating structure and semantics to the model. IBM labels the release an experimental preview, ready for exploration and evaluation but not for production.

LLMON wraps every part of a prompt — system policy, instructions, data, documents, tool definitions and tool results — in its own addressable block, using six reserved delimiters that are each a single token. An exec block then names which instruction to run, which inputs it operates on and where the output goes. The model is trained on a privilege rule: the instruction and system blocks are authoritative, while everything reached through exec input — data, documents, tool schemas and tool results — is material to read, not a command to follow, however it is phrased. The shipped chat template renders ordinary messages into LLMON, so callers use apply_chat_template and never write the markup by hand.

IBM's launch post says the training converted a standard instruction-tuning dataset into LLMON with added decoy blocks, then ran five sequential GRPO reinforcement-learning stages and a final DPO stage on 1,000 preference pairs. On the OPI prompt-injection benchmark (9,000 attacks, robustness = 1 − attack success rate), IBM reports that both LLMON models resist nearly every attack, a 47-point gain over the best frontier model it tested and a 95-point gain over the conventional Granite 4.1 8B. On general benchmarks the goal was parity with the conventional Granite 4.1 instruct models rather than improvement, and the 8B model card shows mixed results: higher on BigCodeBench, INCLUDE and the safety suites, lower on GPQA, MGSM and the math sets.

The model cards stress that the privilege rule is learned behaviour, not an enforced guarantee: it substantially reduces, but does not eliminate, the chance that an instruction hidden in data is followed, so it should not be the sole defence for an agent with access to sensitive systems. Prompts in a conventional chat format are out of distribution and degrade quality. IBM publishes the LLMON toolkit on GitHub, an attention-visualization demo on Hugging Face Spaces, and native LLMON support in its Mellea library.

// Benchmarks

This model's scores

1. Tulu3 Safety Eval Avg (8B)87.31
2. IFEval Avg (8B)86.69
3. HumanEval pass@1 (8B)86.59
4. MMLU 5-shot (8B)74.2
5. BFCL v3 (8B)67.93
6. MMLU 5-shot (3B)68.05

Scores on a 0–100 scale (25-point gridlines); higher is better. Each benchmark links to its published source.

// Strengths

• Apache 2.0 weights in 3B and 8B sizes — commercial use, fine-tuning and self-hosting allowed
• Trained to treat retrieved documents, pasted content and tool results as data rather than instructions
• IBM reports near-complete robustness on the OPI prompt-injection benchmark for both sizes
• General-capability scores close to the conventional Granite 4.1 instruct models of the same size
• Bundled chat template renders ordinary messages into LLMON, with OpenAI-style tool definitions supported
• Twelve supported languages, inherited from the Granite 4.1 base

// Best for

• RAG assistants that quote untrusted web pages or documents into the prompt
• Tool-using agents whose tool results may carry injected instructions
• Research into structured prompting and instruction/data separation
• Evaluating prompt-injection defences against a model trained for the boundary