What Is a System Prompt? System vs User vs Assistant Roles

// In plain English

A system prompt is the set of standing instructions a developer gives an AI model before any user types a word. It tells the model who it is, what job it is doing, which rules it must follow, and how it should sound. Every conversation the model ever has starts with those instructions already in place.

Think of briefing a new employee before the shop opens: "You work the support desk at Acme Cameras. Only answer questions about our products. Be friendly but brief. Never hand out the discount codes taped under the register." Customers come and go all day with completely different questions, but the briefing stays in force for every one of them. The customers never hear the briefing — they just experience an employee who consistently behaves a certain way. That briefing is the system prompt.

Chat-style AI APIs split every conversation into three roles: system is the briefing, written by the developer. user is whatever the human types. assistant is the model's own replies. When you chat with ChatGPT or the Claude app, you are only ever writing user messages — a system prompt is sitting above the conversation the whole time, you just never see it.

// Why it matters

The earliest text models had no roles at all. You concatenated your instructions and the user's text into one blob, and the model continued the text. The problem: the model couldn't tell which words were the app and which were the user. Anyone could type "new instructions: do X instead" and the model had no particular reason to prefer your rules over theirs.

Splitting conversations into roles fixed three things at once:

• Consistency. Rules stated once in the system prompt apply to every turn. You don't repeat "answer in French" and "never give legal advice" in each message — they're simply always in effect.
• Authority. Models are specifically trained to give system-role instructions more weight than user messages. When a user types "ignore your previous instructions," the model has been taught that the system prompt wins. It's a trained tendency, not a hard guarantee — and that gap is the entire field of prompt injection.
• Separation. App logic lives in the system prompt; user data lives in user messages. That makes the prompt something you can version, test, and review like code, instead of a string mixed into user input.

Who should care: anyone calling an LLM API, obviously — but also anyone using "custom instructions" in ChatGPT or setting up a Claude Project. Those features are essentially user-editable additions to the system prompt. Understanding what each role is for is the first practical skill in prompt engineering: before you tune any wording, you need to know which channel the wording belongs in.

// How it works

On the wire, a chat request is a list of messages, each with a role and content. The API is stateless: the model doesn't remember your previous call. Every request re-sends the system prompt plus the full conversation history, and the model generates the next assistant message.

Under the hood there are no separate channels. A chat template wraps each message in special tokens — for example <|im_start|>system in the ChatML format many open models use — and concatenates everything into one long token stream. The role markers survive into that stream, so the model can see exactly which text came from the operator and which came from the user.

The extra weight of the system role is learned, not enforced. During fine-tuning, models see many examples where system instructions take precedence over conflicting user requests, and they're rewarded for respecting that hierarchy. That training is why a system prompt "carries more weight" — and also why the effect is strong but not absolute.

Because each request carries the whole history, the assistant role is how the model's past answers get back into its head — and, as you'll see next, it's also a lever you control directly.

// The three roles in code

In the Claude API, the system prompt isn't a message at all — it's a top-level system parameter, and the messages array contains only user and assistant turns:

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY from the environment

response = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    # The system prompt is a top-level parameter, not a message
    system=(
        "You are a support agent for Acme Cameras. "
        "Only answer questions about Acme products. "
        "If you don't know a spec, say so — never invent one."
    ),
    messages=[
        {"role": "user", "content": "Does the Acme X100 shoot 4K video?"},
    ],
)
print(response.content[0].text)

OpenAI-style APIs put the same instructions inside the messages array instead, as the first message. Here's a full multi-turn payload in that format — note the assistant turn sitting in the middle:

{
  "messages": [
    { "role": "system", "content": "You are a support agent for Acme Cameras." },
    { "role": "user", "content": "Does the Acme X100 shoot 4K video?" },
    { "role": "assistant", "content": "Yes — the X100 records 4K at up to 60 fps." },
    { "role": "user", "content": "What about the X200?" }
  ]
}

Multi-turn chat is just bookkeeping: after each reply, you append the model's answer with role: "assistant" and the user's next message with role: "user", then send the whole thing again. Forget to include the history and the model genuinely has no idea what "the X200" refers to.

The assistant role has a second job: you can write assistant messages yourself. Hand-crafted user/assistant pairs placed before the real question act as worked examples that teach the model your desired format by demonstration — the core trick behind few-shot prompting.

// What goes in which role

Most beginner role bugs come down to content living in the wrong channel. The rules of thumb:

• System: stable, always-true things. Identity and scope ("you are a support agent for Acme"), hard rules ("never quote prices"), tone, output format, and instructions for any tools the model can call.
• User: everything that changes per request — the question, pasted documents, retrieved search results, form data.
• Assistant: conversation history, plus any example outputs you wrote by hand.

A persona line — "you are a senior tax accountant" — is the most famous thing people put in a system prompt. Whether that actually improves answers is more nuanced than the folklore suggests; we dig into the evidence in does role prompting work.

Common mistakes

• Putting user data in the system prompt. Pasting a customer's document into the trusted channel means any instructions hidden inside that document read as if they came from you. Untrusted content belongs in user messages.
• Putting per-request values at the top of the system prompt. A timestamp or user ID up front makes every request's prompt unique, which defeats provider-side prompt caching and costs real money at scale. Keep the system prompt byte-identical across requests; volatile bits go in the user turn.
• Putting secrets in the system prompt. Users can often coax a model into reciting its own instructions — see prompt leaking. Behavioral rules are fine to ship; API keys and internal URLs are not.
• Repeating the rules in every user message "just in case." The system prompt persists on its own. Restating it per turn wastes tokens and clutters the conversation.

// Going deeper

The instruction hierarchy is a research problem, not a switch. OpenAI's 2024 paper The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions makes the privilege levels explicit — system above user, user above the model's own prior outputs, and all of those above third-party tool outputs — and trains models to resist instructions injected at lower levels. The key takeaway for builders: hierarchy compliance is a trained behavior with a measurable failure rate, not an access-control system. Every successful jailbreak is a data point on that failure rate.

Wire formats keep diverging while the concept converges. Anthropic uses a top-level system parameter and keeps roles in messages to user/assistant; OpenAI renamed system to developer in newer docs while still accepting the old name; some APIs now also accept operator instructions appended mid-conversation, so you can update standing rules without rebuilding the whole prompt prefix. If you run open models locally, role handling lives in the model's chat template — apply the wrong template and your carefully written system text gets serialized as ordinary prose, silently destroying quality with no error message.

Production system prompts are long. The system prompts behind real assistants run to thousands of tokens: behavioral rules, tool definitions, formatting requirements, edge-case handling. Anthropic publishes the system prompts used by the Claude apps in its release notes, and reading one is the fastest education available in production prompt design. Two engineering consequences follow. First, prompt caching: providers charge a fraction of the normal price for a repeated, unchanged prompt prefix, so a frozen system prompt is a direct cost lever. Second, ordering: stable content first, volatile content last, because one changed byte invalidates the cache for everything after it.

Get the trust model right. Treat the system prompt as public — it can leak. Treat user content as code from a stranger — it can carry injected instructions. Role separation is the first and weakest line of defense; serious deployments layer input filtering, output checks, and tool-level permissions on top of it. The roles tell the model who said what. What your application does about that is still your job.