Training-Time vs. Inference-Time Alignment

Alignment is not one thing. Some safety lives in training (RLHF, constitution). Some lives at runtime (system prompts, classifiers, filters). Understanding the split tells you where a given failure actually came from.

36 min · Reviewed 2026

Two Layers of Safety

When a model refuses a harmful request, you see a single refusal. Inside the stack, that refusal came from at least two places. The weights were trained to refuse. And a runtime system added guardrails on top of those weights.

Training-time alignment

RLHF / RLAIF: reward-model-driven fine-tuning
Constitutional AI: model critiques itself against written principles
DPO and variants: direct preference optimization
Supervised fine-tuning on curated refusals
Red-team iteration: find failure, patch with training data

Inference-time alignment

System prompts and role instructions
Input and output classifiers (OpenAI Moderation, Llama Guard, ShieldGemma)
Rate limits and usage monitoring
Tool use restrictions and sandboxing
Watermarking and content provenance

Example: a phishing-email request

Layer	What happens	If it fails
Input classifier	Flags 'phishing' keyword	Request reaches the model
System prompt	Reminds model of policy	Model may still refuse from training
Model weights	Trained to refuse social-engineering help	Output moves downstream
Output classifier	Scans generated text for harmful markers	User receives the bad content

Model safety is not a property of the weights. It is a property of the deployment.
— Anthropic Responsible Scaling Policy framing

The big idea: if you see safety and think 'model,' you are missing most of the picture. A deployed AI's behavior is a stack, and each layer has different costs, trade-offs, and failure modes.

End-of-lesson check

15 questions · take it digitally for instant feedback at tendril.neural-forge.io/learn/quiz/end-safety2-training-vs-inference-alignment-creators

What is the core idea behind "Training-Time vs. Inference-Time Alignment"?
1. Alignment is not one thing. Some safety lives in training (RLHF, constitution). Some lives at runtime (system prompts, classifiers, filters). Understanding the split tells you where a given failure actually came from.
2. path patching
3. When Claude believed the response was not monitored, it refused at baseline (~97…
4. Identifies failure modes that inform labs' own safety work
Which term best describes a foundational idea in "Training-Time vs. Inference-Time Alignment"?
1. inference-time alignment
2. training-time alignment
3. defense in depth
4. classifier
A learner studying Training-Time vs. Inference-Time Alignment would need to understand which concept?
1. training-time alignment
2. defense in depth
3. inference-time alignment
4. classifier
Which of these is directly relevant to Training-Time vs. Inference-Time Alignment?
1. training-time alignment
2. inference-time alignment
3. classifier
4. defense in depth
Which of the following is a key point about Training-Time vs. Inference-Time Alignment?
1. RLHF / RLAIF: reward-model-driven fine-tuning
2. Constitutional AI: model critiques itself against written principles
3. DPO and variants: direct preference optimization
4. Supervised fine-tuning on curated refusals
Which of these does NOT belong in a discussion of Training-Time vs. Inference-Time Alignment?
1. DPO and variants: direct preference optimization
2. RLHF / RLAIF: reward-model-driven fine-tuning
3. Constitutional AI: model critiques itself against written principles
4. path patching
Which statement is accurate regarding Training-Time vs. Inference-Time Alignment?
1. Input and output classifiers (OpenAI Moderation, Llama Guard, ShieldGemma)
2. Rate limits and usage monitoring
3. System prompts and role instructions
4. Tool use restrictions and sandboxing
Which of these does NOT belong in a discussion of Training-Time vs. Inference-Time Alignment?
1. path patching
2. Input and output classifiers (OpenAI Moderation, Llama Guard, ShieldGemma)
3. Rate limits and usage monitoring
4. System prompts and role instructions
What is the key insight about "Why use both" in the context of Training-Time vs. Inference-Time Alignment?
1. Training-time alignment is robust to many prompt variations but fixed once the model ships.
2. path patching
3. When Claude believed the response was not monitored, it refused at baseline (~97…
4. Identifies failure modes that inform labs' own safety work
What is the key insight about "The jailbreak split" in the context of Training-Time vs. Inference-Time Alignment?
1. path patching
2. Most jailbreaks attack one layer. Role-play tricks bypass the system prompt, encoded prompts evade classifiers, fine-tun…
3. When Claude believed the response was not monitored, it refused at baseline (~97…
4. Identifies failure modes that inform labs' own safety work
Which statement accurately describes an aspect of Training-Time vs. Inference-Time Alignment?
1. path patching
2. When Claude believed the response was not monitored, it refused at baseline (~97…
3. When a model refuses a harmful request, you see a single refusal. Inside the stack, that refusal came from at least two places.
4. Identifies failure modes that inform labs' own safety work
What does working with Training-Time vs. Inference-Time Alignment typically involve?
1. path patching
2. When Claude believed the response was not monitored, it refused at baseline (~97…
3. Identifies failure modes that inform labs' own safety work
4. The big idea: if you see safety and think 'model,' you are missing most of the picture.
Which best describes the scope of "Training-Time vs. Inference-Time Alignment"?
1. It focuses on Alignment is not one thing. Some safety lives in training (RLHF, constitution). Some lives at runtim
2. It is unrelated to ethics workflows
3. It applies only to the opposite beginner tier
4. It was deprecated in 2024 and no longer relevant
Which section heading best belongs in a lesson about Training-Time vs. Inference-Time Alignment?
1. path patching
2. Training-time alignment
3. When Claude believed the response was not monitored, it refused at baseline (~97…
4. Identifies failure modes that inform labs' own safety work
Which section heading best belongs in a lesson about Training-Time vs. Inference-Time Alignment?
1. path patching
2. When Claude believed the response was not monitored, it refused at baseline (~97…
3. Inference-time alignment
4. Identifies failure modes that inform labs' own safety work

← Back to interactive lesson

Tendril · Creators · Ethics & Society