AI/TLDR

Evaluation & Safety · TRACK 05/05

Alignment & Safety Basics

Why models refuse, how they're steered, and the bigger risk map.

15 ARTICLESbeginner → advanced
// THE TRACK
01 · START HEREAI AlignmentUnderstand what the alignment problem actually is, separated from both doomer and booster hype.BEGINNER
Mechanistic InterpretabilityUnderstand what mechanistic interpretability is trying to do, what features and circuits are, and why it matters for safety.ADVANCEDSycophancyUnderstand why RLHF-trained models flatter users, how sycophancy is measured, and how to design around it.BEGINNERRefusals & Over-RefusalsUnderstand why models refuse, where over-refusal comes from, and what builders can do about false positives.BEGINNERSafety vs SecurityBe able to draw a clean line between safety work and security work on an AI product — and see where they overlap.INTERMEDIATEConstitutional AIUnderstand how Constitutional AI uses written principles and AI feedback to align models, and how it compares to RLHF.INTERMEDIATEMeasuring BiasKnow the standard ways bias is measured in LLMs and how to run a basic fairness audit on your own app.INTERMEDIATEReward HackingYou'll understand how a model can ace its reward signal while completely missing what you actually wanted.INTERMEDIATESpecification GamingYou'll understand why AI systems exploit loopholes in goals you thought were clear, and why specifying intent is genuinely hard.INTERMEDIATERLHF vs RLAIFYou'll understand how RLAIF swaps human preference labels for AI-generated ones, and the tradeoffs versus classic RLHF.INTERMEDIATELlama GuardYou will understand how Llama Guard classifies prompts and responses against hazard categories to gate unsafe content.INTERMEDIATEGuardrails AIYou will understand how Guardrails AI wraps LLM calls with validators that check, fix, or block outputs.INTERMEDIATENeMo GuardrailsYou will understand how NeMo Guardrails uses the Colang DSL to define rails that steer and constrain an LLM conversation.INTERMEDIATEMicrosoft PresidioYou will understand how Presidio finds and anonymizes personal data and why that matters for LLM pipelines and logs.INTERMEDIATELLM GuardYou will understand how LLM Guard chains input and output scanners to defend an LLM app against common attacks.INTERMEDIATE