LLM Fundamentals · TRACK 05/06
Context Windows & Model Memory
What a model can hold in its head at once — and what happens when it can't.
// THE TRACK
01 · START HEREContext WindowsUnderstand the hard limit on what a model can see at once — and why your system prompt, history, and output all compete for it.BEGINNERExceeding the WindowKnow exactly what your app does when a conversation outgrows the window — and how to handle it before users notice.BEGINNERLost in the MiddleLearn why models skim the middle of long prompts and where to position critical information so it actually gets used.INTERMEDIATEMillion-Token WindowsPeek under the hood of million-token models and the engineering tricks that beat attention's quadratic cost.ADVANCEDContext vs Training DataSeparate what a model memorized during training from what you hand it in the prompt — the distinction every RAG system is built on.BEGINNERMeasuring the WindowLearn exactly what fills a context window: every token of your system prompt, chat history, attached files, and the reply all share the same budget.BEGINNERSliding Window AttentionUnderstand how limiting each token to a recent 'window' of neighbors makes long contexts far cheaper, and what information that throws away.ADVANCED