Working with LLM APIs · TRACK 05/05
Cost, Caching & Rate Limits
Token economics: what you pay, what you cache, what throttles you.
// THE TRACK
01 · START HEREAPI PricingRead any provider's pricing page fluently: per-million-token rates, why output costs more, and where cached tokens fit in the bill.BEGINNERPrompt CachingUnderstand how prompt caching stores KV state for reusable prompt prefixes — and how to structure requests so 90% of your input tokens cost almost nothing.INTERMEDIATEHandling 429 ErrorsBuild retry logic that actually works under load — exponential backoff with jitter, retry-after headers, and a client-side queue.INTERMEDIATEThe Batch APITrade latency for a 50% discount: learn how batch endpoints work and which jobs — evals, backfills, classification — belong there.INTERMEDIATEEstimating API CostBuild a back-of-envelope cost model for any LLM feature — tokens per call times calls per day times the rate card — so the bill never surprises you.BEGINNERSpending Limits & AlertsPut hard and soft guardrails on your LLM spend — provider caps, budget alerts, and app-side limits — so a bug or abuse can't drain your account.BEGINNERRPM, TPM & Usage TiersUnderstand the two ceilings every provider enforces — requests and tokens per minute — how usage tiers raise them, and how to stay under both.INTERMEDIATE