Fine-Tuning & Model Customization · TRACK 01/04
Fine-Tuning Fundamentals
What fine-tuning can (and can't) change, and how to prepare for it.
// THE TRACK
01 · START HEREFine-TuningUnderstand what actually happens to a model's weights when you fine-tune it, and why teams do it instead of relying on prompts alone.BEGINNERPretraining vs Fine-TuningUnderstand the two-stage paradigm behind every modern LLM: expensive pretraining that builds a base model, then cheap fine-tuning that shapes its behavior.INTERMEDIATEFine-Tuning Capabilities & LimitsLearn which behaviors fine-tuning reliably changes (style, format, tone) and why injecting new factual knowledge usually disappoints.BEGINNERFine-Tune vs Prompt & RAGWalk away with a clear decision tree for choosing between prompting, RAG, and fine-tuning before you spend money on training.BEGINNEREvaluating the ResultLearn the before-and-after evaluation workflow that proves a fine-tune actually improved your task without quietly breaking everything else.INTERMEDIATECost BreakdownGet realistic dollar figures for every fine-tuning route — rented GPUs, hosted APIs, and full runs — including the hidden costs of data and serving.INTERMEDIATEDataset PreparationKnow exactly how to format, size, clean, and split a training dataset so your fine-tune learns the right lessons instead of your data's bugs.INTERMEDIATEFull Fine-Tuning vs PEFTUnderstand the trade-off between updating all of a model's weights and updating a tiny adapter, and which choice fits your GPU budget.INTERMEDIATEHyperparametersLearn what each training knob does, the defaults that usually work, and how to read loss curves for overfitting.INTERMEDIATEInstruction TuningUnderstand the supervised fine-tuning step that converts a text-predicting base model into something that actually follows your instructions.BEGINNERChat Templates & Loss MaskingLearn the two formatting details that silently break fine-tunes: matching the model's exact chat template and masking the loss so it trains only on the assistant turn.INTERMEDIATESpotting OverfittingRecognize the tell-tale symptoms of an overfit fine-tune in both loss curves and outputs, and apply the concrete fixes that restore generalization.INTERMEDIATEUnslothYou will understand what Unsloth is, how it speeds up fine-tuning and cuts memory use, and when it is the right pick for solo or low-GPU training.INTERMEDIATEAxolotlYou will understand how Axolotl turns a single YAML file into a reproducible fine-tuning run and when its config-driven, multi-GPU approach is the right fit.INTERMEDIATELLaMA-FactoryYou will understand how LLaMA-Factory lets you fine-tune a wide range of LLMs and VLMs through a CLI or web UI with little or no code.INTERMEDIATEHugging Face PEFTYou will understand what the PEFT library does, how it implements LoRA/QLoRA and adapters, and why it underpins so many fine-tuning tools.INTERMEDIATEms-swiftYou will understand what ms-swift is, the breadth of models and methods it supports, and why its multimodal coverage stands out.INTERMEDIATE