LLM Fine-Tuning Explained: What Actually Changes Inside the Model

LLM fine-tuning has a reputation for being either trivial (three lines of a library call) or forbidding (a research project with a cluster attached). Neither is true. It is a small, well-defined operation: you run the same next-token prediction the model was pretrained on, over data you chose, with a mask that decides which tokens get graded, and you let the optimizer move the weights. Everything hard about it is either arithmetic about memory or judgement about data.

This is the first part of a fifteen-part reference series that takes fine-tuning apart. It is the training-side companion to our series on the complete inference path an LLM takes to answer a question. Inference explains how a trained model behaves. This series explains how it got that way, and what the run costs.

By the end of this part you will be able to say precisely what changes inside a model during fine-tuning, place it against prompting and retrieval, and size a training run on the back of an envelope well enough to know whether it fits on the card you have. That last skill is the one that separates people who plan runs from people who discover OOM errors.

Fine-tuning changes weights, and nothing else

A pretrained language model is a fixed set of matrices. Prompting changes what you put in front of those matrices. Retrieval changes what context you put in front of them. Fine-tuning changes the matrices themselves.

The mechanism is the objective the model was already trained on. A causal language model assigns a probability to the next token given everything before it. Training maximises that probability on the token that actually came next, which is the same as minimising token-level cross-entropy, also called negative log-likelihood, summed over the positions you decided to grade.

That formula is worth staring at, because two knobs hide inside it and they are the two knobs the whole field turns. The first is which tokens are in the sum, which is loss masking, covered in the part on the three kinds of masking in LLM training. The second is what data the sequence came from, which turns out to dominate quality and is the subject of Part 10.

Notice what fine-tuning is not. It is not a knowledge transplant. The model learned its facts and its capabilities during pretraining, over trillions of tokens. A supervised fine-tune on ten thousand examples cannot install a new body of knowledge, and treating it as though it can is the most expensive misconception in the field. What a fine-tune reliably installs is behaviour: the format the model answers in, the tone it uses, when it stops, when it refuses, which of several plausible continuations it prefers. If you have read our walkthrough of how a neural network learns from one neuron to backpropagation, the machinery underneath is exactly what you saw there, at scale.

Where fine-tuning sits in the pipeline

Modern instruction-following models are built in three stages, and each one does something the previous stage structurally cannot. Knowing which stage owns which job stops you from asking the wrong stage for a result.

Pretraining gives raw competence and costs millions of dollars. Supervised fine-tuning (SFT) teaches the model to follow instructions in a particular format, and costs anywhere from a few dollars to a few hundred. Preference tuning teaches judgement, which answer is better when several are fluent, and it is where helpfulness, house tone and refusal calibration actually get installed. Parts 9 through 12 of this series are about the SFT stage. Part 13 covers the preference stage, from classic RLHF through DPO to the verifiable-reward methods behind the 2026 reasoning models.

The practical rule: you almost always run preference tuning on top of an SFT checkpoint, never from a raw base model. And you almost never need pretraining at all.

When to fine-tune, and when not to

Fine-tuning is one of four ways to change what a model does for you, and it is the third one you should reach for. The cheap options are genuinely good, and the honest answer for most teams is that a better prompt plus retrieval beats a mediocre fine-tune.

Three signals say fine-tune. Your prompt has grown into a specification that you paste into every call and it still drifts. You need a specific output shape reliably, every time, and a schema-checking retry loop is not cutting it. Or your inference bill is dominated by a long system prompt that a fine-tune could bake into the weights, which is the same economics we walk through in the guide to routing and cost control for LLM calls.

Two signals say do not. The knowledge you need changes weekly, in which case you want retrieval, not weights. Or you cannot yet measure whether the model is better, in which case fine-tuning will produce a change you cannot evaluate. Build the eval first. Our notes on building evals for LLM features in Python are the right prerequisite.

The cost model: bytes per parameter, not parameter count

Here is the single most useful thing in this series, and it takes one line. Memory is not set by parameter count. It is set by bytes per parameter times parameter count, and bytes per parameter swings by a factor of sixteen depending on what you are doing.

Inference in 8-bit costs about 1 byte per parameter. Inference in bf16 costs 2. LoRA training holds the base at 2 and pays almost nothing on top. Standard mixed-precision full fine-tuning with AdamW costs 16. That 16-fold swing is larger than most of the model-size gaps people worry about, which produces results that look paradoxical until you do the arithmetic.

Why does the rate differ so much? Because inference and training hold different numbers of things at once. Inference holds the weights and, while generating, a KV cache. Training holds the weights, a gradient for every weight, the optimizer’s state for every weight, and every intermediate tensor the forward pass produced that the backward pass will need.

Those four things are the four tenants, and the next part in this series takes each one apart and derives the 16 bytes. For now, the itemisation is enough to plan with.

Multiply 16 by the parameter count and you have the static state of a full fine-tune before activations. For the model this series uses throughout, Qwen2.5-1.5B at 1.54 billion parameters, that is about 24.6 GB. Add roughly 8 GB of activations at batch 4 and sequence 1024 and you are at about 33 GB, which does not fit a 32 GB card without help. That single multiplication screens any model in five seconds:

One honest caveat about units. Vendors quote memory in GiB, which is 2 to the 30 bytes, while byte-math like the above lands in decimal GB, which is 10 to the 9. So 24.6 GB is about 22.9 GiB, and a card advertised as 32 GB gives you roughly 29.8 GiB once the driver and CUDA context take their cut. Keep a 2 to 3 GB safety margin in your head, because the last gigabyte is where OOM errors live.

The diagnostic that follows from it

If a model is using far more memory than its size suggests, there are only two common causes, and the fix differs.

Either you are training, in which case the extra memory is the other three tenants doing real work, or you are serving through an engine that pre-allocates a KV cache pool at startup. The second one looks alarming in nvidia-smi and is usually just a configuration number. Our series part on continuous batching and paged attention covers why serving engines reserve that way.

The three-rung ladder

Almost everything in practical fine-tuning is a choice between three rungs, and the whole point of the sequence is that each rung removes a cost the previous one paid.

Full fine-tuning updates every weight. It pays 16 bytes per parameter, produces a complete model copy per task, and moves every weight, which means it can erode capabilities the model already had. This is called catastrophic forgetting, and you cannot see it in your training loss.

LoRA freezes the base and learns a small low-rank patch beside each targeted weight matrix. Because the frozen weights receive no gradient and need no optimizer state, 14 of the 16 bytes vanish for about 99 percent of the parameters. A run that needed 24.6 GB now needs about 4 GB, and the artefact you ship is an adapter of tens of megabytes instead of a multi-gigabyte checkpoint.

QLoRA quantises the frozen base to 4 bits, so the one cost LoRA could not remove drops by a factor of about four. That is what puts a 7B, 13B, or even a 30B dense model on a single 32 GB card.

Read the bottom-right corner of that table carefully, because it contains the most commonly skipped fact in the field. QLoRA on a small model that already fits is a loss, not a win. Four-bit storage means every matrix multiply has to de-quantise the weights back to bf16 first, which is real work on the critical path. Part 12 puts a number on that tax and gives the decision rule: reach for QLoRA when a model would not otherwise fit, not by default.

What the rest of this series covers

The order is deliberate. Foundations first, because every practical decision later is a memory or numerics decision in disguise, then the actual runs, then the efficient methods.

• Parts 2 to 4, the memory and numerics foundation. The four tenants and where the 16 bytes come from. Activation memory, which scales with batch and sequence rather than parameters and is the tenant that actually OOMs you. Then the number formats: why bf16 beat fp16 for training, why a 32-bit master copy of the weights exists, and why NF4 and FP8 are different kinds of 4-bit and 8-bit.
• Parts 5 and 6, the optimizer. What a gradient is, what an optimizer decides, and why AdamW’s two running statistics per weight cost 8 bytes and are worth it. Then AdamW line by line, including what the W actually fixed.
• Part 7, masking. Three different things share the name: the loss mask you set, the causal mask the architecture applies, and the padding mask your collator builds. Getting the first one wrong produces a worse model with no error message.
• Parts 8 to 10, the practical run. Choosing a base model and standing up a bench. A full supervised fine-tune end to end, twice, once as a bare PyTorch loop and once through a trainer. Then data, which is where the real quality lives.
• Parts 11 to 13, doing it cheaply and doing it well. LoRA, QLoRA, and preference tuning.
• Parts 14 and 15, scale and proof. What a second GPU actually buys, why the wire between two cards decides which parallelism strategy is viable, and then evaluation: the ladder from perplexity to human review, the three rules that make a holdout trustworthy, and the six silent failure modes a loss curve will never show you.

Every part uses the same running example so the numbers stay comparable: Qwen2.5-1.5B base, 28 layers, hidden size 1,536, trained on the roughly 10,000 human-written instruction pairs in the no_robots dataset, on one 32 GB card. Where the arithmetic changes at scale we switch to Qwen2.5-7B and say so.

Key takeaways

• Fine-tuning runs the same next-token cross-entropy objective as pretraining, on your data, with a mask that decides which positions are graded. There is no new loss function.
• It changes behaviour reliably and knowledge poorly. Use retrieval for facts and fine-tuning for format, tone and task discipline.
• Memory is bytes per parameter times parameters. The rate runs from about 1 for 8-bit inference to 16 for standard mixed-precision full fine-tuning with AdamW.
• Of those 16 bytes, 12 belong to the optimizer, which is why almost every memory technique targets the optimizer state.
• Multiply parameters by 16 to screen a full fine-tune in five seconds, then add activations, then keep a 2 to 3 GB margin because vendors quote GiB and byte-math gives GB.
• The three rungs are full fine-tuning, LoRA, and QLoRA. Each removes a cost the previous rung paid, and QLoRA is a fitting tool rather than a speed tool.
• Preference tuning is a separate stage on top of SFT, and it is the only stage that can express that one answer is better than another.

Frequently asked questions

Does fine-tuning teach a model new facts?

Not reliably. Knowledge and capability come from pretraining over trillions of tokens, and a supervised fine-tune on thousands of examples mostly teaches format, style and task behaviour. If you need the model to know something specific and current, retrieval is the correct tool. Fine-tuning can help the model use retrieved facts in the shape you want.

How much VRAM do I need to fine-tune a 7B model?

Full fine-tuning a 7B model needs roughly 112 GB of static state at 16 bytes per parameter, so it does not fit on one consumer card. LoRA brings the same job to about 18 GB because the frozen base only pays for its weights, and QLoRA brings it to about 8 GB by storing that base in 4 bits. On a 32 GB card, plain LoRA is the simpler and faster choice for a 7B.

Should I fine-tune or use RAG?

Use retrieval when the problem is which facts the model sees, and fine-tuning when the problem is how the model behaves. They are complementary rather than competing, and many production systems use both: retrieval supplies the context, and a fine-tune enforces the output shape and tone. If your knowledge changes weekly, do not put it in weights.

Why is the optimizer 12 of the 16 bytes per parameter?

AdamW keeps three 32-bit numbers for every parameter: a high-precision master copy of the weight, a running average of the gradient, and a running average of the squared gradient. Three values at 4 bytes each is 12 bytes. In exchange, every weight gets its own adaptive step size, which is what makes transformer training reliable.

Is LoRA as good as full fine-tuning?

For format, style and instruction adaptation it usually matches full fine-tuning at a fraction of the cost. For teaching a genuinely new capability such as heavy code or maths ability, careful research finds that LoRA learns less and also forgets less, so full fine-tuning still earns its cost there. Part 11 covers the trade honestly.

Can I fine-tune without a GPU?

Not usefully for a model of this size. Fine-tuning needs the forward pass, the backward pass and an optimizer step to be fast enough to iterate, which means a GPU with enough VRAM for all four memory tenants. Renting a single cloud GPU by the hour is the cheap path, and Part 8 covers which instance sizes match which method.

Sources and further reading

• Rajbhandari et al., ZeRO: Memory Optimizations Toward Training Trillion Parameter Models, the source of the 16 bytes per parameter accounting used throughout this series.
• Micikevicius et al., Mixed Precision Training, which introduced the 32-bit master weight copy that makes up 4 of the optimizer’s 12 bytes.
• Hu et al., LoRA: Low-Rank Adaptation of Large Language Models.
• Dettmers et al., QLoRA: Efficient Finetuning of Quantized LLMs.
• Zhou et al., LIMA: Less Is More for Alignment, the clearest evidence that supervised fine-tuning is mostly teaching format rather than knowledge.
• Hugging Face, Methods and tools for efficient training on a single GPU.