LLM Fine-Tuning Explained: What Actually Changes Inside the Model

Read a model’s guess the way a training loop reads it

Start with the smallest complete thing a language model does. You hand it the text “The cat sat on the” and it tells you what it thinks comes next. That single act, repeated, is the whole job, at training time and at serving time alike. Three ideas make it up.

A parameter is one number

Inside the model is a very large pile of numbers. Each one is called a parameter, or equivalently a weight. They are ordinary decimal numbers, things like 0.0271 and -0.4413. Qwen2.5-1.5B, the model this series uses for every calculation, holds 1.54 billion of them, arranged into 28 stacked layers. Those figures come from the Qwen team’s technical report.

Those numbers are the model. Every fact it can recall and every habit it has is stored as their particular values. When somebody says a model has 7 billion parameters, they are telling you how many numbers are in the pile. Nothing more.

A token is a chunk of text

Models do not read characters, and they do not quite read words either. They read tokens, which are chunks of text of roughly word size. Common words are usually one token each. Rarer ones get split, so “unhappiness” might arrive as three separate tokens. Every model ships with a fixed list of all the tokens it knows, called its vocabulary. Qwen2.5’s vocabulary holds about 151,000 entries.

The output is one score for every token in the vocabulary

Here is the part that surprises people. The model does not output a word. It outputs one number for every single token in its vocabulary, so about 151,000 numbers, all at once. Each number is a raw score meaning “how much do I like this token as the next one”.

Those raw scores are called logits. A logit can be negative, it can be large, and the whole set of them does not add up to anything in particular. It is a scoreboard. Nothing on it is a probability yet.

To turn a scoreboard into probabilities you apply softmax. Softmax does two things in order. First it makes every number positive, by raising e (about 2.718) to the power of each score. Then it makes them add up to exactly 1, by dividing each result by the total. Take a toy vocabulary of three tokens with scores of 2.0, 1.0 and 0.1:

• Raise e to each score: 7.39, 2.72, 1.11.
• Add those up: 11.22.
• Divide each one by that total: 0.66, 0.24, 0.10.

Those three numbers sit between 0 and 1 and they sum to 1, so they are probabilities. The model is saying it is 66 percent confident in the first token, 24 percent in the second and 10 percent in the third. A real vocabulary has 151,000 entries instead of 3, and the arithmetic is identical.

That is the complete result of one pass through the model, which is called a forward pass: a probability for every token the model knows. Serving is what you do with those probabilities when you want text out, and the complete path an LLM takes to answer a question covers that side of the story. Training is what you do with those probabilities when you already know which token was correct.

Score that guess: what cross-entropy loss actually measures

Training needs a single number saying how badly the model just did. One number, because the entire training loop is an attempt to make one number smaller. That number is called the loss.

Building it takes three small steps.

Step 1. Look up one probability. You are training on real text, so you know which token actually came next. Say it was “mat”. Ignore the other 150,999 numbers the model produced and look up only one: the probability it gave to “mat”. Call that number p.

Step 2. Turn that probability into a penalty. You want a score that is near zero when p is close to 1, and that grows as p shrinks. The function that does exactly this is the negative natural logarithm, written as minus log p. You never need to compute a logarithm by hand. You need its shape, and this table gives you the whole shape.

Notice the asymmetry. Moving from 0.99 to 0.90 costs a tenth of a point. Moving from 0.10 to 0.01 costs more than two full points. As p heads toward zero the penalty heads toward infinity. That is deliberate. A model that is confidently wrong is worse than one that is merely unsure, and the loss is built to say so.

Step 3. Add up over the positions you are grading. A training example is a whole sequence, not one token. So you repeat steps 1 and 2 at every position you care about and add the results. Divide by the count if you prefer an average. Either way you end up with one number for the whole sequence.

That total has a name: cross-entropy. It is the loss every part of this series refers to, and the loss in every training script you will read. It has a second name as well, negative log-likelihood, which describes the same arithmetic from the other end. The likelihood is the probability the model assigned to the whole of the real text, which means every position’s probability multiplied together. The logarithm turns that long chain of multiplications into a plain sum, and the minus sign flips it so that lower means better. Cross-entropy and negative log-likelihood are the same number. When a paper uses one name and your library uses the other, nothing has changed.

Here is a version you can run right now. It needs nothing but the Python standard library.

import math

# The only input that matters: the probability the model gave
# to the token that actually came next.
for p in (0.99, 0.90, 0.50, 0.10, 0.01):
    print(f"p = {p:.2f}   loss = {-math.log(p):.2f}")

# A whole sequence is just the average over the graded positions.
probs = [0.72, 0.31, 0.95, 0.04]
print("sequence loss:", sum(-math.log(p) for p in probs) / len(probs))

The last line prints about 1.19. Three of those four positions were fine and the fourth, at p equal to 0.04, contributed most of the total on its own. That is the asymmetry doing its job.

The same thing written as a formula, with every symbol named

You will meet this idea written compactly. This is the compact form:

L(theta) = - sum over graded t of log p_theta( y_t | y_<t )

Every symbol in that line, in words:

• L is the loss, the single number produced by step 3. Lower is better.
• theta is the Greek letter theta, and it stands for all 1.54 billion parameters at once. So L(theta) reads as “the loss you get when the parameters are set to these particular values”.
• t is a position in the sequence. Position 1, position 2, and so on.
• y_t is the token that actually appeared at position t. The right answer.
• y_<t is everything before position t. The context the model was allowed to see.
• The vertical bar reads as the word “given”. So p_theta( y_t | y_<t ) is “the probability this model gives to the true token at position t, given the tokens before it”. That is exactly the number p from step 1.
• log is the natural logarithm, and the minus sign at the front flips its sign so that a high probability produces a low loss.
• “sum over graded t” is step 3. The word graded is load-bearing. You choose which positions go into that sum.

That last point is the first of the two knobs this entire field turns. In a chat fine-tune you normally grade only the assistant’s tokens and skip the user’s, because you want the model to learn how to answer rather than how to ask. Choosing which positions count is called loss masking, and Part 7 is entirely about it. The second knob is which text the sequence came from in the first place, which is Part 10. There is no third knob. Fine-tuning runs the same loss function as the original pretraining run, the next-token objective Brown et al. used to train GPT-3, on different text and with a different set of graded positions.

Say exactly what fine-tuning changes, and what it cannot

Now the question in the title. A pretrained model is a fixed pile of numbers. Three things can change what comes out of it, and only one of them touches the pile.

1. Prompting changes what you put in front of the numbers.
2. Retrieval changes what context you put in front of the numbers.
3. Fine-tuning changes the numbers.

That distinction explains everything downstream. Fine-tuning is the only one of the three that leaves you holding a different model file at the end. Send it the same prompt, with no special context and no clever wording, and you get a different answer, because 1.54 billion numbers are now slightly different.

What it installs reliably

Behaviour. The shape of the answer, the tone, when to stop talking, when to refuse, which of several fluent continuations to prefer. If you have a prompt that says “always reply as JSON with these four keys, never add commentary, keep it under 80 words” and the model obeys four times out of five, a fine-tune on a few thousand examples of that exact behaviour is the right tool. It moves the model’s default rather than arguing with it on every call.

What it does not install

Knowledge. This is the most expensive misconception in the field, so it is worth being blunt about it. The model learned its facts during pretraining, from trillions of tokens, over weeks, on thousands of GPUs. A supervised fine-tune on ten thousand examples is a rounding error against that. It will not install your product catalogue, your internal documentation or last quarter’s numbers in any way you can depend on. It will cheerfully produce text that sounds as though it has, which is worse than failing outright, because the failure is silent.

Zhou et al.’s LIMA result is the clearest published evidence for that split. A strong base model plus a thousand carefully written examples produced a well-behaved assistant. A thousand examples is far too few to teach anything factual, so what the fine-tune supplied was format and style. The knowledge was already sitting in the weights.

Allen-Zhu and Li arrive at the same place from the other direction. In their controlled experiments, whether a model can answer questions about a fact is settled by how that fact appeared during pretraining, and a later fine-tune on question-and-answer pairs does not rescue one that was stored badly. Their follow-up puts the storage capacity at roughly 2 bits of knowledge per parameter.

When the problem is which facts the model has in front of it, the tool is retrieval: fetch the relevant documents at question time and put them in the context window. Our walkthrough of retrieval augmented generation with pgvector builds one end to end. The two combine well in practice. Retrieval supplies the facts, and a fine-tune enforces the shape of the answer wrapped around them.

Place fine-tuning against prompting, retrieval and preference tuning

Fine-tuning is not one thing either. Modern instruction-following models are built in three stages, and each stage does something the previous one structurally cannot.

1. Pretraining teaches raw competence: grammar, facts, reasoning patterns, the ability to continue text at all. It costs millions of dollars and months of compute. Kaplan et al. and then Hoffmann et al. mapped how that cost trades off against model size and training tokens. You will almost certainly never do this.
2. Supervised fine-tuning, usually shortened to SFT, teaches the model to imitate good answers in a particular format. Costs run from a few dollars to a few hundred. This is what the word “fine-tuning” means in nearly every sentence you will read, and Parts 9 to 12 do it hands on.
3. Preference tuning teaches judgement: given two answers that are both fluent and both plausible, which one is better. SFT structurally cannot express that, because SFT only ever shows the model one target answer per prompt. Helpfulness, house tone and refusal calibration get installed here. Part 13 covers it.

The practical rule is that preference tuning runs on top of an SFT checkpoint and never on a raw base model, and that pretraining is somebody else’s job. A checkpoint, in case the word is new, is just a saved copy of all the model’s weights at one point in training.

Three signals that say fine-tune, and two that say do not

The honest answer for most teams is that a better prompt plus retrieval beats a mediocre fine-tune. Three situations genuinely justify moving the weights.

1. Your prompt has grown into a specification that you paste into every call, and the model still drifts off it.
2. You need a specific output shape every single time, and a schema check with a retry loop is not getting you there.
3. Your inference bill is dominated by a long system prompt that a fine-tune could bake into the weights, so every request gets shorter.

Two situations say do not, at least not yet.

1. The knowledge you need changes weekly. Weights are the wrong place for it. Use retrieval.
2. You cannot yet measure whether the model got better. A fine-tune will then hand you a change you have no way to evaluate. Build the eval first. Our notes on building evals for LLM features in Python are the right prerequisite, and Part 15 covers evaluating a fine-tune specifically.

Follow one weight through one training step

Everything so far has described a single forward pass. Training is a loop, and the loop has four steps. Walk one full turn of it slowly, because every memory number in the rest of this article falls straight out of what the loop has to hold at once.

Picture the model as a mixing desk with 1.54 billion sliders. Each slider is one parameter. Above the desk is a single readout showing the loss, the “how wrong was that” number from the last section. The job is to get the readout down.

1. Forward pass. Push a batch of training text through the model and get probabilities out. A batch is simply several examples processed together, because a GPU is far more efficient that way. On the way through, each of the 28 layers computes intermediate results and hands them to the next layer. Those intermediate results are called activations. The forward pass does not discard them, because step 3 is going to need them.
2. Loss. Compare the probabilities against the tokens that actually came next, exactly as the last section described. One number.
3. Backward pass. For every slider on the desk, answer one question: if I push this slider up by a hair, does the readout go up or down, and how fast? That answer is one number per slider, and it is called that slider’s gradient. A large gradient means this slider matters a great deal right now. A gradient near zero means it barely moves the readout, so leave it alone. Working this out for all 1.54 billion sliders in one sweep, running backwards from the readout through the layers, is the backward pass. It is also why the activations had to be kept: you cannot work out how a layer’s input affected the loss without knowing what that input was. Part 5 draws the same idea as a ball rolling down into a valley, and the walkthrough of how a neural network learns, from a single neuron up derives it from first principles.
4. Optimizer step. The gradient says which way to push each slider. It says nothing about how far. That decision belongs to a separate component called the optimizer. The default across the field is AdamW, and AdamW is not satisfied with the current gradient alone. For every single slider it keeps a small running record: an average of that slider’s recent gradients, and an average of their squares. It uses that record to give each slider its own step size, so a slider with a wild and noisy history moves cautiously while a steady one is allowed to move faster.

What AdamW keeps between steps is called the optimizer state, and it is the reason this section exists at all. Two running averages per parameter, plus a high-precision copy of the parameter itself, is three extra numbers for every one of the 1.54 billion sliders. Nobody notices that until they try to fit a run on a card. Part 5 explains what those running averages buy, and Part 6 walks the update rule line by line.

Then the loop starts again with the next batch. A fine-tuning run is a few thousand turns of it.

Size any fine-tuning run with one multiplication

This is the single most useful thing in the series, and the rest of this section exists only to make you believe it.

The memory a job needs is the parameter count multiplied by the bytes each parameter costs. Both halves matter, and people only ever remember the first one. The per-parameter rate swings by a factor of sixteen depending on which job you are running, which is a bigger swing than most of the model-size gaps anyone worries about.

What “a byte per parameter” means

Every parameter is a decimal number, and a computer stores a decimal number using a fixed number of bits. Eight bits make one byte. Two formats matter here.

• bf16 uses 16 bits, which is 2 bytes. It is the standard format for a model’s weights during training.
• fp32 uses 32 bits, which is 4 bytes. Twice the size, and it holds far more decimal places.

Part 4 covers why several different 16-bit formats exist and why bf16 won for training. For this article, 2 bytes and 4 bytes is everything you need.

Count what the training loop holds at once

Go back through the four steps and count. Everything the loop needs has to sit in the graphics card’s own memory, called VRAM, which is a separate and much smaller pool than your machine’s system RAM. At the moment the optimizer takes its step, all of the following are in VRAM at once.

1. The weights themselves, in bf16. One number per parameter, 2 bytes each. That is 2 bytes per parameter.
2. The gradients from the backward pass. Exactly one per weight, also in bf16. Another 2 bytes per parameter.
3. The optimizer state. AdamW keeps three fp32 numbers for every parameter: a high-precision master copy of the weight, the running average of its gradient, and the running average of its squared gradient. Three numbers at 4 bytes each is 12 bytes per parameter.

2 plus 2 plus 12 is 16. That is the figure quoted everywhere: standard mixed-precision AdamW costs 16 bytes per parameter. Mixed precision is the name for the arrangement just described, where the fast arithmetic happens in bf16 while a 32-bit copy of each weight is kept alongside it, so that thousands of tiny updates accumulate instead of rounding away to nothing. Part 2 derives all sixteen of those bytes one tenant at a time, and Part 5 explains why the field cheerfully pays the optimizer’s twelve.

There is a fourth thing in memory and it does not obey the per-parameter rule at all: the activations from step 1. Their size depends on how many sequences you push through at once and how long those sequences are, so they cannot be written as bytes per parameter. This is why the series always calls the 16-byte figure the static state, meaning weights plus gradients plus optimizer state, the part that stays the same size all run long. Activations are counted separately, every time. Part 3 is about them.

Those four things, weights, gradients, optimizer state and activations, are the four tenants of the card. The word is worth keeping. It is the frame Part 2 builds on, and it is what lets you tell two identical-looking memory problems apart a little further down this page.

Why serving the same model is so much cheaper

Serving is a different job with a different set of tenants. There is no backward pass, so there are no gradients. There is no optimizer, so there is no optimizer state. Only the weights have to be resident.

Serving does add one thing back. While a model is generating an answer it saves some intermediate work for every token it has already produced, so it does not have to redo that work for the next one. That saved pile is called the KV cache. It grows with the length of the conversation rather than with the size of the model, and it comes back in the diagnostic further down.

So serving costs 2 bytes per parameter with the weights in bf16, or about 1 byte if they have been compressed into an 8-bit format. Training the same model costs 16. That sixteen-fold gap produces results which look impossible until you do the multiplication.

A 27 billion parameter model being served at 1 byte per parameter needs about 27 GB for its weights. A 1 billion parameter model being fully fine-tuned needs 16 GB of static state plus its activations, which lands around 22 GB in total. The larger model sits in the same memory bracket as the smaller one, because its per-parameter rate is sixteen times lower.

The running example, worked in full

Qwen2.5-1.5B has 1.54 billion parameters. Full fine-tuning it under standard mixed-precision AdamW:

1.54e9 parameters x 16 bytes = 24.64e9 bytes = about 24.6 GB of static state

Then add roughly 8 GB of activations, at a batch of 4 sequences of 1,024 tokens each. Treat that 8 as an order of magnitude rather than a measurement, because the exact bytes depend on which intermediate results the framework decides to keep, and different frameworks and settings keep different sets. So the run is about 33 GB in total: 24.6 GB of static state plus roughly 8 GB of activations.

The target hardware for every calculation in this series is one 32 GB consumer card. The run wants 33 and the card offers 32, so it does not fit. The gap is also worse than that single gigabyte suggests, once the units are handled honestly.

GB and GiB, because the difference decides this case

Two different units share the same spoken name and differ by about 7 percent, which happens to be exactly the size of the margin you are working in.

• A gigabyte, written GB, is 1,000,000,000 bytes in the decimal sense, which is 10 to the power 9. This is what you get when you multiply parameters by bytes, as above.
• A gibibyte, written GiB, is 1,073,741,824 bytes, which is 2 to the power 30. It is about 7 percent larger than a GB.

GPU vendors, drivers and nvidia-smi all work in the binary unit, even when the box says GB. So a card sold as 32 GB is really 32 GiB of chips, and once the driver and the CUDA context have taken their share you get roughly 29.8 GiB to actually use.

Now compare like with like. Divide each decimal figure by 1.074 to get gibibytes.

• 24.6 GB of static state is 22.9 GiB.
• Roughly 8 GB of activations is roughly 7.5 GiB.
• Together that is about 30.4 GiB, against about 29.8 GiB of usable card.

It does not fit, and that is before you have left any safety margin at all. Leave 2 to 3 GB of headroom in every estimate you make. Memory fragmentation and one unusually long batch both live in that last gigabyte, and both of them produce the same out-of-memory crash at 3 a.m.

The five-second screen

All of this collapses into one multiplication you can do in your head. Parameters times 16 gives decimal GB. Divide by 1.074 for gibibytes. Then add activations separately, because nothing in that multiplication accounts for them.

Every entry in the third column is one multiplication. Every entry in it is also static state only. None of them includes activations, and forgetting that is the most common sizing mistake in the field.

Diagnose a card that is using far more memory than the model’s size

Here is a situation you will meet within a week of touching a GPU. Someone loads a 1B-class model onto a 32 GB card, and nvidia-smi reports 22 GB in use. The model’s weights are a couple of gigabytes. Something looks badly wrong.

There are two common causes. They produce an identical symptom and they call for completely different fixes.

Cause A: the model is being trained. Then the 22 GB is the four tenants doing precisely what the last section said they would do, at 16 bytes per parameter plus activations. Nothing is broken and no memory is being wasted. If you need that number smaller, the move is to remove tenants entirely, which is what the ladder in the next section does.

Cause B: the model is being served through an engine such as vLLM, which grabs most of the card at startup to hold its KV cache pool, and keeps hold of it whether or not any request has arrived. The 22 GB is a reservation, not consumption. The fix is a configuration number: lower gpu_memory_utilization, or lower max_model_len so that each sequence needs less cache. Our part on continuous batching and paged attention explains why serving engines reserve in a block rather than allocating on demand.

Telling the two apart takes one question: is anything on this card computing gradients? If yes, it is A. If the process is a serving engine, it is B. The symptom is identical and the diagnosis is trivial, once you know that the two jobs keep different lists of tenants.

Choose between full fine-tuning, LoRA and QLoRA

Almost every practical fine-tuning decision is a choice between three rungs of one ladder. Each rung removes a cost that the rung below it paid.

Rung 1: full fine-tuning

Every parameter is trainable, so every parameter carries a gradient and an optimizer state. That is where all sixteen bytes come from. You get maximum flexibility and three costs with it. It is the most expensive option in memory. It produces a complete multi-gigabyte model copy for every task you tune. And because it moves every weight, it can quietly degrade abilities the base model already had.

That degradation has a name, catastrophic forgetting. The thing to know now is that your training loss will look excellent the whole time it is happening, because the loss only measures performance on your data. Part 15 covers how to catch it before your users do.

Rung 2: LoRA

LoRA starts from a question. If the model already knows the language and you are only teaching it a behaviour, does the change to the weights really need 1.54 billion degrees of freedom? The empirical answer turned out to be no.

So LoRA freezes the base model. A frozen weight is one the training loop has been told never to change. It still takes part in the forward pass, so it still costs its 2 bytes of storage. It receives no gradient and gets no optimizer state, so 14 of its 16 bytes simply vanish.

Then, beside each weight matrix it targets, LoRA adds a small trainable patch. The arithmetic is worth doing once. Qwen2.5-1.5B moves rows of 1,536 numbers between its layers, so many of its weight matrices, which are simply rectangular grids of numbers, come out at 1,536 by 1,536. That is 2,359,296 numbers in one matrix. Rather than learn a change to all of them, LoRA learns two skinny matrices instead: one that is 1,536 by 16, and one that is 16 by 1,536. Count them: 24,576 plus 24,576, so 49,152 numbers, which is about 2 percent of the original. That 16 is called the rank, written r, and it is the main knob you turn. Multiply the two skinny matrices together and the result has exactly the same shape as the original, so the learned patch can be laid straight on top of it.

Two things change as a result. The static state for a Qwen2.5-1.5B fine-tune drops from about 24.6 GB to about 4 GB. And the file you ship at the end is just those skinny matrices, a few tens of megabytes, which is called an adapter. Twenty different task adapters can sit beside one copy of the base model instead of twenty multi-gigabyte checkpoints. Part 11 is the full treatment.

Rung 3: QLoRA

LoRA removed the gradients and the optimizer state for the frozen base. It could not remove the base’s own 2 bytes per parameter, because the forward pass still has to run through those weights. QLoRA attacks that last cost by storing them in 4 bits instead of 16.

Squeezing a number into fewer bits is called quantisation. You give up decimal places to buy space. QLoRA uses a specific 4-bit code called NF4, designed around the fact that a trained model’s weights cluster near zero in a bell-curve shape, so it spends its sixteen available codes where the weights actually are rather than spreading them evenly. That takes the base from about 2 bytes per parameter down to about 0.5.

The catch sits on the critical path. A 4-bit number cannot be multiplied directly, so before each matrix multiply the values have to be expanded back into bf16. That expansion is called dequantisation, and it costs real time on every forward pass and every backward pass. QLoRA is a fitting tool. Reach for it when a model would not otherwise fit at all. Running it on a model that already fits buys you nothing and costs you speed, and Part 12 puts a number on that slowdown.

Every figure in that table is static state only, meaning weights, gradients and optimizer state. Activations are in none of them, and activations do not shrink under LoRA or QLoRA. Both methods run the same forward and backward pass as a full fine-tune, over the same batch and the same sequence length, so the same roughly 8 GB is still owed on top of every row in the 1.5B column. Add it back before comparing any row against a 32 GB card. At the 7B scale the activation figure moves with batch size and with which memory-saving options are on, so there is no single number worth quoting. Part 3 covers those options.

One honest limit on the ladder before you leave it. For format, tone and instruction following, LoRA usually matches full fine-tuning at a fraction of the cost. For teaching a genuinely new capability such as heavy mathematics or code, Biderman et al. measured LoRA underperforming full fine-tuning, while also forgetting less of what the base model already knew. Both halves of that finding are real, and Part 11 covers the trade in full.