Skill Suites
Code-first guides on AI, DevOps, and shipping software
Real proof-of-concepts you can run today. No fluff, no filler, just working examples.
Latest articles
-
Data Is the Actual Job: Formats, Quality, Splits and Synthetic Data
Your fine-tuning dataset decides everything the loop does not. Formats, deduplication, decontamination, honest splits, synthetic data and model collapse.
-
Supervised Fine-Tuning End to End: The First Real Run
A full supervised fine-tune twice over: a bare PyTorch loop and the same job through a trainer. Hyperparameters that work, plus how…
-
Choosing a Base Model and Building a Fine-Tuning Bench
How to pick a base model for fine-tuning, three candidates read as datasheets, where to run the bench, and the toolchain stack…
-
Masking in LLM Training: Loss, Causal and Padding
Three things share the name masking. Loss masking with -100, the causal mask the architecture applies, and the padding mask you build.…
-
AdamW Explained, Line by Line
AdamW dissected: where it runs, the lineage behind the name, the four update lines, why decoupled weight decay was a real fix,…
-
Gradients and Optimizers: From SGD to Adam, and Why m and v Cost 8 Bytes
What a gradient is, what an optimizer decides, and why Adam’s two running statistics per weight are worth 8 bytes each on…
-
Number Formats for Training: FP32, FP16, BF16, TF32, FP8 and NF4
bf16 vs fp16, why loss scaling exists, why mixed precision keeps an fp32 master copy, and the difference between a storage format…
-
Activation Memory: Why the Forward Pass Costs More Than the Weights
Activation memory scales with batch, sequence, hidden size and depth, not parameters. The lifecycle, the quadratic attention trap, and every lever that…
-
Training Memory: The Four Tenants and the 16 Bytes Per Parameter
Why a 3 GB model needs 25 GB to train. The four memory tenants derived from first principles, where the 16 bytes…
Browse by topic
- AI Use Cases40 posts
- Java & Spring15 posts
- LLM Inference11 posts
- System Design10 posts
- LLM Fine-Tuning10 posts
- Hosting & SSH9 posts
- AI Security2 posts
- Neural Networks2 posts
