Tag: loss masking
-
Masking in LLM Training: Loss, Causal and Padding
Three things share the name masking. Loss masking with -100, the causal mask the architecture applies, and the padding mask you build.…
Three things share the name masking. Loss masking with -100, the causal mask the architecture applies, and the padding mask you build.…