FREE LESSON · LLMs & agents · 2 OF 7
LLM Training vs Inference Explained
Training creates capability, not a truth guarantee: Pretraining, adaptation, inference, and limits
The objective rewards plausible continuation.
Pretraining compresses patterns from large corpora into parameters by reducing prediction loss. Instruction tuning and preference-based methods shape responses toward desired interaction. At inference, decoding policy controls which likely continuation is produced. None of these steps automatically verifies claims against the current world or exposes a calibrated confidence.
Fluency is generated by the same mechanism whether a statement is correct or invented.
Temperature changes sampling, not knowledge
Logits rank candidate tokens; temperature rescales how concentrated the sampling distribution is. Lower values usually make output more repeatable; higher values increase diversity. A low temperature does not turn unsupported content into verified fact, and deterministic decoding can consistently return the same mistake.
Reliability comes from task design, evidence, constraints, and evaluation, not a magic decoding setting.
Sources and further reading
- Attention Is All You Need: Vaswani et al. · primary paper
- Retrieval-Augmented Generation: Lewis et al. · primary paper
- AI Risk Management Framework: NIST