Inference is the model doing the work right now, on this specific request. Training already happened; inference is the model spending that training the moment you hit send.
Think about a trained chef cooking tonight’s dinner. The chef learned for years, but dinner doesn’t get cooked in culinary school. It’s cooked when you place the order, give the allergy notes, and the chef works with the ingredients in front of them. A model is similar: training is how it learned patterns, inference is when it uses them for this request. It can feel like the AI is learning every time it answers, but in normal use it’s running the model, not updating its brain.
How it shows up
When you ask Claude to summarize a transcript, inference is the run that turns transcript into summary. When Codex reads a codebase and suggests a patch, inference happens while it reads, reasons, and writes. That’s why input matters so much: a great chef handed stale ingredients and vague instructions still makes a poor dinner, and a strong model with weak context still gives weak output. Inference can include reasoning, where a model spends extra internal steps thinking before it answers, and you may see streaming, where the answer appears word by word while the run is still going.
Why you care
Inference is where cost, speed, and quality meet. Every run uses compute: the more context you send, the more it reads; the longer the answer, the more it writes. That’s why we’re practical about the right model for the right job. You don’t need the strongest model to sort simple items with clear rules, but you probably do for strategy, ambiguous writing, or a messy codebase. AI quality is more than “which model did you buy?” It’s the model, the prompt, the context, and the run happening right now.