Reasoning model

Reasoning models moved chain-of-thought from the prompt into training. OpenAI's o1 (preview released in September 2024) was described as trained with large-scale reinforcement learning to use a long internal chain of thought, with performance improving both with more RL training compute and with more test-time compute spent thinking. DeepSeek-R1 (January 2025) published a recipe: R1-Zero was trained from a base model with reinforcement learning alone, using Group Relative Policy Optimization (GRPO) and rule-based rewards for answer correctness and output format on maths and code problems whose answers can be checked automatically; lengthening reasoning, self-verification and backtracking emerged without supervised examples. The released R1 added a small cold-start SFT set and further stages to fix readability and language mixing.

Operationally, the model emits reasoning tokens before the visible answer. They occupy the context window, count against the output-token limit and are billed as output tokens even when the provider hides them, which is why cost and latency can be several times those of a non-reasoning call for the same visible answer. Providers expose the trade-off as a control: an effort level, or a token budget for thinking; some return the raw reasoning, others only a summary or nothing. Settings that work on ordinary models - temperature, prefilled answers, few-shot examples of short answers - may be restricted or counter-productive, and "think step by step" instructions add little because the behaviour is already trained in.

The gains concentrate in domains with verifiable answers: competition mathematics, programming, formal logic, multi-step planning and agentic tool use. On lookup-style questions, short classification and extraction tasks the extra tokens mostly add latency, and long reasoning can "overthink" simple problems into wrong answers. Reasoning also does not eliminate hallucination; a model can reason carefully from a false premise it invented.

The visible reasoning is not a faithful log of the computation. In Anthropic's 2025 study "Reasoning Models Don't Always Say What They Think", models were given hints to the answer and then checked for whether their chain of thought admitted using them: averaged across hint types, Claude 3.7 Sonnet mentioned the hint 25% of the time and DeepSeek-R1 39%. Reasoning traces are useful for debugging and for monitoring misbehaviour, but they should not be treated as an audit trail or as an explanation in the sense required for decisions about people. They can also contain text the final answer would have filtered out, such as quoted sensitive input, so logging and display of reasoning need the same data-handling rules as other model output.