OpenAI introduced o1-preview and o1-mini on September 12, 2024 as a new model family with a philosophical difference from GPT-4o: they reason internally before emitting the final answer. It’s not just visible chain-of-thought; it’s a hidden process where the model elaborates, reconsiders, explores paths, and only then responds. Results in mathematics and code are dramatic. For other tasks, the latency and cost trade-off isn’t always worth it.

Key takeaways

  • o1 introduces a hidden "thinking" phase where the model elaborates before responding; reasoning tokens are billed but not shown.

  • In AIME it scores 83% vs GPT-4o’s 13%; in programming competitions (Codeforces) it sits at 89th percentile vs GPT-4o’s 11th.

  • o1-preview costs $15/M input tokens and $60/M output; o1-mini is cheaper but more limited.

  • In the preview version it doesn’t support function calling, streaming or multimodal vision.

  • For chatbots, summaries, and quick questions, GPT-4o or Claude 3.5 Sonnet remain more efficient.

What makes it different

Traditional models (GPT-4o, Claude 3.5 Sonnet) generate response tokens sequentially from the first token. o1 introduces a prior user-invisible "thinking" phase where the model can reformulate the problem, explore approaches, discard strategies, before elaborating the answer. This internal reasoning consumes additional tokens (so-called reasoning tokens), which are billed but not shown to the user.

The approach is partially inspired by techniques like Tree of Thoughts and MCTS but integrated at training level, not prompt level.

Where it excels

Benchmarks tell a clear story:

  • On maths problems like AIME, o1-preview scores 83% vs GPT-4o’s 13%.

  • On PhD-level physics problems, o1 reaches 78% vs 57% of human experts.

  • In programming competitions like Codeforces, o1-preview sits at 89th percentile vs GPT-4o’s 11th.

For tasks requiring long chains of reasoning (olympiad maths, proofs, complex debugging, deep causal analysis), the qualitative leap is real and not just statistical.

Where it doesn’t add much

For conversational tasks, creative writing, simple summaries, direct factual questions, o1 offers no significant advantage over GPT-4o, while adding latency and cost. A response GPT-4o generates in one second may take ten to twenty in o1 while it "thinks". For a fluid conversation, that breaks the experience.

Additionally, o1 has architectural limitations in the preview version:

  • Doesn’t support function calling the same way as GPT-4o.

  • Doesn’t stream.

  • Isn’t multimodal.

For workflows that depend on these capabilities, it’s not a direct replacement.

The cost factor

o1-preview costs $15 per million input tokens and $60 per million output. o1-mini is cheaper: $3 input, $12 output, and it is the sweet spot when the use benefits from reasoning. For comparison, GPT-4o costs $2.50 and $10 respectively.

Real cost is higher than nominal because hidden reasoning tokens consume output billing. An apparently short response may have internally consumed ten times the visible tokens.

When to use and when not

The pragmatic rule after weeks of use: for problems where the answer requires chained, step-by-step reasoning and where correctness matters more than speed, o1 is worth it:

  • Complex technical research with multiple interdependent variables.

  • Legal analysis with multiple premises and exceptions.

  • Maths or programming problems with rich structure.

  • Strategic planning with chained-consequence decisions.

For chatbots, content generation, summaries, translations, quick questions or function calling, o1 introduces cost and latency without proportional benefit. GPT-4o or Claude 3.5 Sonnet are better choices in those contexts.

Industry impact

o1 marked a conceptual shift. Previously, progress came mainly from scaling parameters and data (GPT-3 to GPT-4). o1 shows that scaling compute at inference time (giving the model more tokens to think) also produces qualitative leaps. This opens a new scaling dimension that doesn’t require larger models.

Shortly after, Anthropic announced similar capabilities in later Claude versions. Google is preparing its response in the Gemini family. The competition in reasoning models is now an active battleground.

Practical integration

For teams looking to incorporate o1, the pragmatic approach is multi-model routing: use GPT-4o or Claude for most queries and escalate to o1 only when the task justifies it. Tools like LiteLLM make this pattern straightforward with a unified proxy that routes based on task complexity or type.

Another useful pattern is the "reviewer" mode: o1 reviews responses produced by cheaper models and flags reasoning errors. The cost per review is low because the responses are already structured; the quality benefit can be high for critical tasks.

Honest limitations

  • Internal reasoning isn’t transparent: OpenAI explicitly hides reasoning tokens from users. This generates legitimate concerns about auditing and debugging.

  • Published benchmarks are somewhat cherry-picked. o1 isn’t universally superior: on everyday tasks it ties or loses to GPT-4o when normalised for cost.

  • Open questions about sustainability: if each model generation requires ten times more inferential tokens, carbon footprint and economic cost scale exponentially.

Conclusion

o1 represents an inflection point in how we think about language models. It’s not a universal GPT-4o replacement but a specialised complement for deep reasoning. For problems where correctness matters more than speed, it’s worth every extra cent.

For most everyday uses, traditional models remain more efficient. The direction it marks, scaling inference compute for reasoning, is probably the next dominant paradigm. Knowing when to apply it forms part of essential technical repertoire for any engineer integrating LLMs in production.

Frequently asked questions

How much does o1-preview cost compared with GPT-4o, and why does the bill come out higher than expected?

o1-preview costs $15 per million input tokens and $60 per million output; o1-mini, $3 and $12; GPT-4o, $2.50 and $10. Real cost exceeds nominal cost because the hidden reasoning tokens from the thinking phase are billed as output even though they are never shown. An apparently short response may have internally consumed ten times the visible tokens. That adds up in a high-volume application.

Can I use o1-preview with function calling, streaming or images?

In the preview version, no: o1 does not support function calling the same way as GPT-4o, does not stream and is not multimodal. For workflows that depend on those capabilities it is not a direct replacement. Also, a response GPT-4o generates in one second may take ten to twenty seconds in o1 while it thinks, which breaks a fluid conversation. For chatbots, summaries or quick questions, GPT-4o or Claude 3.5 Sonnet remain more efficient.

How do I integrate o1 into my stack without paying for reasoning on every query?

With multi-model routing: use GPT-4o or Claude by default, and escalate to o1 only when the task requires chained, step-by-step reasoning and correctness matters more than speed. Tools like LiteLLM make that pattern straightforward with a unified proxy that routes by task complexity or type. Another useful pattern is reviewer mode: o1 reviews responses produced by cheaper models and flags reasoning errors, at low cost because those responses are already structured.