Updated: 2026-07-07

GPT-4o ("o" = omni) was presented by OpenAI on May 13, 2024. What’s new isn’t that GPT-4 can process image and audio (that existed via separate APIs), but that a single native model processes text, image, and audio for both input and output. Result: human-conversation latency (~320ms), better multimodal understanding, and a price 50% lower than GPT-4 Turbo.

Key takeaways

  • The three modalities (text, vision, audio) are fused in the same base model: the model hears tone, emotion, and interruptions, not just transcribed words.

  • ~50% cheaper than GPT-4 Turbo: $5/1M input tokens vs $10.

  • ~320ms latency in audio mode: genuinely natural conversation.

  • HumanEval 90%: best coding result among models available in May 2024.

  • The Realtime API bidirectional WebSocket opens voice-first applications that previously required complex pipelines.

What’s different

All three modalities are now fused in the same base model:

  • Text: quality comparable to, or slightly above, GPT-4 Turbo.

  • Vision: more integrated image analysis than GPT-4V; advanced OCR, diagram analysis, and screenshot understanding for UI automation.

  • Audio: native input (transcription plus emotional understanding) and output (synthesis). The model understands tone, emotion, and interruptions.

  • Video: video frames as inputs for screen-recording analysis or content moderation.

The difference vs the "Whisper to GPT-4 to TTS" pipeline is latency and information preservation: the previous pipeline lost voice tone, emotion, and pauses. GPT-4o processes them directly.

Pricing and speed

Model Input Output First token
GPT-4 Turbo $10/1M $30/1M ~1000ms
GPT-4o $5/1M $15/1M ~500ms
GPT-4o mini $0.15/1M $0.60/1M ~300ms

For real-time chat apps, the latency gap is noticeable to the end user.

Benchmarks

Benchmark GPT-4o GPT-4 Turbo Claude 3 Opus
MMLU 88.7 86.4 86.8
HumanEval 90.2 85.4 84.9
GSM8K 95.8 92.0 95.0

GPT-4o stands out especially in coding (HumanEval 90%) and multilingual tasks.

Unlocked use cases

GPT-4o makes practical what previously required a complex pipeline: voice assistants with real conversational latency for phone-based customer service and interactive tutors, accessibility interfaces for people with visual or motor limitations, simultaneous translation in meetings with tone preservation, complex document understanding (forms, invoices, technical drawings) in a single API call, and UI automation that describes screenshots for testing or support pipelines.

The real-time API

The most impactful post-launch novelty: a bidirectional WebSocket connection with audio streaming. The client can interrupt, the server detects the interruption and adjusts the response. Real 320ms latency.

# Bidirectional WebSocket connection
# Audio stream in -> understanding + reasoning -> audio stream out
# The model detects interruptions and adjusts in real time

This opens patterns previously impractical: phone support bots, voice interfaces for IoT, interactive tutorials with instant feedback. The integration complexity (WebSocket plus audio streaming) is real, but the result is qualitatively different from any prior pipeline.

Honest limitations

With honesty:

  • Audio bandwidth in current models is ~24 kHz: enough for voice, not for high-fidelity audio.

  • Tool use plus audio in the same call is more complex than in text mode.

  • Vision hallucinations: low-quality OCR or confusing text in images produces errors more often than in plain text.

  • The context window is 128k tokens, the same as GPT-4 Turbo, smaller than Claude 3 Opus or Gemini 1.5.

  • Audio cost adds up fast: roughly $0.06/min of input, $0.24/min of output.

GPT-4o vs GPT-4 Turbo

After several weeks of production use:

  • Text quality: GPT-4o matches or slightly beats Turbo on most tasks.

  • Complex reasoning in edge cases: Turbo is occasionally better.

  • Latency: GPT-4o wins clearly.

  • Cost: GPT-4o wins clearly.

  • Multimodal: GPT-4o wins clearly.

Default for new applications: GPT-4o. Stay on Turbo only for reasoning-heavy cases you have verified empirically favour it.

Conclusion

GPT-4o is the starting point for evaluating what can now be built with multimodal LLMs. Fusing modalities into a single model changes the economics of applications that depend on voice or vision: you no longer need to orchestrate three separate services to get a natural conversation. The Realtime API for voice is the most real differentiator for voice-first applications.

Frontier models move fast: Claude 3.5 Sonnet answered a month later, and months after that GPT-5 merged GPT-4o and o3 into a single adaptive model. Even so, GPT-4o remains a genuine leap in price, quality, and modality for anyone building on the OpenAI API today.

This article is also available in Spanish: GPT-4o: multimodalidad nativa de OpenAI.

Sources

  1. OpenAI: Hello GPT-4o
  2. TechCrunch: OpenAI debuts GPT-4o ‘omni’ model now powering ChatGPT
  3. Wikipedia: GPT-4o
  4. CNN Business: OpenAI unveils newest AI model, GPT-4o