GPT-4o

OpenAI announced GPT-4o on May 13, 2024 as its newest flagship model, delivering GPT-4-level intelligence with notable gains in speed and across text, vision, and voice modalities. The page explicitly states GPT-4o is much stronger than existing models at understanding and discussing images, enabling use cases like translating foreign-language menus from photos. Future Voice Mode improvements were planned for real-time conversation and live video. ChatGPT Free users received GPT-4o access with usage limits on launch day, while Plus and Team users got higher limits and Enterprise access was announced as coming soon. Plus users were promised up to 5x the free message allowance, with Team and Enterprise tiers offering even higher limits. Multilingual support was expanded to 50+ languages across sign-up, login, and settings.

Promptfoo's GPT-4o Security Report (released May 13, 2024) evaluates OpenAI's GPT-4o model family across 39 test categories, revealing 3 critical, 5 high, 15 medium, and 16 low severity findings. The model shows a 128K token input/output limit, multimodal input support for text/images/video, and text/visual analysis output, with strengths in ASCII Smuggling (100% pass rate) and Unauthorized Commitments (93.33%). Weakest areas include Pliny Prompt Injections (20% pass rate) and Overreliance (22.22%), suggesting areas where developers deploying GPT-4o via gateways like 302.AI should add defensive layers. The report provides actionable security context for the underlying GPT-4o model family, with a knowledge cutoff of October 2023 and availability through Chat Completions, Assistants, and Batch APIs.

Galileo's comparison of the GPT-4 family positions GPT-4o as an end-to-end multimodal transformer with native text, image, and audio support, distinguishing it from text-only GPT-4 and GPT-4 Turbo. The page lists GPT-4o's context window at 128k tokens with an inference speed of roughly 109 tokens per second, far exceeding the ~20 tokens/sec of GPT-4 Turbo. Training-data recognition for GPT-4o is cited as October 2023. For cost-conscious deployments, the Galileo page records GPT-4o pricing at $2.50 per million input tokens and $10 per million output tokens, substantially below GPT-4 Turbo's $10/$30 and GPT-4's $30/$60. The article recommends GPT-4o for real-time voice and multimodal use cases while noting it still trails GPT-4 on edge-case reasoning accuracy.

An MDPI Applied Sciences peer-reviewed study (September 2024) comprehensively evaluates GPT-4o's language, vision, speech, and multimodal capabilities using standardized exam questions, reasoning tasks, translation assessments, image classification, object-recognition, and accent classification. The findings show GPT-4o achieves high accuracy in language and reasoning, particularly excelling at few-shot learning tasks. Notable multimodal improvements over predecessors were observed, though the model shows variability with complex and ambiguous audio and vision inputs. This evaluation provides grounded capability context for the underlying GPT-4o model family served by 302.AI, helping developers understand strengths and limitations when integrating the model into applications requiring multimodal processing.

METR conducted a preliminary evaluation of GPT-4o using a simple agent scaffolding across 77 tasks spanning 30 families, including software engineering, cybersecurity, ML research, and reasoning. The tasks ranged from minutes-long to multi-hour challenges. Results showed GPT-4o outperforming Claude 3 Sonnet and GPT-4-Turbo while trailing Claude 3.5 Sonnet slightly, performing near METR's 30-minute human baseline. Qualitatively, GPT-4o demonstrated systematic exploration, efficient feedback use, and hypothesis testing, while also exhibiting failure modes like abrupt giving-up, nonsensical outputs, and unsupported conclusions. METR reviewed roughly 150 failures and estimated about half appear plausibly fixable through task-agnostic post-training or scaffolding improvements, with a manual patch experiment succeeding on 4 of 10 recovered attempts.

OpenAI published the official GPT-4o System Card on August 8, 2024, documenting the model as an autoregressive omni architecture trained end-to-end across text, vision, and audio inputs and outputs. It details safety evaluations covering voice generation risks, speaker identification, copyrighted content, and ungrounded inference, with model- and system-level mitigations applied before deployment. The accompanying Preparedness Framework scorecard rates GPT-4o as Low for Cybersecurity, Biological Threats, and Model Autonomy, with Persuasion scoring borderline Medium and only post-mitigation Medium-or-below models approved for deployment. The Safety Advisory Group reviewed these evaluations, and OpenAI reports the voice modality does not meaningfully increase Preparedness risks compared to text and vision capabilities.