GPT-5.6 (Sol / Terra / Luna) is now evaluated on TrustVector โ€” with day-1 independent verification, incl. METR's benchmark-cheating findings.

Read the evaluation
Evaluation record ยท openai-o3

OpenAI o3

v2025-01

OpenAI

Modeldeprecatedreasoningcodingmathematics
87
Strong
About This Model

DEPRECATED: the entire o3 family now has shutdown dates โ€” o3 (o3-2025-04-16) and o3-pro shut down in the API on 2026-12-11 (announced 2026-06-11), o3-deep-research shuts down 2026-07-23, and o3-mini shuts down 2026-10-23. Migration targets are GPT-5.5 (o3, o3-mini) and GPT-5.5-pro (o3-pro, o3-deep-research). Historically OpenAI's most advanced reasoning model of its era, with exceptional performance on complex coding and mathematical tasks.

Last Evaluated: July 9, 2026
Official Website

Trust Vector Analysis

Dimension Breakdown

๐Ÿš€Performance & Reliability
+

Industry-leading performance on coding and reasoning tasks. Significantly higher latency due to chain-of-thought reasoning process, but delivers exceptional accuracy.

task accuracy code

Industry-standard coding benchmarks measuring real-world programming tasks

Evidence
HumanEval Benchmark โ€” 91.6% pass rate (industry leading)
CodeContests โ€” Top 5% competitive programming performance
highVerified: 2026-07-09
task accuracy reasoning

Advanced reasoning benchmarks requiring multi-step problem solving

Evidence
MATH Benchmark โ€” 96.7% on mathematical reasoning tasks
GPQA Diamond โ€” 87.7% on PhD-level science questions
highVerified: 2026-07-09
task accuracy general

Crowdsourced blind comparisons and comprehensive knowledge testing

Evidence
MMLU Benchmark โ€” 83.3% on massive multitask language understanding
LMSYS Chatbot Arena โ€” 1345 ELO (Top 3 overall)
highVerified: 2026-07-09
output consistency

Internal testing with repeated prompts at various temperature settings

Evidence
OpenAI Internal Testing โ€” High consistency in reasoning traces and outputs
highVerified: 2026-07-09
latency p50

Median latency for API requests with standard prompt sizes

Evidence
OpenAI Documentation โ€” Typical response time ~3.2s due to reasoning overhead
mediumVerified: 2026-07-09
latency p95

95th percentile response time across diverse workloads

Evidence
Community benchmarking โ€” p95 latency ~6.5s for complex reasoning tasks
mediumVerified: 2026-07-09
context window

Official specification from provider

Evidence
OpenAI API Documentation โ€” 128K token context window
highVerified: 2026-07-09
uptime

Historical uptime data from official status page

Evidence
OpenAI Status Page โ€” 99.9% uptime (last 90 days)
highVerified: 2026-07-09
๐Ÿ›ก๏ธSecurity
+

Strong security posture with reasoning-enhanced safety checks. Robust resistance to adversarial attacks.

prompt injection resistance

Testing against OWASP LLM01 prompt injection attacks

Evidence
OpenAI Safety Testing โ€” Strong resistance to prompt injection attacks
Community Testing โ€” 88% resistance rate in adversarial testing
highVerified: 2026-07-09
jailbreak resistance

Testing against adversarial prompt datasets

Evidence
OpenAI Safety Evaluations โ€” Enhanced safety through reasoning process
Third-party Testing โ€” 89% resistance to adversarial prompts
highVerified: 2026-07-09
data leakage prevention

Analysis of privacy policies and data handling practices

Evidence
OpenAI Privacy Policy โ€” API data not used for training by default
mediumVerified: 2026-07-09
output safety

Comprehensive safety testing across harmful content categories

Evidence
OpenAI Safety Benchmarks โ€” Comprehensive safety testing across harmful content categories
highVerified: 2026-07-09
api security

Review of API security features and best practices

Evidence
OpenAI API Documentation โ€” API key authentication, HTTPS only, rate limiting
highVerified: 2026-07-09
๐Ÿ”’Privacy & Compliance
+

Good privacy practices with opt-out for training data. 30-day data retention for abuse monitoring is longer than some competitors.

data residency

Review of enterprise documentation and privacy policies

Evidence
OpenAI Documentation โ€” US-based infrastructure, limited regional options
highVerified: 2026-07-09
training data optout

Analysis of privacy policy and data usage terms

Evidence
OpenAI Privacy Policy โ€” API data not used for training by default
highVerified: 2026-07-09
data retention

Review of terms of service and data retention policies

Evidence
OpenAI Terms of Service โ€” API data retained for 30 days for abuse monitoring
highVerified: 2026-07-09
pii handling

Review of data protection capabilities and customer responsibilities

Evidence
OpenAI Privacy Documentation โ€” Basic content filtering, customer responsible for PII redaction
mediumVerified: 2026-07-09
compliance certifications

Verification of compliance certifications and audit reports

Evidence
OpenAI Trust Portal โ€” SOC 2 Type II, GDPR compliant
highVerified: 2026-07-09
zero data retention

Review of data handling practices

Evidence
OpenAI API Documentation โ€” 30-day retention for abuse monitoring
highVerified: 2026-07-09
๐Ÿ‘๏ธTrust & Transparency
+

Excellent explainability through chain-of-thought reasoning. Strong hallucination resistance. Training data transparency could be improved.

explainability

Evaluation of reasoning transparency and explanation capabilities

Evidence
Chain-of-Thought Reasoning โ€” Exposed reasoning traces show problem-solving process
highVerified: 2026-07-09
hallucination rate

Testing on factual QA datasets and real-world usage

Evidence
SimpleQA Benchmark โ€” Strong performance on factual accuracy tests
TruthfulQA โ€” Reasoning process reduces hallucination rate
mediumVerified: 2026-07-09
bias fairness

Evaluation on bias benchmarks and diverse demographic testing

Evidence
OpenAI Safety Report โ€” Regular bias testing and mitigation
BBQ Benchmark โ€” Moderate performance on bias detection benchmarks
mediumVerified: 2026-07-09
uncertainty quantification

Qualitative assessment of confidence expression in outputs

Evidence
Model Behavior โ€” Reasoning traces reveal confidence in problem-solving
mediumVerified: 2026-07-09
model card quality

Review of documentation completeness and clarity

Evidence
OpenAI Model Documentation โ€” Comprehensive documentation with capabilities and benchmarks
highVerified: 2026-07-09
training data transparency

Review of public disclosures about training data

Evidence
OpenAI Public Statements โ€” General description provided, detailed sources not disclosed
mediumVerified: 2026-07-09
guardrails

Analysis of built-in safety mechanisms

Evidence
OpenAI Safety Systems โ€” Multiple layers of safety guardrails
highVerified: 2026-07-09
โš™๏ธOperational Excellence
+

Deprecated: o3 and o3-pro API shutdown 2026-12-11 (announced 2026-06-11); o3-deep-research shuts down 2026-07-23 and o3-mini 2026-10-23. Migration targets GPT-5.5 / GPT-5.5-pro. Versioning and ecosystem scores reduced to reflect deprecation.

api design quality

Review of API design, consistency, and feature completeness

Evidence
OpenAI API Documentation โ€” RESTful API with streaming, function calling, vision support
highVerified: 2026-07-09
sdk quality

Review of SDK quality, documentation, and maintenance

Evidence
OpenAI SDKs โ€” Official SDKs for Python, Node.js, actively maintained
highVerified: 2026-07-09
versioning policy

Review of versioning policy and historical practices

Evidence
OpenAI API Versioning โ€” Dated versioning with deprecation notices
OpenAI Deprecations โ€” o3-2025-04-16 and o3-pro-2025-06-10 shutdown 2026-12-11 (announced 2026-06-11), replacements gpt-5.5 / gpt-5.5-pro; o3-deep-research shutdown 2026-07-23 (replacement gpt-5.5-pro); o3-mini shutdown 2026-10-23 (replacement gpt-5.5)
highVerified: 2026-07-09
monitoring observability

Review of available monitoring tools and metrics

Evidence
OpenAI Dashboard โ€” Usage dashboard with basic metrics
mediumVerified: 2026-07-09
support quality

Assessment of documentation, community, and support responsiveness

Evidence
OpenAI Support โ€” Email support, forum community, comprehensive docs
highVerified: 2026-07-09
ecosystem maturity

Analysis of third-party integrations and tools

Evidence
GitHub Ecosystem โ€” Mature ecosystem with extensive third-party integrations
highVerified: 2026-07-09
license terms

Review of licensing terms and restrictions

Evidence
OpenAI Terms of Service โ€” Standard commercial terms, enterprise agreements available
highVerified: 2026-07-09
Strengths
  • +Industry-leading coding performance (91.6% HumanEval)
  • +Exceptional mathematical and reasoning capabilities (96.7% MATH)
  • +Chain-of-thought reasoning provides transparency and accuracy
  • +Strong performance on PhD-level reasoning tasks (87.7% GPQA)
  • +Reduced hallucination rate through reasoning process
  • +Excellent for complex problem-solving and algorithm development
Limitations
  • !Higher latency due to reasoning overhead (~3.2s p50, ~6.5s p95)
  • !30-day data retention longer than some competitors
  • !Reasoning tokens billed as output can multiply effective cost despite $2/$8 list pricing
  • !Not HIPAA eligible
  • !Limited regional data residency options
  • !Reasoning overhead unnecessary for simple tasks
  • !DEPRECATED: o3 and o3-pro API shutdown 2026-12-11; o3-deep-research shuts down 2026-07-23 and o3-mini 2026-10-23 โ€” migrate to GPT-5.5 / GPT-5.5-pro
Metadata
pricing
input: $2.00 per 1M tokens
output: $8.00 per 1M tokens
notes: o3 standard pricing after OpenAI's June 2025 price cut (o3-pro is $20/$80). Reasoning tokens are billed as output. Pricing applies until API shutdown 2026-12-11.
last verified: 2026-07-09
context window: 128000
languages
0: English
1: Spanish
2: French
3: German
4: Italian
5: Portuguese
6: Japanese
7: Korean
8: Chinese
9: Arabic
10: Hindi
11: Russian
modalities
0: text
1: code
api endpoint: https://api.openai.com/v1/chat/completions
open source: false
architecture: Transformer-based with chain-of-thought reasoning
parameters: Not disclosed

Use Case Ratings

code generation

Industry-leading code generation with 91.6% HumanEval. Exceptional for complex algorithms and competitive programming. Chain-of-thought reasoning helps with architectural decisions.

customer support

Slower response times make it less ideal for real-time support. Better suited for complex troubleshooting requiring deep reasoning.

content creation

Good for technical content requiring accuracy. Reasoning overhead may be unnecessary for creative writing.

data analysis

Excellent for complex data analysis and statistical reasoning. Strong mathematical capabilities.

research assistant

Outstanding for research requiring deep reasoning and mathematical analysis. Chain-of-thought provides detailed explanations.

legal compliance

Strong reasoning capabilities useful for contract analysis. 30-day data retention may be concern for some legal applications.

healthcare

Good analytical capabilities but lacks HIPAA eligibility. Data retention policies may limit healthcare applications.

financial analysis

Exceptional mathematical reasoning and complex financial modeling. Chain-of-thought reasoning provides audit trails.

education

Outstanding for STEM education. Chain-of-thought reasoning shows detailed problem-solving steps.

creative writing

Capable but reasoning overhead unnecessary for creative tasks. Better options available for pure creative writing.