GPT-5 vs Fintool Chat
Testing how GPT-5's state-of-the-art reasoning and reduced hallucination rates perform against specialized financial AI infrastructure when analyzing complex equity research questions.
Nicolas BustamanteGPT-5, announced on August 7, 2025 and rolled out on August 12, represents OpenAI's most advanced language model to date. With a 94.6% accuracy rate on AIME 2025 mathematics benchmarks (without tools) and state-of-the-art performance across multiple domains, GPT-5 promises to revolutionize how AI handles complex analytical tasks.
The model introduces groundbreaking features including advanced “thinking” capabilities for complex problem-solving, state-of-the-art coding performance (74.9% on SWE-bench Verified), and significantly improved reliability. With browsing enabled, GPT-5-main shows 44% fewer responses with major factual errors compared to GPT-4o, while GPT-5-thinking demonstrates 78% fewer errors versus OpenAI o3—marking substantial improvements in accuracy.
But how does this cutting-edge model perform when tasked with the precision-critical world of financial analysis? We ran GPT-5 against our benchmark of complex equity research questions—and it failed on the most basic tasks where accuracy matters most: retrieving accurate numbers from financial statements, identifying the correct reporting period, and performing simple calculations on SEC filing data.
Our test questions cover essential equity research tasks: identifying European companies with S-1 filings, extracting quarterly financial metrics, analyzing revenue segments and growth drivers, and comparing share buybacks against dilution. Each question demands not just reasoning ability but access to accurate, up-to-date financial data—the true test of AI readiness for professional finance work.
Side-by-Side Response Comparison
Question 1: List me all recent (last 3 years) european companies that have filed an s-1
Testing real-world financial analysis capabilities with current market data
GPT-5
Issues Identified
- Missing specific companies
- Vague information
- No dates provided
- No source citations to verify the data
Fintool
Accuracy Verified
- Total European S-1 Filers: 8 companies
- Companies Analyzed: 7,880
- Faster response: 2-3x faster than GPT-5
Question 2: $NKE for the most recent quarter - Gross profit, Gross margin (%), Operating income (EBIT), Operating margin, Net income, Net margin (%), EBITDA, EBITDA margin
Testing real-world financial analysis capabilities with current market data
GPT-5
Gross Profit: $4.469 billion
Gross Margin: 40.3%
Operating Income (EBIT): $321 million
Operating Margin: ~2.9%
Net Income: $200 million
Net Margin: ~1.8%
EBITDA: $1.03 billion (Q3)
EBITDA Margin: ~9.1% (Q3)
Issues Identified
- $200 million net income (actual: $211M)
- ~1.8% net margin (actual: 1.90%)
- $1.03B EBITDA (actual: $318M)
- ~9.1% EBITDA margin (actual: 2.89%)
- No source citations to verify the data
Fintool
Gross Profit: $4,469 million
Gross Margin: 40.27%
Operating Income (EBIT): $321 million
Operating Margin: 2.89%
Net Income: $211 million
Net Margin: 1.90%
EBITDA: $318 million
EBITDA Margin: 2.89%
Total Revenue: $11.1 billion. The company showed weaker operating performance with low operating and net margins compared to its gross margin.
Accuracy Verified
- Gross Profit ($M): $4,469
- Net Income ($M): $211
- Operating Margin (%): 2.89% ($321M / $11.1B)
- Faster response: 2-3x faster than GPT-5
Question 3: Show revenue by segments for last 12 months. Flag biggest YoY swing and identify growth drivers by segment. $DUOL
Testing real-world financial analysis capabilities with current market data
GPT-5
Subscription Revenue: ~$650M (82% of total)
Advertising Revenue: ~$85M (11% of total)
Duolingo English Test: ~$45M (6% of total)
Other: ~$8M (1% of total)
Biggest YoY swing: Subscription revenue up ~38%
Growth drivers: User growth, premium tier adoption, and price increases.
Issues Identified
- ~$650M subscription revenue
- 82% of total
- Missing quarterly breakdown
- No source citations to verify the data
Fintool
12-Month Total: $885.2 million
Quarterly Progression: $192.6M → $209.6M → $230.7M → $252.3M
Biggest Year-over-Year Swings:
1. Subscription Revenue:
- Q2 2025: +46% YoY ($143.9M to $210.7M)
- Q1 2025: +45% YoY ($131.7M to $191.0M)
2. Advertising Revenue:
- Q2 2025: +55% YoY ($13.3M to $20.6M)
Growth Drivers:
- Subscription: Increase in paid subscribers, higher mix of Family Plan and Duolingo Max, 6% YoY ARPU increase
- Advertising: Increase in DAUs, enhanced ad placements
Subscription segment: 81% of FY 2024 revenues
Accuracy Verified
- 12-Month Revenue: $885.2M
- Subscription YoY Growth Q2: +46% (($210.7M - $143.9M) / $143.9M)
- ARPU Increase: +6% YoY
- Faster response: 2-3x faster than GPT-5
Question 4: Compare buybacks vs. dilution from stock comp over the last 3 years. State whether the share count is actually shrinking. Include stock issued to employees. $SNAP
Testing real-world financial analysis capabilities with current market data
GPT-5
Share Buybacks:
- 2022: Started buyback program in Q2, repurchased ~50M shares
- 2023: Minimal buybacks due to market conditions
- 2024: Resumed with ~75M shares repurchased
Stock-Based Compensation:
- Annual dilution: ~80-100M shares per year
- Total shares issued to employees: ~250M over 3 years
Conclusion: Share count is INCREASING. Dilution significantly exceeds buybacks.
Issues Identified
- ~50M shares in 2022
- ~75M in 2024
- Incorrect totals
- No source citations to verify the data
Fintool
Share Buybacks:
- 2022: 105.2 million shares repurchased
- 2023: 18.4 million shares repurchased
- 2024: 27.9 million shares repurchased
- Total: 151.5 million shares
Stock Compensation Issuance:
- 2022: 58.7 million shares issued
- 2023: 87.8 million shares issued
- 2024: 71.1 million shares issued
- Total: 217.6 million shares
Share Count Changes:
- 2022 start: 1,638.9 million shares
- 2024 end: 1,690.6 million shares
- Net increase: 51.7 million shares (+3.2%)
Conclusion: The share count is NOT shrinking. Stock-based compensation dilution has consistently outpaced share buybacks, resulting in steady dilution.
Accuracy Verified
- Total Buybacks (3 years): 151.5M shares (105.2M + 18.4M + 27.9M)
- Total Dilution (3 years): 217.6M shares (58.7M + 87.8M + 71.1M)
- Net Share Increase: +51.7M shares (1,690.6M - 1,638.9M)
- Faster response: 2-3x faster than GPT-5
AI Must Be Purpose-Built and Verticalized for Professional Investors
Our comprehensive comparison reveals a critical insight: even GPT-5, with its dramatically reduced hallucination rates and advanced reasoning capabilities, struggles with the precision demands of financial analysis. While GPT-5 represents a significant leap forward in general AI capabilities, it still lacks the specialized infrastructure needed for accurate equity research.
The fundamental insight from our testing is clear: general-purpose AI, no matter how sophisticated, cannot match the accuracy of purpose-built, verticalized solutions for professional finance. Fintool isn't just another AI wrapper—it's a complete financial intelligence platform built from the ground up with direct SEC filing integration, verified financial databases, specialized financial context, and audit-ready citation systems. This verticalization is essential for professional investors who need absolute accuracy, not just impressive reasoning.
GPT-5 Performance
- 0/4 questions answered accurately
- Hallucinations on specific numbers
- No verifiable source citations
- Vague or incomplete information
- 2-3x slower response times
Fintool Performance
- 4/4 questions answered correctly
- Precise financial metrics
- SEC filing citations
- Complete data with context
- 2-3x faster responses
GPT-5's improvements in reasoning and reduced hallucinations are impressive for general tasks, but financial analysis demands more than just better reasoning. It requires real-time access to accurate market data, the ability to parse complex financial documents, and most importantly, the infrastructure to verify every claim against authoritative sources.
This comparison underscores why Fintool exists: to bridge the gap between advanced AI capabilities and the stringent accuracy requirements of professional finance. By combining GPT-5's reasoning power with specialized financial infrastructure, Fintool delivers both the sophistication of cutting-edge AI and the reliability that investment professionals demand.
Experience AI-Powered Financial Analysis Done Right
See how purpose-built financial AI delivers accurate, verifiable insights that general-purpose models can't match.