Contents · 5 sections
DeepFinanceBench is an ultra long-horizon benchmark for measuring models on real-world Private Equity and Investment Banking workflows that would take a human professional between 20 and 30 hours to complete. DFB was composed and reviewed by Private Equity (PE) and Investment Banking (IB) professionals from firms like J.P. Morgan, UBS, and Evercore, and institutions like Harvard Business School, Wharton, and Columbia Business School.
- Task count: 21
- Average human time: 27.1 hours per task
- Average rubric criteria: 84 criteria per task
- Average environment size: 19 files per data room
- Average prompt length: 240 words, ~29% shorter than GDPval prompts
Philosophy
There is a fundamental gap in how AI models are evaluated on real-world high-finance work. We believe this gap stems from a deeply flawed understanding of how analysts in high-finance perform real-world tasks.
Existing evals measure the models in two ways: their ability to retrieve answers from a set of financial documents and their ability to produce numerically correct final outputs. However, real-world Private Equity and Investment Banking workflows are meaningfully more complicated.
In software benchmarks (like SWE-bench and DeepSWE), models are graded on their output's ability to pass tests. If tests pass and the fix works, the path (or trajectory) the agent took matters less. Finance has no similar test suite. A high-finance valuation is like a pull request that has to survive line-by-line code review, where every number cites its source and every assumption needs to hold up in front of the MD, the IC, and a client.
In PE and IB, while two analysts could produce identical numbers for the same task, one output could rest on completely incorrect assumptions. Change a single number upstream, and the entire workflow no longer makes sense. The trajectory of an ultra long-horizon high-finance task is full of assumptions and judgment calls. Without being able to audit the steps and judgment of the analyst, a professional in high-finance cannot blindly trust the work that is presented to them.
Failure modes of current evals
Existing high-finance benchmarks have two main limitations in how they currently evaluate models in high-finance tasks:
- Overemphasizing question-answer ability from financial documents
- Grading that is limited to numerical correctness of the outputs
Existing benchmarks often fail to consider the real-world working environment that high-finance professionals face daily. A set of financial documents and revisions in varied formats, long email chains, sparse meeting notes, and other data room objects almost invariably contains conflicting information that must be reconciled. The model's ability to judge and choose from overlapping sources is rarely measured.
- FinanceBench: Questions are simple and too easy; this is not realistic or representative of the deal work analysts are hired for.
- Finance Agent Benchmark: Each question is a short research task on public SEC filings (average human time: 16.8 minutes; DFB up to 20 hours).
- BizBench: Checks one final number against an answer key; DFB grades end-to-end workflows against up to 140 criteria.
- FinanceQA: Spoon-feeds the model relevant 10-K sections with each question, leaving no room to evaluate model trajectory.
- BigFinanceBench: Uses solid trajectory reasoning but limits the scope of questions and stops well short of end-to-end workflows.
Benchmarks like OfficeQA by Databricks are great at properly tracking a company taxonomy, but rarely step beyond grading pure numerical correctness. For high-finance ultra long-horizon tasks, trajectory is crucial to understand and audit the underlying method and assumptions that were put into the work.
ELO-based preference ranking used by benchmarks like GDPval misses the nuance in the complex long-horizon workflows by only comparing binary final outputs, and doesn't grade on the atomic trajectory.
There are benchmarks, like Halluminate's Westworld Finance Diligence Bench, that have attempted to measure semi-long-horizon work. These benchmarks only study the agent's trajectory and leave it out of the score, since every point on their leaderboard is from checks on the final output. While this might be an effective way to check final answers, this fundamentally is not how finance work is judged in the real-world. Correct numbers without a source or auditability will be scrutinized by an investment committee. DFB grades whether every number is sourced and every judgment call is auditable and traceable.
These highlighted approaches do not represent real-world uses. DFB was built to demonstrate that the path of the task itself carries the assumptions and therefore grading must account for the trajectory when evaluating model performance. Auditability is paramount to internal and external stakeholders like lenders, clients, and brokers. Understanding how a model (or an analyst) came to an assumption is the key to understanding what could go right, wrong, or sideways in a multi-million dollar deal.
Methodology
Task construction
DeepFinanceBench covers four primary pillars of high-finance work:
- Valuation: estimate what a business is worth and justify the assumptions.
- Due diligence: investigate the records, identify important risks, and explain their effect on the deal.
- Underwriting: assess the deal's risks and economics, then determine acceptable financing or investment terms.
- Financial modeling: build a model whose assumptions, calculations, and outputs remain consistent.
Each ultra long-horizon task pairs a request with a data room: the collection of records needed to complete the assignment. Tasks use real data that has been scrubbed and anonymized. The request states the objective, constraints, and expected deliverables; from there the agent must work out how to complete the analysis. Every DFB problem statement has on average 240 words; we find this to be realistic in crafting a real-world end-to-end workflow. This compares to other evals that have prompts that are very short, but only handle simple arithmetic or quantitative tasks.
Each DFB task comes with a data room of on average 22 files, comprising the same material analysts work with from day-to-day. File types include spreadsheets (.xlsx), Word documents (.docx), emails (.eml), PDFs, and raw CSV extracts. Inside are emails, chats, and deal-team notes; models and projections; CIMs, SPAs, term sheets, commitment letters, and transaction data; diligence reports; IC minutes; and market data and comps.
Each ultra long-horizon task is representative of end-to-end work done at top PE and IB firms like Evercore, J.P. Morgan, and UBS. Every task uses real data that has been anonymized and scrubbed. The work simulates analyst work on real deals, not just simple Q&A on sets of documents or finding one off answers. There is ambiguity, and there are conflicting sources.
Scoring
Graders in DFB evaluate the finished deliverable together with evidence of the trajectory taken. Sources, tool results, calculations, and revisions can show whether the agent completed the required work, and agents are graded on labeled rubrics with label categories:
- Output: Are the calculations, conclusions, and requested deliverables correct?
- Workflow: Did the agent find the relevant evidence and use it in the analysis?
- Judgment: Does the evidence support the agent's assumptions and decisions?
Each rubric criterion was written and verified by experts from the world's top PE and IB firms.
By tracking the steps taken with respect to these three categories, experts and reviewers are able to audit both workflow and judgment, irrespective of final output. When it comes to ultra long-horizon tasks that can take 20-30+ hours for an analyst to complete, trajectory is invaluable.
DFB tasks test whether an agent can carry decisions through a long assignment and abide by "best practices" along the way. A source chosen early may affect a forecast, the amount a company can borrow, its valuation, and the final recommendation. When an input changes, every calculation that depends on it needs to change too. But that doesn't mean that an agent should only be penalized for upstream decisions that go wrong; we should also evaluate the quality of steps and judgments made along the way, irrespective of output. This is how we capture trajectory.
DFB graders give partial credit when the models take the right steps on the trajectory.
Quality assurance
Every task in DFB was reviewed by multiple layers of agentic verifiers and human expert peers with backgrounds from top firms like Evercore and institutions like Harvard Business School and Wharton. Each task on average has 90 rubric criteria.
Expert review and verifiers established four parameters:
- The task is clear and the source files contain enough information to complete it.
- The reference solution is supported by the evidence.
- The rubric checks the requested work and accepts valid solutions.
- A failed criterion reflects the agent's work, rather than a faulty reference, grader, or environment issue.
Reviewers traced disputed grades and expert commentary back to the source files and agent traces. This distinguishes model mistakes from evaluation mistakes, ensuring that errors in the evaluation process do not influence our conclusions about model capability.
Harness
Unlike many benchmarks, DeepFinanceBench evaluates models in the harness they are most likely to be used in. This means that our environment supports a multi-harness setup where:
- Claude models run in claude-code v.2.1.318
- GPT models run in codex v.0.166.0
- Gemini, Muse, DeepSeek, GLM, and Kimi models run in opencode-ai v.1.19.3
- Qwen models run in qwen-code v.0.26.6
- Grok models run in x.ai v.1.0.70
- Mistral models run in mistral-vibe v.3.0.4
- Node.js runs on v.22.23.4
Each harness has nearly every native tool enabled, including web search. The one exception is the ask-user tool, which we turn off so every run can finish on its own. Many current evals cut off the agent's access to the internet so agents cannot look up the answer keys. DFB doesn't need to. Every run gets its own sandbox, and the answer key and rubric sit in separate, isolated grading environments.
The goal of DFB is to test agents in environments that are representative of those that PE and IB professionals use in their day-to-day work. This approach provides a more realistic evaluation of model performance rather than using a single universal harness that imposes arbitrary or artificial constraints on the models.
Results
Quantitative Analysis
GPT-6.1 class models dominate the Pareto frontier
GPT-6.1 Sol with Extra high reasoning averaged $0.78 per completed task with a 54% score. The next cheapest model scoring above 30% was GPT-6.1 Terra at $0.36 per completed task, with a 44% score. The GPT-6 family accounted for 3 of the 4 cheapest models for finance across the 16 tested.
Score vs. cost per task
Fable for Finance: not worth the cost
While Fable 5.1 has been regarded for its intelligence, this did not translate to high-finance work: it scored below GPT-6 Astra, Claude Opus 5.5, Claude Sonnet 5.5, and GPT-6.1 Sol. Furthermore, Fable 5.1 was the most expensive model we tested, at $11.53 per task, while scoring 46% on end-to-end tasks.
Score vs. tokens per task
DeepFinanceBench score
Mean score out of 100 across 20 tasks, 10 attempts each. Higher is better.
| Rank | Model | Lab | Score (0–100) | 95% confidence interval |
|---|---|---|---|---|
| 1 | GPT-6 Astra (xhigh) | OpenAI | 55.5 | ±8.0 |
| 2 | Claude Sonnet 5.5 (max) | Anthropic | 54.5 | ±9.5 |
| 3 | GPT-6.1 Sol (xhigh) | OpenAI | 54.1 | ±7.8 |
| 4 | Claude Opus 5.5 (max) | Anthropic | 50.2 | ±9.2 |
| 5 | Claude Fable 5.1 (max) | Anthropic | 45.7 | ±5.4 |
| 6 | GPT-6.1 Terra (xhigh) | OpenAI | 44.3 | ±6.9 |
| 7 | Gemini 4 Argon (high) | 37.9 | ±5.3 | |
| 8 | Kimi K3.1 Pro (max) | Moonshot AI | 35.5 | ±4.2 |
| 9 | Grok 4.7 (xhigh) | SpaceXAI | 34.4 | ±4.5 |
| 10 | Qwen 3.8 Max (xhigh) | Alibaba | 30.8 | ±4.0 |
| 11 | DeepSeek V4.1 Flash (max) | DeepSeek | 27.9 | ±3.9 |
| 12 | GLM 5.3 (max) | Z.ai | 26.3 | ±4.5 |
| 13 | GPT-6 Luna (xhigh) | OpenAI | 25.1 | ±4.8 |
| 14 | Mistral Large 4 (high) | Mistral | 21.9 | ±4.6 |
| 15 | DeepSeek V4 Pro (max) | DeepSeek | 20.7 | ±4.2 |
Qualitative analysis
-
The gap is widening on the frontier
Anthropic and OpenAI models stand head and shoulders above the rest. No other providers crack the top 6. Gemini 4 is the only other model that reaches 40% success on high-finance tasks.
-
Even frontier models are weak at turning evidence into proper judgment
In most DFB tasks, agents are able to find the relevant information without recognizing its implications. They also have difficulty challenging a supplied assumption or evaluating better opportunities unless strictly told to, even in situations where a human analyst would instinctively do so. This means that when a CFO says one thing and an analyst says another, there is little reconciliation effort between balancing the importance of both perspectives.
-
Upstream mistakes cause cascading downstream failures
Incorrect earnings, source selection, or completion timing can affect debt capacity, fees, cash flows, valuations, and investor returns. Atomic output checks reveal those consequences, but the number of failed checks should not be interpreted as the number of independent reasoning failures.
-
Models often execute calculations correctly after selecting the wrong inputs
In a lending task, agents correctly calculated debt capacity using an incorrect earnings subtotal. In a pharmaceutical royalty task, they correctly applied payment caps and milestone rules to a forecast that omitted an entire sales channel. The agents knew how to perform the calculations but failed to verify the numbers feeding them.
-
Agents find decision-relevant evidence without acting on it
In one acquisition task, moving the closing date changed shareholder payouts and lowered the required offer by $1.25 million. All attempts by frontier models missed this saving, even though several had identified the buyer's right to choose the date.
The path toward AGI in dealmaking
For AGI to be useful in dealmaking, its work needs to hold up when someone checks it. Across high-finance assignments that take professionals 20-30 hours, agents must choose sound assumptions, use them consistently, and explain the decisions that matter.
DeepFinanceBench evaluates both the result and the work behind it. Auditability gives professionals a way to check the analysis and gives researchers a way to find the mistakes that need fixing. That brings us closer to measuring whether agents can take an assignment from a data room to a decision people can rely on.