Skip to content
‹ Back to IndustriesSoftware & Technology

GPT-6 Astra beat the next-best model by 14.4 points in Hebbia's banker deck tests

Hebbia's prerelease tests of OpenAI's GPT-6 Astra found a 14.4-point lead on banker presentation requirements and 99.5% relevant citations. A separate Hebbia and AfterQuery credit study put a post-trained NVIDIA Nemotron model within three points of Astra at about one-seventh the cost. For deal teams, model choice looks task by task.

This story was produced through MarketScale. See how Software & Technology teams put it to work with Executive Thought Leadership.

By MarketScale Newsroom · HebbiaGpt-6 AstraOpenaiAfterquery
Share
GPT-6 Astra beat the next-best model by 14.4 points in Hebbia's banker deck tests

Key takeaways

01

Hebbia's own numbers suggest a two-model shop: GPT-6 Astra led on citations and client-ready slides, while a post-trained open model came within three points of Astra on credit tasks at roughly one-seventh the run cost.

02

Score sourcing on two separate measures, coverage of required source types and relevance of each cited excerpt, because a model can pass one and fail the other.

03

In Hebbia and AfterQuery’s credit study, Nemotron’s pre-training failure was missing qualifiers or exceptions even after finding the right clause—making completeness a key test alongside retrieval.

Free workspace

Turn your Software & Technology expertise into content.

Record interviews, organize footage, and write with AI on a free trial of the MarketScale platform for qualifying companies. No demo required, no credit card.

Try it Free

Every number in a diligence deck has two readers to get past. One is the partner who checks it against the data room. The other is the client, who will notice if the slides look machine-made. Hebbia, whose Matrix platform links research answers to the documents behind them, says a prerelease build of OpenAI's GPT-6 Astra beat every alternative it tested on both counts.

The results come from tests Hebbia ran in August and September 2026, which OpenAI published as a customer story on November 20. The headline gap is on slides: Astra led the next-best model Hebbia tried by 14.4 percentage points on meeting presentation requirements. That's a strong showing on the part of the job that, by Hebbia's account, produces the overnighters.

Hebbia has also published a second set of numbers. It ran a credit study with AfterQuery, and that one shows a far cheaper model getting close to Astra. Together, the two point anyone buying AI research tools for a bank, a fund or a corporate development team toward picking models task by task.

Inside Hebbia's Matrix test

Hebbia tested two steps. First, the model picks the documents or Matrix cells that support an answer. Then it turns that analysis into native PowerPoint slides that follow the user's structure and guidelines. Hebbia illustrated the flow with a hypothetical case: a banker assessing a client's acquisition of an industrial-software company, with about 2,000 files in the data room. The banker digs into customer concentration, contract terms and churn, then needs a diligence summary and valuation deck in the firm's template.

The method is the part worth copying. Every model ran through the same financial research cases and was scored against the same grading criteria. Domain experts wrote those criteria or audited them against the source documents.

GPT-6 Astra in Hebbia's prerelease tests

13.6 points
Lead over the best alternative on citing the required types of sources
99.5%
Share of cited excerpts that were relevant to the question
97.9%
Share of applicable presentation requirements met
14.4 points
Lead over the best alternative on presentation requirements

OpenAI customer story on Hebbia, November 2026

The two sourcing numbers catch different failures. The 13.6-point lead measures coverage, meaning whether the model brought in the kinds of sources a question requires. The 99.5% figure measures precision, meaning whether each excerpt it cited was relevant. A model could cite every required source type and still pad its answers with irrelevant passages. It could also cite cleanly from too narrow a set of documents. Hebbia reports that Astra did well on both.

The slide results fit the same pattern. Kavita Anand, a member of Hebbia's technical product staff, said earlier models tended to be strong either on slide craft or on following the brief and guidelines, and rarely on both. Astra was the first model Hebbia had seen rank first on both. For a firm with strict brand templates, that combination could decide whether an AI draft gets edited or rebuilt from scratch.

Why the citation trail matters most

Anand's case for traceability is about money. Hebbia's users base decisions worth millions of dollars on what the AI returns, she said, so citations do two jobs: they let a reviewer trace a claim, and they give confidence that the model found the right answer. In a data room holding thousands of documents, she added, even a one-point gain adds up.

In Matrix, a reviewer can open a citation to see the underlying document with the passage highlighted, and slide footnotes lead back to the same material. Hebbia's example of a review is specific: check a customer-concentration figure against the source table and correct the reporting period before the deck goes to the investment committee. The human check stays in the process. What the model changes is how much of the reviewer's time goes to verifying and reformatting, and how much is left for judging the deal.

Anand expects a better first draft to turn all-night deck sessions into a few minutes of review. That is a projection, not a measured result. Trendoviz notes that the story names none of the alternative models and gives no full per-model scores, so the leads can't be rebuilt independently, and that the scores come from a prerelease build.

For a deal team piloting Matrix or a similar tool, those gaps point to the number worth tracking in-house: time from first draft to an approved deck, measured on the team's own deals. Benchmark leads belong in the vendor conversation. Review hours belong in the business case.

Where a cheaper model gets close

Hebbia's other study tested a different kind of work. AfterQuery and Hebbia post-trained NVIDIA's open Nemotron 3.5 Super VL model on expert-written credit tasks built around syndicated loan agreements and their amendments. On a held-out set of 100 tasks, the share fully solved rose from 47% to 74%. GPT-6 Astra solved 77% with no extra training. AfterQuery puts the cost of the full 100-task run at $24.72 for Astra and an estimated $3.63 for the post-trained Nemotron, roughly one-seventh.

Share of 100 held-out credit tasks fully solved (%)
AfterQuery, in partnership with Hebbia · © MarketScaleDownload chart

The method matters for reading the chart. Each model ran each task once, and a judge model graded every answer against an expert-written rubric. Between three and eight tasks per model were dropped because the judge recorded no score. Fable 5.1 was the most accurate model at 85%, with a run cost of $49.66, about 14 times the post-trained Nemotron's. GPT 6 Sol scored 63% at $5.35. Both Nemotron cost figures are estimates based on token counts and OpenRouter list pricing.

The failure the training fixed is instructive. Before post-training, Nemotron had the clause that decides the answer on screen in 99% of runs, yet its answers still left out a qualifier, an exception or half of a two-part question. After training, the number of grading criteria failed for missing content fell from 58 to 13, and section citations per answer rose from one to 10. The model already found the right clause. What it learned was to put the proviso in its answer, and in credit work the proviso is often the answer.

Scale explains why Hebbia cares. Its customers manage more than $30 trillion in assets, according to AfterQuery, and one loan can sit under a stack of amendments while a portfolio can mean thousands of documents. Adithya Ramanathan, Hebbia's head of applied research, said the results show credit expertise can be applied to tens of millions of documents at a practical cost. AfterQuery's suggested setup is a post-trained open model for high-volume, well-scoped work, with edge cases sent to a frontier model for extra review.

Hebbia's own numbers suggest a two-model shop: GPT-6 Astra led its banker tests on citations and slides, while a post-trained open model came within three points of Astra on credit tasks at roughly one-seventh the run cost.

There are limits to that reading. The deck tests and the credit tasks measure different work, so Astra's 97.9% and 77% can't be compared. The $3.63 also covers only running the tasks. The expert task writing, Hebbia's credit-workflow taxonomy and the post-training run on 32 H200 GPUs all sit outside it. A firm without former credit investors on staff to write and check tasks would be starting further back.

Running a model bake-off the Hebbia way

Both Hebbia studies share a structure a buyer can borrow: a fixed set of cases from real workflows, grading criteria written or checked by people who do the work, and every model run side by side. That's a test a research technology lead can run on internal deals before signing a contract, whichever vendor is quoting the win.

When a vendor cites a model lead, ask how many cases were tested, which alternatives lost, and whether the tested build matches the one you'll run in production.

Three steps follow from the published methods. Score source coverage and citation relevance separately, as Hebbia did. Record cost per task next to accuracy, the way the credit study's chart plots both. And grade completeness directly, including whether an answer kept the qualifier or exception, since that was where the base model in the credit study kept falling short.

The next thing to watch is agents. Anand described a future role for agents that monitor information in the background and flag signals to finance professionals hours before competitors see them. Hebbia presents this as a direction, not a shipped feature. An agent like that would run constantly across huge document volumes, which is exactly the job where Hebbia's credit study found cost per document mattering as much as the top score.

Featured companies

Your experts belong here

Every story in MarketScale Software & Technology starts with a company putting its solutions engineers, product teams, and customer engineers on the record. Buyers are already reading this topic. The only question is whose experts they find.

Buyers ask AI engines who to consider, and published expert answers are what those engines cite.

Book DemoSee how it works15 minutes, straight to a calendar.

About the author

MarketScale Newsroom
MarketScale NewsroomEditorial Team, MarketScale

The MarketScale Newsroom reports on the companies, technologies, and trends shaping 16 B2B industries. It turns primary sources and expert commentary into clear, useful coverage for the people doing the work.

B2B Weekly

The week in Software & Technology, and sixteen other industries, every Monday.

Ten stories, one-line takes, five minutes. Free.

Software & Technology: are you visible to AI?

Before they reach out, Software & Technology buyers ask AI engines which vendors to trust. Explore how your experts, customers, and partners can become useful content for buyers and AI search.

Free Trial

You just read one Software & Technology expert. Your company is full of them.

This article was produced through MarketScale. The same platform turns your solutions engineers, product teams, and customer engineers into the articles, video, and social content Software & Technology buyers are searching for. Start a free trial and see it with your own people. For qualifying companies, no credit card, no demo required.

NPS +73 · 1,000+ creators · 38+ countries

What your free trial includes

Hands-on access to the MarketScale platform
Media requests to your crowd, remote recording, AI writing tools
No demo required. No credit card.
For qualifying companies. Company confirmation required.

More Software & Technology Insights

Why Microsoft patches Windows in two days but makes third-party apps wait

Why Microsoft patches Windows in two days but makes third-party apps wait

Microsoft Digital, Microsoft's internal IT group, makes Windows patches land within two days but holds new third-party app versions for a short deferral. The pause gives vendors and security researchers time to spot a compromised release. Intune's automatic app updates automate that second clock, though the documented version lacks rollout rings.

  • 01Microsoft's IT patches Windows within two days but intentionally defers new third-party app versions. They believe a short delay might allow vendors and researchers time to identify compromised releases.
  • 02Microsoft Digital doesn't wait for a CVE to reach a severity threshold before updating third-party apps: disclosure can lag by months, so the current version is treated as the safest one.
  • 03Intune's generally available auto-update lacks rollout rings or rollback capabilities. The ring, deferral, and pause controls described by Microsoft Digital pertain to an expanded version highlighted in an upcoming private preview.

Nov 21, 2026

550+ FedRAMP cloud services must rethink how they rank flaws

550+ FedRAMP cloud services must rethink how they rank flaws

FedRAMP requires more than 550 certified cloud services to adopt new vulnerability rules by Dec. 7, prioritizing flaws by internet reachability and exploitability. Corrective action plans can extend remediation until March 7, 2027. Many vendors are shopping for new tools, but the shift can require broader engineering changes beyond a scanner swap.

  • 01A high-severity bug walled off from the internet can now wait its turn, while a reachable, exploitable one moves to the front of the patch queue at more than 550 FedRAMP cloud services.
  • 02Reachability depends on how each environment is built, so vendors whose current scanning tools can't rank flaws by exposure face an engineering decision before March 7, 2027, and a simple purchase may not settle it.

Nov 20, 2026

55% of enterprises plan to own part of their AI infrastructure, research finds

55% of enterprises plan to own part of their AI infrastructure, research finds

Commissioned research finds 55% of enterprises plan to own at least part of their AI infrastructure, a choice the article links to control concerns in regulated sectors. For organisations that own the hardware, power and cooling demands can shift onto the facility. Respondents’ top rollout hurdles were scaling AI IT clusters, cost, and sourcing power and cooling infrastructure.

  • 0155% of enterprises plan to own at least part of their AI infrastructure, motivated by control over data, privacy, and compliance in regulated sectors like healthcare, finance, and government.
  • 02Three hurdles block AI deployment at nearly equal rates: scaling clusters (27%), cost (25%), and power and cooling sourcing (22%), with no single obstacle dominating.
  • 03More than half of enterprises say fewer than one in five AI proofs of concept reach production, delayed by lack of in-house expertise (33%) and integration complexity (32%).

Nov 20, 2026

Explore More Software & Technology Insights

Read more expert perspectives from across Software & Technology.

Browse Software & Technology Hub

About the Expert

MarketScale Newsroom
MarketScale Newsroom

Editorial Team

MarketScale

The MarketScale Newsroom reports on the companies, technologies, and trends shaping 16 B2B industries. It turns primary sources and expert commentary into clear, useful coverage for the people doing the work.

For B2B teams

Your experts could be publishing here

Stories like this one run on content MarketScale captures from real practitioners. See how your team's expertise becomes coverage in Software & Technology and beyond.

Book a Demo

Or call us. No forms required. We pick up. 214-945-2512