Skip to content
‹ Back to IndustriesHealthcare

Up to 0.864 in U.S. and Austrian ICUs, 0.698 in China

ELDER-ICU, an ICU mortality prediction model for patients 65+, reached up to 0.864 AUROC in U.S. and Austrian cohorts but 0.698 at a site in China. The results highlight that performance can vary across settings, so hospitals buying AI built elsewhere should validate it on local patients and ensure contracts allow recalibration or retraining when needed.

This story was produced through MarketScale. See how Healthcare teams put it to work with Executive Thought Leadership.

By MarketScale Newsroom · Telehealth Connect Summit 2026Clinical AiAi Model ValidationElder-icu
Share
Up to 0.864 in U.S. and Austrian ICUs, 0.698 in China

Key takeaways

01

A vendor's validation claim is only as good as the patients it was tested on: one ICU mortality model scored 0.804 to 0.864 AUROC in U.S. and Austrian hospitals and 0.698 in China.

02

Match the fix to the failure. Recalibration improved probability accuracy at every site, while only retraining on local data recovered the bigger ranking losses at the South Korean and Chinese sites (+0.066 and +0.076 AUROC).

03

Checking AI on local patients takes staff and contract terms: whether the license permits local recalibration or retraining, and who on the team can run it.

Free workspace

Turn your Healthcare expertise into content.

Record interviews, organize footage, and write with AI on a free trial of the MarketScale platform for qualifying companies. No demo required, no credit card.

Try it Free

A risk prediction model for older ICU patients scored as high as 0.864 AUROC in U.S. and Austrian hospitals. At a site in China, the same model scored 0.698. For anyone signing a contract for a prediction tool trained on another health system's patients, that gap is a purchasing problem as much as a research finding.

The study behind those numbers, which tested a model called ELDER-ICU, was cited in Telehealth.org's coverage of the Telehealth Connect Summit 2026, held in Tunis from Sept. 29 to Oct. 1. Health ministries, the World Health Organization, the International Telecommunication Union, the Global Digital Health Partnership (GDHP) and private technology companies met there to get digital health and AI out of pilot mode. The site's Nov. 9 report by Julia Ivanova drew on attendee Maya Sherne, Telehealth.org's general manager.

The idea most useful to a hospital technology buyer came from the Multilateral Perspectives on Digital Health panel. Speakers laid out AI adoption in three moves: adopt an existing model, adapt it to the national context, then advance it with local expertise and data. Their real point was capacity. Countries need people who can evaluate AI systems themselves instead of relying entirely on technology built somewhere else.

That sounds like a ministry-level aspiration, but it applies just as well to a single hospital. A vendor's validation claim is only as good as the patients it was tested on. For a chief medical information officer, the practical questions are how far a model's accuracy travels and what it takes to bring it back when it doesn't. The ELDER-ICU study puts numbers on both.

An ICU model tested in four countries

The study, published in npj Digital Medicine, tested ELDER-ICU, a machine learning tool that predicts in-hospital death for ICU patients aged 65 and older using clinical variables recorded in the first 24 hours after admission. Its developers originally built and validated it on data from the United States and Amsterdam. This round, researchers ran it at 12 ICU sites in the U.S., Austria, South Korea and China, testing both geographic differences and shifts over time, including the COVID-19 period.

In the U.S. and Austrian cohorts, the model held up, with AUROC between 0.804 and 0.864. At the South Korean site it slipped to 0.753. At the Chinese site it fell to 0.698.

ELDER-ICU accuracy (AUROC) by setting before any local updating
npj Digital Medicine (Nature), 2026 · © MarketScaleDownload chart

The patients behind those scores were very different. Mortality ranged from about 6% at the South Korean site to about 22% at the Chinese site, with every other site falling between 10% and 21%. The authors concluded that rigorous validation and proactive adaptation are essential before clinical use wherever populations and clinical practice vary.

The study compared countries. The same reasoning could apply wherever a hospital's patient mix or care practices differ from the data a model learned on.

AUROC measures how well a model ranks patients by risk: 0.5 is a coin flip, and 1.0 means it perfectly separates patients who died from those who survived. Calibration is a separate test of whether the predicted probabilities match real outcomes, which matters when clinicians act on a stated risk percentage.

Three fixes, each with a different price

The study gets most useful after the drop. Researchers tried three ways of updating the model, and each one asks more of the hospital than the last.

Recalibration is the lightest. It adjusts the probabilities the model outputs to match local outcomes without rebuilding anything. Two standard methods, isotonic regression and Platt scaling, improved calibration across sites. Incremental training goes further: it fine-tunes the model on newly collected local data while keeping its core architecture, and it improved performance at most centers. The authors note this method has seen little use in critical care so far.

Retraining is the heaviest option, and it produced the biggest gains where they were needed most. AUROC rose by 0.066 at the South Korean site and 0.076 at the Chinese site, the two places where the original model had slipped furthest.

Taken together, the results suggest a practical order of work. Measure performance on local patients first. Recalibrate when the model ranks patients well but its stated probabilities are off. Save retraining for sites where the ranking itself breaks down. Each step needs something different: outcome data the hospital can label, people (in-house or vendor) to run the update, and contract terms that allow the model to be changed at all.

The same study also shows where this matters less. Performance stayed strong in the U.S. cohorts, the country the model was first built on, and in Austria. A hospital whose patients resemble a vendor's development population may need only a local check and a calibration pass, a far smaller project than the retraining the Asian sites needed.

Why validation so often goes undone

The same panel offered a reason that work gets skipped: agreeing on technical standards doesn't mean anyone puts them into practice. Speakers described fragmented projects piling up when health ministries, telecom agencies and private companies build systems on their own. Their answer was national interoperability standards, digital public infrastructure and government oversight to tie the efforts together.

Standards still need people to apply them, and in the U.S. that capacity gap shows up in rural planning. A May 2026 National Consortium of Telehealth Resource Centers webinar on state Rural Health Transformation Fund plans noted that rural providers increasingly rely on technical and operational support that may not be fully reflected in current workforce strategies. That matters for AI. If no one on staff can run a local validation, a hospital could be left relying on whatever the vendor tested.

Questions for the next AI contract

Before signing, ask for performance results from sites whose patients look like yours, whether the license allows local recalibration or retraining, and who runs and pays for that work after go-live.

The summit's final day turned to execution, with workshops on building interoperable health systems and designing telehealth and AI projects that move from planning to implementation. Panelists also argued that tight resources can push health systems toward new ways of delivering care, and that countries can learn from each other instead of copying models built in wealthier systems. A teleradiology session with participants from Egypt, Tunisia and the WHO focused on that kind of exchange.

For hospitals whose patients differ most from a model's training data, the ELDER-ICU results make the stakes concrete: at the Chinese site, retraining improved AUROC by 0.076. That work is possible only if the license allows the model to be changed and someone, in-house or at the vendor, is assigned to run it after go-live.

Featured companies

Your experts belong here

Every story in MarketScale Healthcare starts with a company putting its clinicians, service-line leaders, and field engineers on the record. Buyers are already reading this topic. The only question is whose experts they find.

Service-line buyers vet vendors quietly, and your clinicians become the proof they find while doing it.

Book DemoSee how it works15 minutes, straight to a calendar.

About the author

MarketScale Newsroom
MarketScale NewsroomEditorial Team, MarketScale

The MarketScale Newsroom reports on the companies, technologies, and trends shaping 16 B2B industries. It turns primary sources and expert commentary into clear, useful coverage for the people doing the work.

B2B Weekly

The week in Healthcare, and sixteen other industries, every Monday.

Ten stories, one-line takes, five minutes. Free.

Healthcare: are you visible to AI?

Before they reach out, Healthcare buyers ask AI engines which vendors to trust. Explore how your experts, customers, and partners can become useful content for buyers and AI search.

Free Trial

You just read one Healthcare expert. Your company is full of them.

This article was produced through MarketScale. The same platform turns your clinicians, service-line leaders, and field engineers into the articles, video, and social content Healthcare buyers are searching for. Start a free trial and see it with your own people. For qualifying companies, no credit card, no demo required.

NPS +73 · 1,000+ creators · 38+ countries

What your free trial includes

Hands-on access to the MarketScale platform
Media requests to your crowd, remote recording, AI writing tools
No demo required. No credit card.
For qualifying companies. Company confirmation required.

More Healthcare Insights

Epic's seven bets beyond the EHR, sorted by when hospitals can actually use them

Epic's seven bets beyond the EHR, sorted by when hospitals can actually use them

Epic is expanding past its EHR into ERP, IT staffing, AI, drug pricing and payer technology. Delivery runs from tools live today to a full EpicOps rollout that may not arrive until 2029. The payer platform has the clearest proof point: St. Luke's and UnitedHealthcare automated status updates on 88% of the claims they exchanged.

  • 01Epic's payer platform is the expansion bet with a measured result: 88% of claim status updates automated between St. Luke's and UnitedHealthcare, plus a 25% drop in status calls.
  • 02EpicOps, Epic's ERP suite, may not be fully available until 2029, so a health system whose ERP commitments end sooner can test individual modules alongside its current system without a full switch.
  • 03Before buying any third-party app that has to work inside Epic, ask the vendor to demonstrate the exact write operation it needs in your own environment, because FHIR support alone does not guarantee write access.

Nov 16, 2026

Avanlee Care’s $3.5M seed tests whether Medicare Advantage plans will pay to reach caregivers

Avanlee Care’s $3.5M seed tests whether Medicare Advantage plans will pay to reach caregivers

Avanlee Care raised a $3.5 million seed round led by BEA Venture Fund, bringing total funding to $6 million. The startup helps Medicare Advantage plans find and support members' family caregivers. Its bet: plans will pay to reach whoever books the visits.

  • 01Medicare Advantage plans can see overdue visits and refills; Avanlee Care’s $3.5M seed backs a per-member-per-month model to reach the family caregiver who can act on that outreach.
  • 02Caregiver support among Medicare Advantage Special Needs Plans rose from 5% of plans in 2025 to 16% in 2026, even as meals and transportation benefits declined. That gives benefit designers a reference point.

Nov 15, 2026

Oura's $625 million Counsel Health deal puts a ring maker in the triage business

Oura's $625 million Counsel Health deal puts a ring maker in the triage business

Oura is buying Counsel Health for $625 million in cash, bringing an AI chat service with on-demand physicians in-house. Counsel will keep selling triage and virtual care to health plans, health systems and employers. Outcomes data is still to come.

  • 01For $625 million in cash, a smart ring company now owns a service pitched to health plans as a replacement for the nurse triage line.
  • 02Counsel's website offers a free conversation with Counsel AI and lets users add a doctor for $29, a public reference point benefits teams could use when comparing virtual care costs.
  • 03The evidence behind the combined model is member satisfaction (4.6 out of 5) and self-reported surveys; clinical outcomes and cost results are the numbers that would confirm it.

Nov 15, 2026

Explore More Healthcare Insights

Read more expert perspectives from across Healthcare.

Browse Healthcare Hub

About the Expert

MarketScale Newsroom
MarketScale Newsroom

Editorial Team

MarketScale

The MarketScale Newsroom reports on the companies, technologies, and trends shaping 16 B2B industries. It turns primary sources and expert commentary into clear, useful coverage for the people doing the work.

For B2B teams

Your experts could be publishing here

Stories like this one run on content MarketScale captures from real practitioners. See how your team's expertise becomes coverage in Healthcare and beyond.

Book a Demo

Or call us. No forms required. We pick up. 214-945-2512