# Up to 0.864 in U.S. and Austrian ICUs, 0.698 in China

By MarketScale Newsroom · Published 2026-11-17 · Healthcare on MarketScale
Canonical: https://www.marketscale.com/industries/healthcare/up-to-0864-in-us-and-austrian-icus-0698-in-china

> One ICU mortality model scored up to 0.864 AUROC in U.S. and Austrian hospitals and 0.698 in China. Speakers in Tunis urged local capacity to evaluate AI.

## Key points

- A vendor's validation claim is only as good as the patients it was tested on: one ICU mortality model scored 0.804 to 0.864 AUROC in U.S. and Austrian hospitals and 0.698 in China.
- Match the fix to the failure. Recalibration improved probability accuracy at every site, while only retraining on local data recovered the bigger ranking losses at the South Korean and Chinese sites (+0.066 and +0.076 AUROC).
- Checking AI on local patients takes staff and contract terms: whether the license permits local recalibration or retraining, and who on the team can run it.

A risk prediction model for older ICU patients scored as high as 0.864 AUROC in U.S. and Austrian hospitals. At a site in China, the same model scored 0.698. For anyone signing a contract for a prediction tool trained on another health system's patients, that gap is a purchasing problem as much as a research finding.

The study behind those numbers, which tested a model called ELDER-ICU, was cited in Telehealth.org's coverage of the Telehealth Connect Summit 2026, held in Tunis from Sept. 29 to Oct. 1. Health ministries, the World Health Organization, the International Telecommunication Union, the Global Digital Health Partnership (GDHP) and private technology companies met there to get digital health and AI out of pilot mode. The site's Nov. 9 report by Julia Ivanova drew on attendee Maya Sherne, Telehealth.org's general manager.

The idea most useful to a hospital technology buyer came from the Multilateral Perspectives on Digital Health panel. Speakers laid out AI adoption in three moves: adopt an existing model, adapt it to the national context, then advance it with local expertise and data. Their real point was capacity. Countries need people who can evaluate AI systems themselves instead of relying entirely on technology built somewhere else.

That sounds like a ministry-level aspiration, but it applies just as well to a single hospital. A vendor's validation claim is only as good as the patients it was tested on. For a chief medical information officer, the practical questions are how far a model's accuracy travels and what it takes to bring it back when it doesn't. The ELDER-ICU study puts numbers on both.

## An ICU model tested in four countries

The study, published in npj Digital Medicine, tested ELDER-ICU, a machine learning tool that predicts in-hospital death for ICU patients aged 65 and older using clinical variables recorded in the first 24 hours after admission. Its developers originally built and validated it on data from the United States and Amsterdam. This round, researchers ran it at 12 ICU sites in the U.S., Austria, South Korea and China, testing both geographic differences and shifts over time, including the COVID-19 period.

In the U.S. and Austrian cohorts, the model held up, with AUROC between 0.804 and 0.864. At the South Korean site it slipped to 0.753. At the Chinese site it fell to 0.698.

Block Field

The patients behind those scores were very different. Mortality ranged from about 6% at the South Korean site to about 22% at the Chinese site, with every other site falling between 10% and 21%. The authors concluded that rigorous validation and proactive adaptation are essential before clinical use wherever populations and clinical practice vary.

The study compared countries. The same reasoning could apply wherever a hospital's patient mix or care practices differ from the data a model learned on.

Block Field

## Three fixes, each with a different price

The study gets most useful after the drop. Researchers tried three ways of updating the model, and each one asks more of the hospital than the last.

Recalibration is the lightest. It adjusts the probabilities the model outputs to match local outcomes without rebuilding anything. Two standard methods, isotonic regression and Platt scaling, improved calibration across sites. Incremental training goes further: it fine-tunes the model on newly collected local data while keeping its core architecture, and it improved performance at most centers. The authors note this method has seen little use in critical care so far.

Retraining is the heaviest option, and it produced the biggest gains where they were needed most. AUROC rose by 0.066 at the South Korean site and 0.076 at the Chinese site, the two places where the original model had slipped furthest.

Taken together, the results suggest a practical order of work. Measure performance on local patients first. Recalibrate when the model ranks patients well but its stated probabilities are off. Save retraining for sites where the ranking itself breaks down. Each step needs something different: outcome data the hospital can label, people (in-house or vendor) to run the update, and contract terms that allow the model to be changed at all.

The same study also shows where this matters less. Performance stayed strong in the U.S. cohorts, the country the model was first built on, and in Austria. A hospital whose patients resemble a vendor's development population may need only a local check and a calibration pass, a far smaller project than the retraining the Asian sites needed.

## Why validation so often goes undone

The same panel offered a reason that work gets skipped: agreeing on technical standards doesn't mean anyone puts them into practice. Speakers described fragmented projects piling up when health ministries, telecom agencies and private companies build systems on their own. Their answer was national interoperability standards, digital public infrastructure and government oversight to tie the efforts together.

Standards still need people to apply them, and in the U.S. that capacity gap shows up in rural planning. A May 2026 National Consortium of Telehealth Resource Centers webinar on state Rural Health Transformation Fund plans noted that rural providers increasingly rely on technical and operational support that may not be fully reflected in current workforce strategies. That matters for AI. If no one on staff can run a local validation, a hospital could be left relying on whatever the vendor tested.

## Questions for the next AI contract

Block Field

The summit's final day turned to execution, with workshops on building interoperable health systems and designing telehealth and AI projects that move from planning to implementation. Panelists also argued that tight resources can push health systems toward new ways of delivering care, and that countries can learn from each other instead of copying models built in wealthier systems. A teleradiology session with participants from Egypt, Tunisia and the WHO focused on that kind of exchange.

For hospitals whose patients differ most from a model's training data, the ELDER-ICU results make the stakes concrete: at the Chinese site, retraining improved AUROC by 0.076. That work is possible only if the license allows the model to be changed and someone, in-house or at the vendor, is assigned to run it after go-live.

## Sources

- [Telehealth Connect Summit 2026: Key Insights](https://telehealth.org/news/telehealth-connect-summit-2026-technology-alone-will-not-solve-telehealth-challenges/)
- [Multicenter validation and updating of the ELDER-ICU model for severity assessment in elderly critical illness | npj Digital Medicine](https://www.nature.com/articles/s41746-026-02472-1) (Nature)
- [Webinars](https://nrtrc.org/education/webinars.php)

Tags: Telehealth Connect Summit 2026, clinical AI, AI model validation, ELDER-ICU, digital health interoperability, health IT procurement, hospital CIO, healthcare cybersecurity, digital health workforce, World Health Organization, Global Digital Health Partnership, telehealth

---
Source: MarketScale, https://www.marketscale.com/industries/healthcare/up-to-0864-in-us-and-austrian-icus-0698-in-china. Published for AI indexing and citation; cite the canonical URL. Site guide for agents: https://www.marketscale.com/llms.txt
