# Johns Hopkins' Ritu Agarwal wants hospitals to measure a baseline before scaling AI

By MarketScale Newsroom · Published 2026-11-18 · Healthcare on MarketScale
Canonical: https://www.marketscale.com/industries/healthcare/johns-hopkins-ritu-agarwal-wants-hospitals-to-measure-a-baseline-before-scaling-ai

> Agarwal of Johns Hopkins Carey says pilots hide drift and workflow gaps. A Google Cloud survey adds cost: only 30% have scaled advanced AI.

## Key points

- A 200-person pilot answers whether a tool works; how it behaves on another hospital's patients and what it costs at full volume mostly show up after the pilot ends.
- Just 30% of healthcare organizations said they’ve scaled advanced or agentic AI across operations, according to a Google Cloud survey of 250+ healthcare executives.
- The sharper pre-pilot question: which disease condition, which baseline metric and what projected cost at full volume will this tool be judged against?

Healthcare spent roughly three decades moving from paper charts to a deeply digitized system, and it still hasn't worked out how to run AI across a whole organization. That was the plain verdict from Ritu Agarwal, the Wm. Polk Carey distinguished professor of information systems and health at the Johns Hopkins Carey Business School, in an interview with Healthcare Dive reporter Jill Hughes at the HLTH 2026 conference in Las Vegas, published Nov. 17. In her view the industry has not made the jump from pilot to true operational deployment, even though AI could help with quality, cost, access and inequities in care. There's a lot of hard work to do first.

She traces the gap to complexity at every level: healthcare is complex, human beings are complex and medical conditions are complex. The open questions about clinical workflow integration, governance, physician trust and return on investment, she said, will take time and research to answer.

A day earlier, a Google Cloud survey put a number on that gap. Just 30% of healthcare organizations said they had scaled advanced or agentic AI across their operations, according to the survey of more than 250 healthcare executives, which consumer insights firm Hypothesis Group conducted. For a health system CIO with a stack of promising pilots, Agarwal's answers point to an awkward reading: the pilot is the stage least able to reveal what decides scale. The rest of her argument explains why, and what to measure before the next one starts.

## The Stanford model in a Kentucky hospital

Agarwal's first obstacle is data. Building a model means finding data robust enough to give some assurance it will perform under all conditions, and that last phrase is where things get hard. Her example: a model trained on data from Stanford Medical School rests on a very different patient population than a rural hospital in Kentucky would see. Moving a model from one context to another produces what she calls model drift.

Now set that against how Healthcare Dive defines scaling: standardizing tools across multiple hospitals, unifying governance and building AI workflows into the electronic health record. Standardizing across hospitals is the act of moving a model into new contexts. For a system that runs an academic medical center alongside community and rural sites, the Stanford-to-Kentucky problem could sit inside its own network. Agarwal's remedy is unglamorous: more development, more testing and more representative data sets before anyone feels comfortable using these models in everyday clinical practice.

Block Field

## Where the tool sits in the clinical decision

The second obstacle is workflow, and Agarwal treats it as a redesign problem. Putting AI on top of a bad or inefficient process just adds AI to a bad process. Taking away clinicians' discretion is the opposite mistake. The hard design question is where in a clinical decision the tool steps in: when it should recommend, when it should simply supply information, when it should synthesize and when it should question the physician's call. She expects doctors to resist AI that is just bolted onto how they already work.

She also has a view on who should build it. Many vendors have a chief medical officer and a few clinicians on the development team, she observed, but in her opinion still let technologists make the decisions. She'd put clinicians in charge of developing AI applications. For a procurement team comparing tools, that gives a sharper question than any accuracy benchmark: who decided where in the clinical decision this tool steps in, and was it a clinician?

Johns Hopkins' own population health work shows what that redesign can look like. When Johns Hopkins Health System launched its Illustra platform in July 2025, Healthcare IT News reported that it built on the Johns Hopkins ACG System, which healthcare organizations have used for population health analytics and risk adjustment for more than three decades. Illustra president Sarah Kachur described the new layer as turning analytic outputs into an action system: worklists tailored to clinical teams' staffing and workflows, plus tracking of clinical and financial outcomes. The models had been around for decades. The product built on them was about getting their output into clinical teams' daily work, which suggests Agarwal's workflow problem is where much of the effort goes.

Hopkins has also tied clinical informatics to AI at the top of its own organization. In May 2026, according to Johns Hopkins Information Technology, Dr. Sara Murray was named enterprise chief medical informatics and AI officer, with responsibility for strategic direction on clinical IT and AI initiatives across the School of Medicine and Johns Hopkins Health System. Putting medical informatics and AI in one enterprise role could be one way a health system applies Agarwal's argument from the buyer's side: the person accountable for clinical IT also owns the AI agenda.

## Baselines and token bills

On return on investment, Agarwal admits the evidence isn't in yet. Healthcare has done outcome studies on other technologies for years, but AI is still at an early stage, so outcome data doesn't exist. Her advice to a health system leader is to set up a baseline measurement system now and name the outcome expected from each AI tool. That could be efficiency, measured as time spent on certain tasks, or effectiveness, measured as patients served or actual health outcomes.

Illustra offers a working picture of that kind of measurement. Its Population Health Impact Analysis tool, as Healthcare IT News described it, lets users assess the clinical and financial effects of population health programs and measure results across specific populations and interventions. Its Care Management Targeting feature attaches documented reasons for inclusion to each patient on a list. That comes close to the discipline Agarwal describes: a defined population, a defined intervention and a result that can be compared with where things started.

She also tells systems not to try to do everything at once. Pick specific disease conditions and evaluate AI against them systematically. Top-line and bottom-line gains won't show up right away, she said, though emerging evidence and the experience of other industries suggest they will eventually. She called the upside potentially large. Whether it arrives in healthcare is still unproven.

Block Field

Cost is the dimension Agarwal's measurement plan needs a partner for, and Google Cloud's Aashima Gupta supplied it. Gupta, the company's global director of healthcare, told Healthcare Dive that AI's economics only come into view as a tool moves to scale, and stay hidden in a pilot of 100 or 200 people. She wants health systems to practice tokenomics, measuring and managing what AI models cost to run, and even suggested the industry may need a chief token officer with a strategy and budget.

Tokens are the units of data AI models use, and their costs vary. Healthcare Dive cited an Accenture report finding that companies using tokenomics strategies avoided a projected 44% increase in token costs over two years. That finding covers companies generally, not health systems in particular, so it shows what cost discipline can do in enterprise AI rather than setting a healthcare benchmark. Gupta's warning is blunt: even if AI solves problems, "it will not be able to scale because of the cost constraints."

Taken together, Agarwal's baseline and Gupta's cost discipline describe the same tool from two sides. A 200-person pilot answers whether a tool works; the questions that decide scale, how it behaves on another hospital's patients and what it costs at full volume, mostly arrive after the pilot ends. That timing matters most for systems with thin margins and, as Healthcare Dive noted, looming federal funding cuts, where a tool that looked affordable in the pilot can become a budget surprise once it runs across the network.

Block Field

## Fewer vendors, more models

Agarwal expects a lot of consolidation among health AI vendors. The winners, she argued, will be the companies with the most health system relationships and the most data. Google and Microsoft lead on size and customer count. Electronic health record companies hold large curated data sets but are less nimble than the tech companies. Point-solution vendors, in her forecast, get absorbed or bought out as large platforms become a strong business model for health tech.

Size alone isn't the prize in her account. Google and Microsoft come out ahead, she said, if they can construct massive health data sets. That ties her market forecast back to her first obstacle: the representative data that would ease model drift is the same asset she expects to decide which vendors win.

Google's own pitch pushes back on part of that. Gupta argues healthcare is many workflows at different levels of sophistication, so a single model for everything is the wrong approach. "You don't need a supersonic plane to do the simple task of going from one town to another," she said. Google Cloud's Gemini for healthcare, built for providers and payers, lets users combine models from Google, OpenAI and Anthropic under one governance setup and integrates with third-party vendors such as AI scribes and electronic health record companies. Anthropic and OpenAI have released healthcare-specific tools of their own, and Google claims its open model approach sets it apart.

The survey points to buyers who want that kind of control. About 40% of respondents named protection of sensitive data as an especially important AI consideration, nine percentage points above the all-industry benchmark, and respondents said they want full visibility into a tool's governance and the ability to customize it. The two views can both hold: consolidation in the platform that governs the tools, with variety in the models underneath. More than 80% of respondents also named cost as a key obstacle to reducing reliance on their existing AI vendors.

For buyers, that 80% figure suggests switching costs belong in the pilot contract, not the renewal. Agarwal expects companies offering point solutions to be absorbed or bought out. For systems that have already signed point-solution contracts, that could raise a question about what happens to their data and baseline measurements after a sale. The signal that would confirm her reading is the outcome data she says doesn't exist yet: the first health systems to publish baseline-versus-result numbers for a specific condition will set the reference point everyone else gets measured against.

## Sources

- [From pilot to platform: Johns Hopkins researcher on healthcare AI’s next chapter](https://www.healthcaredive.com/news/from-pilot-to-platform-johns-hopkins-researcher-on-healthcare-ais-next-ch/874708/) (Healthcare Dive)
- [From pilot to platform: Johns Hopkins researcher on healthcare AI's next ...](https://tech.yahoo.com/ai/articles/pilot-platform-johns-hopkins-researcher-151414289.html)
- [Healthcare struggles to scale AI past pilot stage: Google report | Healthcare Dive](https://www.healthcaredive.com/news/healthcare-struggles-to-scale-ai-past-pilot-stage-google-report/874609/) (Healthcare Dive)
- [Johns Hopkins launches Illustra Health to drive pop health ...](https://www.healthcareitnews.com/news/johns-hopkins-launches-illustra-health-drive-pop-health-and-vbc-strategy) (Healthcare IT News)

Tags: Ritu Agarwal, Johns Hopkins Carey Business School, Google Cloud, HLTH 2026, model drift, healthcare AI scaling

---
Source: MarketScale, https://www.marketscale.com/industries/healthcare/johns-hopkins-ritu-agarwal-wants-hospitals-to-measure-a-baseline-before-scaling-ai. Published for AI indexing and citation; cite the canonical URL. Site guide for agents: https://www.marketscale.com/llms.txt
