Skip to content
‹ Back to IndustriesHealthcare

Johns Hopkins' Ritu Agarwal wants hospitals to measure a baseline before scaling AI

A Google Cloud survey found that only 30% of healthcare organizations have scaled advanced or agentic AI across their operations. Johns Hopkins professor Ritu Agarwal says health systems need a measured baseline, clinician-led workflow design and representative data before they scale. Small pilots hide the rest.

This story was produced through MarketScale. See how Healthcare teams put it to work with Executive Thought Leadership.

By MarketScale Newsroom · Ritu AgarwalJohns Hopkins Carey Business SchoolGoogle CloudHlth 2026
Share
Johns Hopkins' Ritu Agarwal wants hospitals to measure a baseline before scaling AI

Key takeaways

01

A 200-person pilot answers whether a tool works; how it behaves on another hospital's patients and what it costs at full volume mostly show up after the pilot ends.

02

Just 30% of healthcare organizations said they’ve scaled advanced or agentic AI across operations, according to a Google Cloud survey of 250+ healthcare executives.

03

The sharper pre-pilot question: which disease condition, which baseline metric and what projected cost at full volume will this tool be judged against?

Free workspace

Turn your Healthcare expertise into content.

Record interviews, organize footage, and write with AI on a free trial of the MarketScale platform for qualifying companies. No demo required, no credit card.

Try it Free

Healthcare spent roughly three decades moving from paper charts to a deeply digitized system, and it still hasn't worked out how to run AI across a whole organization. That was the plain verdict from Ritu Agarwal, the Wm. Polk Carey distinguished professor of information systems and health at the Johns Hopkins Carey Business School, in an interview with Healthcare Dive reporter Jill Hughes at the HLTH 2026 conference in Las Vegas, published Nov. 17. In her view the industry has not made the jump from pilot to true operational deployment, even though AI could help with quality, cost, access and inequities in care. There's a lot of hard work to do first.

She traces the gap to complexity at every level: healthcare is complex, human beings are complex and medical conditions are complex. The open questions about clinical workflow integration, governance, physician trust and return on investment, she said, will take time and research to answer.

A day earlier, a Google Cloud survey put a number on that gap. Just 30% of healthcare organizations said they had scaled advanced or agentic AI across their operations, according to the survey of more than 250 healthcare executives, which consumer insights firm Hypothesis Group conducted. For a health system CIO with a stack of promising pilots, Agarwal's answers point to an awkward reading: the pilot is the stage least able to reveal what decides scale. The rest of her argument explains why, and what to measure before the next one starts.

The Stanford model in a Kentucky hospital

Agarwal's first obstacle is data. Building a model means finding data robust enough to give some assurance it will perform under all conditions, and that last phrase is where things get hard. Her example: a model trained on data from Stanford Medical School rests on a very different patient population than a rural hospital in Kentucky would see. Moving a model from one context to another produces what she calls model drift.

Now set that against how Healthcare Dive defines scaling: standardizing tools across multiple hospitals, unifying governance and building AI workflows into the electronic health record. Standardizing across hospitals is the act of moving a model into new contexts. For a system that runs an academic medical center alongside community and rural sites, the Stanford-to-Kentucky problem could sit inside its own network. Agarwal's remedy is unglamorous: more development, more testing and more representative data sets before anyone feels comfortable using these models in everyday clinical practice.

Model drift, as Agarwal uses the term, is the change in a model's performance when it is built on one patient population and applied to another. A pilot run at a single site tests the model on that site's patients, so it can't show how the model will behave at the next hospital.

Where the tool sits in the clinical decision

The second obstacle is workflow, and Agarwal treats it as a redesign problem. Putting AI on top of a bad or inefficient process just adds AI to a bad process. Taking away clinicians' discretion is the opposite mistake. The hard design question is where in a clinical decision the tool steps in: when it should recommend, when it should simply supply information, when it should synthesize and when it should question the physician's call. She expects doctors to resist AI that is just bolted onto how they already work.

She also has a view on who should build it. Many vendors have a chief medical officer and a few clinicians on the development team, she observed, but in her opinion still let technologists make the decisions. She'd put clinicians in charge of developing AI applications. For a procurement team comparing tools, that gives a sharper question than any accuracy benchmark: who decided where in the clinical decision this tool steps in, and was it a clinician?

Johns Hopkins' own population health work shows what that redesign can look like. When Johns Hopkins Health System launched its Illustra platform in July 2025, Healthcare IT News reported that it built on the Johns Hopkins ACG System, which healthcare organizations have used for population health analytics and risk adjustment for more than three decades. Illustra president Sarah Kachur described the new layer as turning analytic outputs into an action system: worklists tailored to clinical teams' staffing and workflows, plus tracking of clinical and financial outcomes. The models had been around for decades. The product built on them was about getting their output into clinical teams' daily work, which suggests Agarwal's workflow problem is where much of the effort goes.

Hopkins has also tied clinical informatics to AI at the top of its own organization. In May 2026, according to Johns Hopkins Information Technology, Dr. Sara Murray was named enterprise chief medical informatics and AI officer, with responsibility for strategic direction on clinical IT and AI initiatives across the School of Medicine and Johns Hopkins Health System. Putting medical informatics and AI in one enterprise role could be one way a health system applies Agarwal's argument from the buyer's side: the person accountable for clinical IT also owns the AI agenda.

Baselines and token bills

On return on investment, Agarwal admits the evidence isn't in yet. Healthcare has done outcome studies on other technologies for years, but AI is still at an early stage, so outcome data doesn't exist. Her advice to a health system leader is to set up a baseline measurement system now and name the outcome expected from each AI tool. That could be efficiency, measured as time spent on certain tasks, or effectiveness, measured as patients served or actual health outcomes.

Illustra offers a working picture of that kind of measurement. Its Population Health Impact Analysis tool, as Healthcare IT News described it, lets users assess the clinical and financial effects of population health programs and measure results across specific populations and interventions. Its Care Management Targeting feature attaches documented reasons for inclusion to each patient on a list. That comes close to the discipline Agarwal describes: a defined population, a defined intervention and a result that can be compared with where things started.

She also tells systems not to try to do everything at once. Pick specific disease conditions and evaluate AI against them systematically. Top-line and bottom-line gains won't show up right away, she said, though emerging evidence and the experience of other industries suggest they will eventually. She called the upside potentially large. Whether it arrives in healthcare is still unproven.

How far healthcare is from scaled AI

30%
of healthcare organizations report scaling advanced or agentic AI across operations
13 points
how far healthcare trails the all-industry benchmark on that measure
Nearly 60%
of executives hit integration and deployment problems moving AI from pilot to a live system
40%
cite unpredictable costs as a reason for pausing AI initiatives

Google Cloud survey of 250+ healthcare executives, conducted by Hypothesis Group, as reported by Healthcare Dive

Cost is the dimension Agarwal's measurement plan needs a partner for, and Google Cloud's Aashima Gupta supplied it. Gupta, the company's global director of healthcare, told Healthcare Dive that AI's economics only come into view as a tool moves to scale, and stay hidden in a pilot of 100 or 200 people. She wants health systems to practice tokenomics, measuring and managing what AI models cost to run, and even suggested the industry may need a chief token officer with a strategy and budget.

Tokens are the units of data AI models use, and their costs vary. Healthcare Dive cited an Accenture report finding that companies using tokenomics strategies avoided a projected 44% increase in token costs over two years. That finding covers companies generally, not health systems in particular, so it shows what cost discipline can do in enterprise AI rather than setting a healthcare benchmark. Gupta's warning is blunt: even if AI solves problems, "it will not be able to scale because of the cost constraints."

Taken together, Agarwal's baseline and Gupta's cost discipline describe the same tool from two sides. A 200-person pilot answers whether a tool works; the questions that decide scale, how it behaves on another hospital's patients and what it costs at full volume, mostly arrive after the pilot ends. That timing matters most for systems with thin margins and, as Healthcare Dive noted, looming federal funding cuts, where a tool that looked affordable in the pilot can become a budget surprise once it runs across the network.

Before approving the next pilot, ask three things: which disease condition it targets, which baseline metric it will be judged against, and what its projected running cost is at full network volume.

Fewer vendors, more models

Agarwal expects a lot of consolidation among health AI vendors. The winners, she argued, will be the companies with the most health system relationships and the most data. Google and Microsoft lead on size and customer count. Electronic health record companies hold large curated data sets but are less nimble than the tech companies. Point-solution vendors, in her forecast, get absorbed or bought out as large platforms become a strong business model for health tech.

Size alone isn't the prize in her account. Google and Microsoft come out ahead, she said, if they can construct massive health data sets. That ties her market forecast back to her first obstacle: the representative data that would ease model drift is the same asset she expects to decide which vendors win.

Google's own pitch pushes back on part of that. Gupta argues healthcare is many workflows at different levels of sophistication, so a single model for everything is the wrong approach. "You don't need a supersonic plane to do the simple task of going from one town to another," she said. Google Cloud's Gemini for healthcare, built for providers and payers, lets users combine models from Google, OpenAI and Anthropic under one governance setup and integrates with third-party vendors such as AI scribes and electronic health record companies. Anthropic and OpenAI have released healthcare-specific tools of their own, and Google claims its open model approach sets it apart.

The survey points to buyers who want that kind of control. About 40% of respondents named protection of sensitive data as an especially important AI consideration, nine percentage points above the all-industry benchmark, and respondents said they want full visibility into a tool's governance and the ability to customize it. The two views can both hold: consolidation in the platform that governs the tools, with variety in the models underneath. More than 80% of respondents also named cost as a key obstacle to reducing reliance on their existing AI vendors.

For buyers, that 80% figure suggests switching costs belong in the pilot contract, not the renewal. Agarwal expects companies offering point solutions to be absorbed or bought out. For systems that have already signed point-solution contracts, that could raise a question about what happens to their data and baseline measurements after a sale. The signal that would confirm her reading is the outcome data she says doesn't exist yet: the first health systems to publish baseline-versus-result numbers for a specific condition will set the reference point everyone else gets measured against.

Featured companies

Your experts belong here

Every story in MarketScale Healthcare starts with a company putting its clinicians, service-line leaders, and field engineers on the record. Buyers are already reading this topic. The only question is whose experts they find.

Service-line buyers vet vendors quietly, and your clinicians become the proof they find while doing it.

Book DemoSee how it works15 minutes, straight to a calendar.

About the author

MarketScale Newsroom
MarketScale NewsroomEditorial Team, MarketScale

The MarketScale Newsroom reports on the companies, technologies, and trends shaping 16 B2B industries. It turns primary sources and expert commentary into clear, useful coverage for the people doing the work.

B2B Weekly

The week in Healthcare, and sixteen other industries, every Monday.

Ten stories, one-line takes, five minutes. Free.

Healthcare: are you visible to AI?

Before they reach out, Healthcare buyers ask AI engines which vendors to trust. Explore how your experts, customers, and partners can become useful content for buyers and AI search.

Free Trial

You just read one Healthcare expert. Your company is full of them.

This article was produced through MarketScale. The same platform turns your clinicians, service-line leaders, and field engineers into the articles, video, and social content Healthcare buyers are searching for. Start a free trial and see it with your own people. For qualifying companies, no credit card, no demo required.

NPS +73 · 1,000+ creators · 38+ countries

What your free trial includes

Hands-on access to the MarketScale platform
Media requests to your crowd, remote recording, AI writing tools
No demo required. No credit card.
For qualifying companies. Company confirmation required.

More Healthcare Insights

Why hospitals stagger security upgrades instead of shutting down

Why hospitals stagger security upgrades instead of shutting down

Parabit Systems' Robert Iraggi told A Weekly Bit that hospital security upgrades must be planned and staggered because the building never closes. OSHA counted 7.8 serious violent injuries per 10,000 healthcare workers in 2013 against under two in construction, and Joint Commission EP 17 requires an annual worksite analysis. Illinois biometric law adds a consent step for facial recognition.

  • 01OSHA counted 7.8 serious violent injuries per 10,000 healthcare workers in 2013; construction, manufacturing and retail each stayed under two.
  • 02Joint Commission EP 17 has required an annual workplace violence worksite analysis since January 2022, and hospitals are expected to act on what it finds—making phased security upgrades easier to document year to year.
  • 03Squire Patton Boggs counted at least 100 Illinois biometric privacy class actions filed in 2025, and Illinois’ BIPA requires notice and consent for collecting biometrics—so a facial-recognition rollout plan should include the notice, consent record, and retention controls before the first camera goes live.

Nov 17, 2026

Sterile processing deviations become the norm without stop-and-document rules

Sterile processing deviations become the norm without stop-and-document rules

Advanced Sterilization Products' Tamara Beeler told Censis Technologies' CtUC 2026 audience that sterile processing deviations harden into norms when departments don’t stop, escalate and document them. She connects ANSI/AAMI ST108:2023 water categories, daily cleaning verification, loaner-tray refusal when sets arrive late or lack IFUs, and root cause analysis when rework repeats on the same device.

  • 01ANSI/AAMI ST108:2023 turned water quality for instrument reprocessing into an enforceable standard; AAMI's earlier TIR34 was guidance with no enforcement behind it.
  • 02Some facilities require loaner trays 48 hours before a case, and ASP's Tamara Beeler says a technician at any level can refuse one that arrives late, undocumented or dirty.
  • 03Rework that returns on the same device is the point at which Beeler wants a root cause analysis opened, with the record kept in the instrument tracking system.

Nov 17, 2026

TheraPanacea aims to spread best-center radiotherapy quality

TheraPanacea aims to spread best-center radiotherapy quality

TheraPanacea CEO Nikos Paragios says the company's two objectives are to carry best-center radiotherapy quality to every patient and to pick each patient's treatment at the start from multimodal data. A 2026 Nature Communications trial of 500 patients found AI-assisted contouring lifted mean Dice overlap from 0.857 to 0.902, and Siemens Healthineers CE-marked its own AI contouring in May 2026.

  • 01TheraPanacea's two stated objectives: carry best-center radiotherapy quality to every patient on the globe, and identify each patient's best treatment at the start from imaging and clinical history.
  • 02A 2026 Nature Communications trial of 500 patients across five centers found AI-assisted contouring lifted mean Dice overlap from 0.857 to 0.902 and cut median contouring time from 55 to 10 minutes, with less variation between centers.
  • 03Siemens Healthineers' Varian business CE-marked AI contouring for Eclipse in May 2026, so Paragios locates TheraPanacea's edge in its team, hospital partnerships and impact focus rather than in the algorithm alone.

Nov 17, 2026

Explore More Healthcare Insights

Read more expert perspectives from across Healthcare.

Browse Healthcare Hub

About the Experts

MarketScale Newsroom
MarketScale Newsroom

Editorial Team

MarketScale

The MarketScale Newsroom reports on the companies, technologies, and trends shaping 16 B2B industries. It turns primary sources and expert commentary into clear, useful coverage for the people doing the work.

RA
Ritu Agarwal

Johns Hopkins Carey Business School

For B2B teams

Your experts could be publishing here

Stories like this one run on content MarketScale captures from real practitioners. See how your team's expertise becomes coverage in Healthcare and beyond.

Book a Demo

Or call us. No forms required. We pick up. 214-945-2512