AI + Healthcare
Beyond Language: Early Benchmark Results Show Med-BEAM Outperforming Leading LLMs in Predicting Clinical Events
In an initial evaluation, Med-BEAM was 4x more accurate, 100x faster, and 250x less expensive than leading large language models.
Large language models have demonstrated remarkable capabilities in healthcare. They can summarize medical records, answer clinical questions, and help make complex information more accessible. But knowing what the medical literature or guidelines say about a condition is not the same as understanding what actually happens when a real patient is treated by a real care team.
That requires more than understanding language. It requires modeling how healthcare unfolds over time: how patients' conditions evolve, what actions clinicians are likely to take in response to those changes, and how different interventions could change what happens next. In order to fully realize the promise of artificial intelligence toward predicting and improving patient outcomes, Knit Health created Med-BEAM, its world model for healthcare.
Med-BEAM is designed to model what happens in healthcare. Trained on billions of healthcare decisions across hundreds of millions of patient records, Med-BEAM learns from real clinical experience to predict future clinical events and how those events might change under different interventions.
In an initial benchmark comparing Med-BEAM with leading frontier large language models, Med-BEAM delivered substantially better performance across accuracy, speed, and cost. The results offer an early demonstration of what a model built specifically to represent the dynamics of healthcare can do and why this approach to AI may ultimately succeed in solving some of healthcare's most consequential problems.
Putting prediction to the test
To assess the capabilities of Med-BEAM in comparison to leading large language models, Knit designed a benchmark focused on predicting the clinical actions that would occur for hospitalized patients over a 24 hour period. The benchmark evaluated Med-BEAM alongside Anthropic’s Claude Opus 4.8, Claude Sonnet 5, and Claude Haiku 4.5.
Each model was given the same health records for 200 patients admitted to the hospital from the emergency department (none of them used to train Med-BEAM), covering each patient’s history up to the moment of admission, along with the full list of possible clinical actions, including medications, procedures, lab orders, imaging, nursing actions and referrals. The models were then asked to predict the clinical actions that would occur over the next 24 hours. Importantly, the test did not nominate a particular outcome in advance or tell models where to focus. Models were evaluated on their ability to predict what would actually happen, rather than on a narrow set of preselected events. This was designed to test a fundamental question: how well can different types of models anticipate the next steps in a patient's clinical care?
Knit evaluated performance in predictive accuracy, inference speed, and cost. Across all three dimensions, Med-BEAM delivered substantially stronger results than its large language model counterparts.
The results
More accurate
Accuracy was measured as average precision, a measure of how well each model put the care that actually happened at the top of the list of its predictions, scored from 0 to 1 across every possible action for all 200 patients. Med-BEAM ranked the actions that actually happened above the ones that didn't far more accurately than any of the language models. Specifically, Med-BEAM scored 0.565, against 0.228 for Claude Opus 4.8, 0.222 for Claude Sonnet 5, and 0.124 for Claude Haiku 4.5.
In sum, Med-BEAM's average precision was 2.48 to 4.56 times greater than that of the large language models tested. The result suggests that modeling the dynamics of healthcare can offer a meaningful predictive advantage over approaches built primarily to understand and generate language.
Faster
Speed was measured as latency per patient: the time taken to produce the full 24-hour prediction for one patient. For the language models, the measured response times from the benchmark runs were 408 seconds for Claude Opus 4.8, 307 seconds for Claude Sonnet 5, and 216 seconds for Claude Haiku 4.5. Med-BEAM’s figure of about two seconds per patient is an estimate of its processing time, which puts it 108 to 204 times faster than the three models.
In healthcare, the value of a prediction depends in part on whether it can be delivered when a decision needs to be made. Faster inference can make predictive intelligence more practical to deploy across complex, high-volume healthcare environments.
Less expensive
For the large language models, costs reflect actual benchmark usage at Anthropic's published rates. Med-BEAM’s cost was estimated based on inference time, the price of compute per hour, and then multiplied by three to match the provider margin. Under this assumption, Med-BEAM demonstrated a significant cost advantage, with estimated per-patient costs 37 to 257 times lower than those of the frontier models.
The economics matter. Predictive capabilities that are both accurate and inexpensive to run have the potential to be applied more broadly, across more patients and more operational decisions.
The starting point and where we’re going from here
These results represent an initial evaluation of Med-BEAM on a defined prediction task. In that context, it performed far more accurately, faster, and more cheaply than frontier language models. This is compelling evidence that we are on our way to providing what clinical AI has been missing: breadth across clinical prediction questions, with better accuracy and at a fraction of the cost.
Knit intends to expand this work in the coming months with more comprehensive benchmarking across additional prediction tasks, models, and clinical settings. That research will help establish where world models can deliver the greatest value and how their performance translates into practical applications.
The opportunity extends beyond predicting what happens to an individual patient. Healthcare systems constantly make decisions about how to allocate limited resources: where patients should receive care, when appointments should be scheduled, how patients should be routed, and when hospital capacity is needed. These decisions shape access, efficiency, and the experience of care.
Knit is building a foundation of clinical intelligence that health systems, technology companies, and AI agents can use to understand patients, anticipate how care will unfold, and evaluate different ways of delivering it. We want the model to learn across institutions while reflecting the capabilities and practice patterns of individual healthcare organizations. Ultimately, we believe this will allow us to rethink how healthcare itself is produced, organizing care around what works best for patients rather than the constraints of today’s systems.