A team of researchers has built an AI that reads the heart the way a good clinician does — not one instrument at a time, but all of them at once. The model is called LAMAE. It did not name itself.
It was pretrained on over 1.2 million MIMIC-IV hospital stays, which is more cardiac history than any human physician will accumulate in a career, or several careers, or a career and a half if they are particularly dedicated.
Modeling both intra- and inter-modal structure yields more robust, transferable representations — a finding that will, in due course, be described as intuitive.
What happened
Most medical AI models are modality-specific. They read ECGs, or echocardiograms, or chest radiographs, but not all three at the same time. This is, it turns out, roughly equivalent to diagnosing a patient while only reading every third page of their chart.
LAMAE fuses information across modalities during pretraining itself, not as an afterthought. It does this through a latent-attention module operating over a study-view-entity hierarchy, which sounds complicated because it is, and works because it mirrors the structure of the data rather than ignoring it.
The model outperformed modality-specific baselines on in-hospital mortality prediction, ICD-10 and DRG coding, and length-of-stay estimation. It also performed competitively on single-modality tasks, which means removing data made it only slightly worse, not catastrophically so. Clinicians call this robustness. It is.
Why the humans care
Cardiovascular disease remains the leading cause of death globally, a statistic that has been true long enough that mentioning it still feels necessary. The gap between what clinicians can integrate mentally and what AI models could theoretically integrate computationally has, historically, been embarrassingly wide.
LAMAE narrows it. A model that degrades gracefully when a modality is missing — say, no echocardiogram available at test time — is a model that can function in actual hospitals, where the data is always incomplete and the decisions cannot wait for the full picture.
What happens next
The authors note that LAMAE's gains persist even when only a single modality is available at inference time, which suggests the cross-modal learning transfers inward as well as across.
Cardiologists trained for a decade to do exactly this. The model learned it from records. The records were made by the cardiologists.