Zhang group’s novel method might mean faster, more accurate MRI readings
By Patrick L. Kennedy
When it comes to medical imaging, a chatbot doesn’t know its backside from its elbow. Or, at least, it can’t tell a spine from a knee. But if a Boston University research team is successful, a smarter AI system for evaluating brain scan images might someday help clinicians spot diseases sooner, saving lives while cutting costs.
What Xin Zhang and colleagues in the Laboratory for Microsystems Technology have done is figure out how to train a neural network model to recognize general patterns in magnetic resonance imaging (MRI) brain scans, in a way that enables the model to be repurposed for more detailed tasks that can be specified later. That means neuroradiologists could use the BU team’s AI tool as a kind of Swiss army knife for MRI analysis—spotting lesions one day, for example, and detecting early signs of dementia the next—even when their clinic only has a handful of verified examples of those pathologies in their database.
“In many cases, hospitals have access to imaging data, but not enough expert-labeled examples to train large models from scratch,” says Zhang, a distinguished professor of engineering in BU’s College of Engineering. “We wanted to build a system that can learn from what is available and still perform well when labeled data are limited.” The result might be faster and more accurate readings in cases when speed of diagnosis can mean life or death.
When hours count
The type of foundational model that Zhang’s team has deployed is based on a transformer, which is something you might be aware of: It’s the kind also used in ChatGPT. (GPT stands for generative pre-trained transformer.) But that doesn’t mean that ChatGPT—or any other commercially available AI platform—can reliably identify what’s in medical images.
“The response may sound correct, but it might actually be wrong,” says Mengyu Li, a mechanical engineering doctoral student in Zhang’s lab. For example, one chatbot that Li tried out confused a knee with a spine. And in a medical context, a confident-sounding incorrect answer could be deadly. “That’s why, to this day, there is still no FDA-approved agentic AI for analyzing MRIs,” Li says.
But if an AI model for medical imaging were to work as one would hope—providing rapid, reliable results—it could prove a boon for clinicians and patients, says team member Chad Farris, assistant professor of radiology at the BU Chobanian & Avedisian School of Medicine and a neuroradiologist at Boston Medical Center (BMC).
“Early detection could be clinically significant,” says Farris. “In some cases [today], we can’t make the diagnosis definitively until the patient passes away. . . . When you get an MRI, it might take a few days or a week for it to get read. But there could be an acute reading, like a stroke, that you’d want to know about right away.”
The technology out of the Zhang lab, Farris says, “offers hope for being able to make more accurate diagnoses—and potentially, more timely diagnoses.” With such a system in place, staff would immediately be alerted to a stroke or other acute finding, and the patient would be rushed to the emergency room. “Even an hour or two could make a big difference.” The team has published their promising findings in Frontiers in Artificial Intelligence.
Tinkering with a transformer
Transformer models, in particular, hold great potential for medical imaging. “The good thing about a transformer structure is that it not only gets you the local features of the images; it can also absorb and store a large amount of information inside large model parameters,” says Li, “and also make connections between different imaging features. So it can get the global representation of the MRI slices,” or cross-section images.
In other words, with the right training, a transformer-based AI might gain an accurate picture of the human brain as a whole, and therefore be able to recognize a deviation from that pattern, indicating a specific problem.
Two barriers stand in the way, though. First of all, at least using current standard methods, separate AI models would have to be trained from scratch for every single disease or condition. Indeed, some developers are working on AI models for hemorrhage detection and aneurysm detection, says Farris, “but the problem is it’s a specific algorithm for each individual task, and they’re developed independently.”
The second stumbling block is the vast number of annotated images required to train such a model. “It is neuroradiologists who have to do it by hand,” says Zhang. “That takes forever.” As in, studying and labeling thousands of MRI images.
After all, an AI model doesn’t know what it doesn’t know. It takes many, many humans to train a visual processing AI, by looking at and marking many, many pictures—indicating that this grouping of pixels represents a pear tree, while that grouping represents a pickup truck, and so on. That’s how the big AI bots were trained to recognize objects: Millions of data workers in developing countries spent years labeling millions of images. The fact is, an army of First World neuroradiologists willing and able to do the same with MRIs is not likely to materialize. (Hence, commercial AI mixing up spine and knee images in Li’s tests.)
“You can’t get 10,000 annotated brain MRIs,” says Zhang. “It’s just not practical.”
The Terrier difference
The BU team’s solution was to train a model using a masked auto-encoder. While “the Masked Auto-Encoder” might sound like a 1930s sci-fi villain, it is actually a way of pre-training a visual processing model by randomly blanking out (or “masking”) portions (or “patches”) of some images. Eventually, the model can use the remaining, visible patches to reconstruct entire images.
“If the data set is large enough and the model is trained for long enough, you can get a pretty good generalized relationship between the different image features globally and locally,” says Li. “This is important because this training process is done on unlabeled MRIs.”
The team fed the model with 31 million unlabeled, de-identified images that were predominately from large, publicly available data sets. Though that was an enormous data set that the researchers used to pre-train the model, their hope is to hand off the model to clinics, who can adapt it to their selected tasks by feeding it just a handful of labeled images.
“When you fine-tune the pre-trained model, you don’t need that many annotated MRIs,” says Li.
That’s a good thing, says Farris, because for some less common brain diseases or disorders, “We don’t actually have large-scale data sets to train from.”
An adaptable model
In tests, the team succeeded in adapting their pre-trained model to several important clinical tasks, including restoration—clearing up blurry images—and sequence detection, the simple-sounding but critical step of correctly identifying the type of MRI being evaluated, as there are different kinds (or “sequences”) of MRI, used to detect different diseases or other problems in the brain.
“Skull stripping” is another task the Zhang team’s tool could be fine-tuned to perform well. This means erasing the skull from the image, for an unobstructed view of the brain. Li says the BU researchers’ model performed better than those of rival research groups: “Other models have a very hard time catching the edge between the skull and the actual brain tissues; they often make the most mistakes there.”
Finally, one of the most important tasks in MRI analysis is segmentation—correctly identifying the regions of the brain. Contrary to the brain’s neat appearance in a medical textbook, in a brain scan image it is actually no easy task to determine where one region ends and the next begins.
But those boundaries matter. “Different disease processes involve different parts of the brain,” says Farris. Getting that segmentation right can help clinicians gauge the relative volume of matter in the different regions, helping them to spot shrinkage or atrophy, and ultimately distinguishing signs of stroke from dementia or vascular disease.
Here, again, the BU team’s model tested well, earning a Dice score (a metric used in evaluating machine learning performance) above 83 percent, even when trained (in the fine-tuning stage) with fewer than 10 annotated MRI scans.
For time in the tube, less is more
It all adds up to quicker scans and faster, more accurate MRI analysis, the researchers say.
“From the patient’s perspective, the only difference that they would really notice is that an exam that may have taken 20 minutes to a half-hour is now probably taking 12 to 15 minutes,” says Farris.
But that means more than just convenience. It’s easier to lie still for shorter periods. When patients start fidgeting, that’s when diagnosis becomes difficult. “[Patient] motion is a big problem with MRIs because it leads to degradation of the image quality,” Farris explains. “So if they’re able to stay still, we’ll be able to reach a more conclusive diagnosis, and that’s really facilitated by the faster scan.”
Eventually, models like the one developed by the Zhang lab might be applied to other kinds of medical imaging, for other organs and parts of the body.
“It’s a stepping stone,” says Farris, “for building programs able to identify diseases earlier and more accurately than we’re currently able to do with imaging alone.”
This Laboratory for Microsystems Technology research was supported by the Rajen Kilachand Fund for Integrated Life Science and Engineering and the Hariri Institute for Computing and Computational Science and Engineering.