Experts best bet for mammogram readings

By John Easton
Medical Center Public Affairs

The mammogram-reading skills of general radiologists vary enormously, according to a study directed by Robert Schmidt, Associate Professor in Radiology. Although the researchers who performed the study were impressed with the overall ability of most general radiologists, they were unnerved by the magnitude of the variance.

Seventy-five general radiologists detected, on average, 70 percent of the breast cancers visible on 100 mammograms, about what previous studies would predict. But the scores ranged from a high of 98 percent, with a lot of false positives (normal exams rated as suspicious), to a low of 8 percent.

"The scores were all over the map, rather than clustered around 70 percent," said Schmidt. "The average score was no surprise. Most previous studies suggest that 15 to 30 percent of cancers are not detected at the first opportunity. But the extreme range was an eye opener."

Moreover, there was only a weak correlation between the test results for each generalist and his or her self-assessed level of expertise. For example, some of those who classified themselves as having advanced skills scored lower than those who rated themselves as novices.

"Imaging technology has steadily improved and continues to get better," Schmidt said. "The human observer may now be the weakest link in the breast-imaging chain."

Most quality-of-care research is performed at academic centers and focuses on the talents of highly specialized experts who devote their careers to a few specific interests, such as mammography. The overwhelming majority of screening mammograms, however, are read by general radiologists, who may read as few as two mammograms a day, by law.

The researchers--from Chicago, New York University, University of New Mexico at Albuquerque and University of Illinois at Peoria--presented good-quality copy films from 100 mammography cases to 100 radiologists who were attending continuing medical education meetings, and to four mammography experts.

Each radiologist was given two-and-one-half hours to complete the exercise, conditions roughly comparable to a routine clinical situation, except for the far higher percentage of cancers. Ordinarily, there are only about five cancers per 1,000 screening mammograms. The 100 films contained 55 normal cases and 45 cases with at least one cancer (five cases had two cancers). The observers were asked to rate the cases as normal or abnormal and to indicate the location of any suspicious lesions. The cancers were typical lesions found by routine screening mammograms, not the most difficult cases. Only 75 percent of the radiologists finished the task. Those who gave answers for fewer than 90 of the 100 cases were not included in the final assessment, "but they clearly would not have improved the scores," noted Schmidt. Those who did not complete the test found only 42 percent of the cancers in the cases they completed.

The experts detected 16 percent more cancers than generalists. Experts, however, were correspondingly more likely to rate normal cases as abnormal.

"Dedicated mammographers seem to have a lower threshold for what they find suspicious than generalists," said Schmidt. "They found significantly more cancers but wanted to take a closer look at more women who did not have cancers."

Although the 70 percent detection rate may seem low, "reading mammograms is extremely difficult, even for experts," cautioned Schmidt. "It requires a lot of training and experience. Very few people who don't do it full-time develop the skills to do it at the highest levels."

That's because reading screening mammograms is very different from other tasks performed by radiologists. Instead of looking for the cause of symptoms in a sick person, screening studies look for signs of minute, pre-symptomatic disease in apparently healthy patients. It requires a different mind-set, said Schmidt, and different sets of behaviors.

"The good news," added Schmidt, "is that this is a problem we know how to solve. We can teach generalists to read these images consistently and with a level of accuracy level closer to that of the experts. But we need to develop methods to make this training more accessible to all radiologists."

On one case, 83 percent of the generalists missed a cancer found by 75 percent of the experts. "As you might expect, we now use that as a teaching case," Schmidt said.

Ongoing technology improvements will also help. Digital mammography units, which enable physicians to manipulate the images in new ways, are entering the market. The first such unit in Chicago is currently being installed in the University's breast unit in the Duchossois Center for Advanced Medicine. And researchers at the University have developed a computer that reads mammograms, providing a second opinion for the radiologist and reducing the error rate. This will probably become commercially available within a year.

Until these technical advances become more common, Schmidt recommends that women seek breast programs at high volume centers such as academic institutions or specialized breast diagnostic centers with dedicated mammographers, who are more likely to have advanced training and experience.