Can computers help doctors reduce diagnostic errors?

If an advanced computer like IBM’s Watson can go on Jeopardy and win, can a computer start diagnosing patients better than a human doctor? The possibility could mean a reduction in a diagnostic error rate that the Institute of Medicine estimates may be as high as 15%.

The concept of how computers would fare against humans in diagnosing patients was the genesis of a recent research study from the American Medical Association that compared the accuracy of common online symptom checkers available on sites like WebMD to doctors. “Our thought was, if there is such a high level of inaccuracy, can computers do better?” says Ateev Mehrotra, MD, MPH, senior author of the study.

Further reading: Should your practice have a blog?

The research revealed two things: doctors easily outperformed the machines, getting the correct diagnosis 72% of the time compared to 34% for the online symptom checkers, and that the diagnostic error rate for doctors was high.

“The doctors outperformed the symptom checkers; I expected that. By what magnitude was the only question,” says Mehrotra, who is also an associate professor at Harvard Medical School. But he was also surprised by the misdiagnosis rate for doctors, who missed the best diagnosis 28% of the time.

The online symptom checkers did far worse, but are they the best measure of what computers are capable of?

True artificial intelligence

The next step in computer-assisted diagnostics is harnessing the power of artificial intelligence (AI). Online symptom checkers are based on traditional computer programs and don’t utilize AI, which can greatly boost performance. Artificial intelligence differs from traditional computer software, which use data and a program to create an output, while AI uses data and outputs to create the program.

To differentiate the two, consider the question, “How many cats are sold in Pittsburgh?”

Blog: Staying connected to patients beyond the office visit

A traditional program would query a database of cat sales to return an answer. An AI-powered program might approximate how many cats are sold in Pittsburgh based on the number of cat food packages sold, says Ervin Sejdic, Ph.D., assistant professor at the University of Pittsburgh, who has worked on AI in healthcare for the last eight years.

Whether AI is ready to compete with human doctors on something as complex as a diagnosis is up for debate.

“I’m 100% certain that AI can do better than a doctor,” says Akli Adjaoute, Ph.D., CEO of Brighterion, a technology company specializing in artificial intelligence and machine learning.

Next: Will AI outperform a physician in head-to-head performance?

“The information doctors are using about blood pressure, lab tests and symptoms are purely data. If you allow the system to learn from the best doctors, the machine will be able to do better than humans.”

But Sejdic doesn’t agree. He says that barring a major computing development, AI will never outperform a physician in head-to-head performance.

Hot topic: 7 tips to protect patient data from visual hacking

“When a patient claims certain symptoms, the doctor examines things like blood pressure and how you look and combines that into one decision,” Sejdic says. AI doesn’t have the ability to physically examine a patient. Maybe in the future, a comprehensive system will combine body temperature, blood pressure, facial recognition features and a host of other measurements to provide a more accurate diagnosis, but that system doesn’t exist today, he adds.

Adjaoute says a successful system could be built today. AI applications in banking identify specific behaviors of cardholders and AI in cybersecurity allows machines to identify people by how they use the keyboard and how they write, so writing a set of rules to build a diagnostic tool is possible, he says.

The obstacle to such a system is the number of data points required for the AI to learn properly, says Sejdic. “If you look at the results for one of the most sophisticated machine learning algorithms, it gets high accuracy for recognizing a cat in the middle of a picture, but it was trained on millions of cat pictures. I have one of the largest databases when it comes to understanding human swallowing, and I only have data for 275 people.”

This lack of data will hamstring any efforts for AI to approach the sophistication level of humans when it comes to diagnosing all the possible problems a patient might demonstrate, he says. AI focused on specific problems with fewer data points have a greater chance at success, so as the technology evolves, he expects it to show up in specialty areas first.

Exclusive: 2016 EHR Report

“Right now, we are capable of achieving accuracy on specific applications for specific problems, but we are still far away from an AI system that can recognize all symptoms,” says Sejdic.

The role of AI in future medicine

While diagnostic errors are a concern, physicians should not worry about being replaced by the AI-laden medical robots from Star Wars.

“AI is just another tool all these clinicians can use,” says Sejdic. “I don’t think we’ll be replacing anyone with a robot or AI system. Many clinicians see the strong potential AI can provide for them either in training or when they are not sure of their diagnosis, they can have a system to confirm they are on the right path.”

Adjaoute sees an even brighter future, where AI helps deliver personalized medicine, using data and machine learning to improve outcomes. “It will revolutionize the whole healthcare system, not just in diagnostics, but medicine. AI will allow us to treat everyone differently and the risk of errors will be low and the treatment will be better. It will take your data, look at the information, analyze what you have and send medication specific to you. It will h