The MOC Learning Knowledge Assessment: A progress report

Robert Roswell, MD, FACP
credit: ABIM

Since its launch in 1990 the American Board of Internal Medicine’s (ABIM) maintenance of certification (MOC) program has been criticized by many internists, who say its requirements are burdensome and unnecessary. The ABIM, for its part, says MOC reassures the public that doctors are staying up to date in their medical knowledge.

In 2022 the ABIM introduced a new way for doctors to maintain certification: the Longitudinal Knowledge Assessment (LKA). A recent article in JAMA discusses the reasons for developing the LKA and why the ABIM believes it is a useful alternative to its traditional long-form examination. Medical Economics spoke recently with Robert Roswell, MD, FACP, an ABIM director and the article’s lead author. The transcript has been edited for length and clarity.

Medical Economics: Why did you and your coauthors decide to publish this article now?

Robert Roswell, MD: Coming out of the COVID-19 pandemic, we noticed an increase in the number of questions and publications about the value of maintenance of certification. We also noticed increased questions at special society meetings.

The “Viewpoint” was written to share the robust analyses that support the benefit of MOC in terms of a host of patient outcomes, most importantly decreased mortality in the care provided by an ABIM Board certified physician. These associations with ABIM certification are important to share with ABIM diplomates, health systems, patients and the medical community. ABIM certified physicians deserve the credit for their work and the public should be aware of the value of an ABIM board certified physician.

ME: Any idea how many doctors are using LKA versus the long-form examination each year?

RR: We see that the majority, about 80%, are choosing the LKA at this point.

ME: Has that 80% figure been true since the time you launched the LKA, or has it grown?

RR: It’s been true since the time we launched. We’ll continue to follow that metric to see what happens with it.

ME: But this is the first time that figure’s been publicly disclosed?

RR: Correct.

ME: Given that LKA doesn’t require preparation, how well can it measure a doctor’s current knowledge?

RR: I’d say probably more accurately because this is the knowledge doctors are using in daily practice, instead of focusing and studying for a length of time and discharging that knowledge on one particular exam. They’re not in a testing center, they’re at home or at work, they can use books, the internet, any way that can assist them in answering the question as they normally do.

ME: Because they’d have all those resources available if they were treating an actual patient? Is that the idea?

RR: Correct.

ME: What would you say to persuade that 30% of diplomates who are either neutral or disagree with the statement that LKA is a fair assessment of clinical knowledge? (Editor’s note: Roswell’s article states: “Across all disciplines, 70% of diplomates agree, 16% are neutral, and 14% disagree with the statement that “the LKA is a fair assessment of clinical knowledge in this discipline.”)

RR: I would inquire as to their thoughts on why it’s not. In our discussions with diplomates we found people talking about relevance, in terms of maybe the questions or content of the exam doesn’t exactly match what they’re seeing in their practice. It’s very difficult to create an assessment that will get to everyone’s particular practice. But the point is to get a comprehensive assessment of the person’s knowledge.

What we’re doing to address that is to develop “practice profiles” as we mentioned in the article. So let’s say you’re an asthma or chronic bronchitis specialist, there might be a way for items on the exam to skew more to what you’re seeing in your practice every day. And our idea is that will help the 30% who think this isn’t a relevant assessment of my practice to bring it closer to what they’re seeing in their practice.

ME: You also write that there are more practice settings represented now among exam content generators than before the launch of LKA. What are some examples of those?

RR: With the long form [exam], or even the knowledge check-in, the number of questions you need to generate is much lower. So in launching the LKA we reached out to get more practicing community doctors to write items for the LKA because of the number of items that need to be tested and validated to assess physicians is much greater. The reason that’s important is these questions should come from physicians practicing in the community and seeing things and know what to assess their colleagues on. It gives relevance and validity to the process.

We want to recruit [question writers] from various practice types, from different parts of the country, different paths from international medical school graduates to people who practice in rural communities to those in private practice to ensure we’re getting diverse perspectives.

ME: A National Board of Physicians and Surgeons representative wrote an article on our website in which she says, “MOC is undermining the foundations of the medical system—its physicians” and, “The cost and burden associated with MOC remains a significant factor driving physicians out of medicine.” How would you respond to those statements?

RR: I’d respond this way: MOC has been around since 1990. On top of MOC other things have been added on in terms of billing practices, federal requirements and mandates, training requirements, and so on. So to say that the MOC is the main contributor to burnout and not the pandemic, the economy, coding and billing, all the requirements that have come in…. I don’t think is the way to go.

ME: Just to be clear, she didn’t say MOC was the main thing, she called it a significant contributing factor.

RR: I don’t agree with that. One thing we’ve done with the LKA is to address physicians’ concerns by saying we recognize it’s a lot of time to go to a testing center and to focus all those studies into one particular exam. This is something physicians have been asking to sort of relieve administrative burden, to make it something we could do on our phones or at home or not have to go to a testing site.

It’s also important to underscore that MOC is associated with patient outcomes. And that’s been done with hundreds of studies going back decades. So it might be an exaggeration to call MOC the main contributor to burnout when it’s been around for 30-plus years and it’s tied to a lot of the different benefits to patients and the ABIM is working to make it administratively less burdensome to diplomates.

ME: Another charge the NBPAS representative makes is financial, and I want to quote it verbatim: “Despite labelling MOC as voluntary, ABMS forces de facto compliance with its onerous and unproven continuing education product by successfully weaving MOC into the national credentialing landscape through its for-profit subsidiary, ABMS Solutions, LLC. Employers and government entities purchase and rely on data from ABMS Solutions for necessary verification of a physician’s credentials.” She’s saying MOC is basically a profit center for ABIM. Your response to that?

RR: We say in our Viewpoint article that the LKA costs $220 per year. Moreover, the reason we put the value of MOC in there is because if you don’t think it’s worth your while than one dollar or any amount is not going to be worth your while. To say all these things are moneymaking schemes I think is a distortion. I think it’s an exaggeration to say this is being done purely for profit when you see the tens of thousands of physicians and hundreds of thousands of patients that were studied and the associated benefit.

ME: Another complaint we often hear about MOC is from doctors saying, “If I’ve taken the time and effort to get board certified in the first place why can’t I be trusted to keep my knowledge up to date?”

RR: I would say a couple of things. One, keeping up with individualized knowledge is great, once you have an objective standard showing where your knowledge gaps are. The issue is when you, as a practicing physician, may not know exactly where your gaps are. What tends to happen is you go to familiar topics and fortify the knowledge you already have, but you don’t know what you don’t know.

And this is what the assessment does. It’s a broad, objective assessment to show you where your gaps may lie. And once you have that, then you can go into your individualized course and fill those gaps in. But if you do it on your own you may never know where your gaps may be.

Also, when we look at the data, at those physicians who maintain certification and those who don’t, there’s a difference. And we point out in the article that because of the knowledge gain between those who maintained certification and those who didn’t, we saw that $5 billion in cost savings. And that comes from diagnostic knowledge, how to take complex data and make the choice that’s best for the patient and is also efficient for the health care system.

So I think there are a lot of data showing why it’s important to stay current with medical knowledge. I think the COVID-19 pandemic was a great example of how things change so quickly in medicine.

I can tell you as a personal anecdote, having family members who’ve been treated by physicians who were board certified and not board certified, I see the difference in quality. So I think it’s really important to maintain your certification and show it to your colleagues and the public.

ME: Are you in clinical practice?

RR: Yes, in cardiology.

ME: Have you found MOC helpful? Has it identified any knowledge gaps for you?

RR: Yes, it always has when I’ve taken the traditional long form. I recently started taking the LKA. I work as a cardiac ICU physician and so the LKA for inpatient internal medicine is ideal for me to see where the gaps are for patients coming to my ICU, principally with a cardiac diagnosis, but they usually have other problems like diabetes, thyroid problems. And it’s a great way for me to see where my gaps are whether it’s thyroid management or glucose management.

So if we can build that value to other physicians and diplomates where they see how the LKA could help them out, I would think there wouldn’t be as many questions about MOC and people would see the actual benefit. So I think our work is aligning the needs of the physician with our programs at ABIM to make sure they’re relevant and doing the job they’re supposed to be doing, which is saving patients’ lives.

ME: Is ABIM working on further modifications to the MOC process?

RR: Absolutely. The practice profiles are things we’re working on now. Balancing the inpatient vs the outpatient LKA. And then looking at machine learning and AI, how it evolves based on those things. And this is where we dialogue with our diplomates and society partners so they’re part of this evolution of MOC and the next steps. Because it’s not a static program. We’re always looking to improve it with the help of our colleagues and our society partners.