- Physicians' Clinical Skills Decline Over Time
- MOC Improves the Clinical Performance and Outcomes of Participating Physicians
- MOC Learning Methods are Grounded in Adult Learning Research
- A Growing Body of Evidence Correlates Physician Communication Skills with Patient Outcomes
- The Public Expects Physicians to Engage in Continuous Learning and Assessment
- National Organizations and Governmental Bodies Expect Physicians to Engage in Continuous Learning and Assessment
- Physician Specialists Support MOC
- Data on Physician Participation in MOC
- Gaps in Our Knowledge About MOC
The American Board of Medical Specialties has created an Evidence Library that is a continuously updated collection of research and data from the literature supporting the value of MOC. For two recent reviews of evidence supporting the validity of MOC components see:
For thoughtful editorials about the goals, implementation and future of MOC, see
For a recent, more critical editorial about MOC, see "Ensuring Physicians' Competence - Is Maintenance of Certification the Answer?" by Iglehart JK et al 2012.
For a short, critical editorial by the ABD Executive Director about the limitations of the research presented in this section on the Value of MOC, see 9. Gaps in Our Knowledge About MOC as well as the recent JAAD CME article by Stratman E et al 2013.
WHAT THE RESEARCH SHOWS – OVERVIEWS
- PHYSICIANS' CLINICAL SKILLS DECLINE OVER TIME
Research suggests that, on average, clinical skills tend to decline over time. Incorporation of new medical knowledge about a specific condition, that should change an approach to patient care, is the most problematic. (Holmboe ES, Lipner R et al 2008)
- MOC IMPROVES THE CLINICAL PERFORMANCE AND OUTCOMES OF PARTICIPATING PHYSICIANS
The ABMS Member Boards’ MOC requirements are anchored in evidence-based guidelines, national clinical and quality standards and specialty best practices. Because the MOC program is relatively new (as it has been introduced gradually during the past decade), we don’t yet have evidence that results from decades of gathering data, but the data are emerging. Early studies show a link between MOC and improved clinical performance and outcomes by participating physicians. Physician engagement in MOC activities has been associated with enhancement in clinical competence, improvement in care processes and the gathering of valuable patient feedback. (ABMS MOC: Myths & Facts)
The evidence, which shows physicians who keep current do provide better quality care and have improved outcomes as measured by Board Certification and engagement in MOC activities, continues to grow. (ABMS MOC: Myths & Facts)
- MOC LEARNING METHODS ARE GROUNDED IN ADULT LEARNING RESEARCH
Many of the learning methods used in MOC programs have a firm grounding in research and a demonstrated ability to address physician competencies in practice-based learning and improvement. The latest principles in adult learning are incorporated into MOC activities, such as self-directed practice improvement modules (PIMs) and interactive workshops. Most member boards use PIMs or incorporate similar approaches in their performance improvement activities. (ABMS MOC: Myths & Facts)
A fundamental difference between traditional recertification and the ABMS MOC program was described in a 2004 article published in The Journal of Continuing Education in The Health Professions. The article emphasized the difference between competency (“know how”), which is the core requirement of recertification, and performance (“show how”), the additional MOC program component, which measures what physicians do in professional practice. The article described the MOC program as a “dramatic shift” in how graduate medical education, specialty certification and recertification are being conducted. (ABMS Fact Sheet)
- A GROWING BODY OF EVIDENCE CORRELATES PHYSICIAN COMMUNICATION SKILLS WITH PATIENT OUTCOMES
These studies form a basis for the importance of patient and peer communication assessments. Please go to What the Research Shows – Details: A Growing Body of Evidence Correlates Physician Communication Skills with Patient Outcomes
- THE PUBLIC EXPECTS PHYSICIANS TO ENGAGE IN CONTINUOUS LEARNING AND ASSESSMENT
A consumer study conducted by ABMS in 2010 demonstrated that 95% of patients believe that it is important for their physicians to maintain their certification. Patients not only expect their doctors to engage in continuous learning and assessment, but many assume that there is an external entity monitoring their participation. In addition, 84 percent of respondents would take some form of action if they found out their physician does not maintain certification, including looking for a new physician or ceasing to refer family and friends to that physician. Patients and family members routinely check their physicians’ certification status at www.CertificationMatters.org; more than one million searches were conducted in 2011 alone. (ABMS MOC: Myths & Facts)
- NATIONAL ORGANIZATIONS AND GOVERNMENTAL BODIES EXPECT PHYSICIANS TO ENGAGE IN CONTINUOUS LEARNING AND ASSESSMENT
The ABMS Board Certification process is becoming an increasingly important indicator for patients, hospitals, insurers and others within the health care industry for assessing a physician’s competence in a given medical specialty. (ABMS MOC: Myths & Facts)
Many national organizations have developed important relationships and collaborations with ABMS to facilitate the growth and acceptance of Board Certification and ABMS MOC programs as a leading indicator of quality and efficacy for health care delivered in this country. (ABMS MOC: Myths & Facts)
- PHYSICIAN SPECIALISTS SUPPORT MOC
Studies have shown that physician specialists believe in the value of MOC. Specifically, specialists believe that those providing patient care should maintain certification. Many say that a main reason for participation in MOC is to update their knowledge. Physicians report that their experience with components of MOC, such as PIMs and examinations, has been beneficial. Among the cited benefits are identifying areas for improvement in practice, providing valuable patient feedback and generating high quality performance data. (ABMS MOC: Myths & Facts)
- DATA ON PHYSICIAN PARTICIPATION IN MOC
Please go to What the Research Shows – Details: Data On Physician Participation in MOC.
WHAT THE RESEARCH SHOWS - DETAILS
1. PHYSICIANS’ CLINICAL SKILLS DECLINE OVER TIME
A systematic literature review of studies relating medical knowledge and health care quality to years in practice and physician age found that a majority of performance indicators decrease with increasing years in practice. (Choudhry NK et al 2005)
In an examination of an ABIM Recertification Examination, candidates further out of training performed less well on items testing new or changing knowledge, while performance on items testing stable knowledge was relatively constant across age groups. (Day SC et al 1988)
Analytic thinking by physicians tends to decline with age while non-analytic processing remains stable. With increasing reliance on prior experience, there is less of a tendency to incorporate novel information. (Eva KW 2002)
Physician care of diabetic hypertensive patients show decreases in quality that directly relate to the time since their last board certification. (Turchin TA et al 2008)
2. MOC IMPROVES THE CLINICAL PERFORMANCE AND OUTCOMES OF PARTICIPATING PHYSICIANS
A study of 76 family practice physicians found that participation in a MOC Part II activity (Self-Assessment Module) and a MOC Part IV activity (Performance in Practice Module) related to diabetes produced greater improvements in 11 of 24 process and intermediate outcome measures of diabetic patient care compared with physicians who did not complete these modules. (Galliher JM et al 2014)
A review of 7924 family physicians who completed a MOC Part IV activity (Performance in Practice Module) related to diabetes found that all physician and patient quality measures improved by completing the module. (Peterson LE et al 2014)
A review of 7319 family physicians who completed a MOC Part IV activity (Performance in Practice Module) related to hypertension resulted in improvement in most quality measures. (Peterson LE et al 2014b)
A review of the American Board of Family Medicine certification and recertification exam showed that family physicians who maintain certification perform better than recent graduates, and continued to improve their test scores with successive re-taking, reaching their highest scores 28-31 years after initial certification. (O’Neill TR et al 2013)
Skills learned by an anesthesiologist in a MOC course saved a patient’s life (letter). (Anson JA (letter) 2013)
Over half of the Family Medicines physicians completing Self-Assessment Modules in either hypertension or diabetes indicated that the experience would lead to changes in their practices. (Hagen MD et al 2006)
Higher ABIM MOC examination scores were significantly associated with improved diabetes care of patients, particularly intermediate outcomes. (Hess BJ et al 2012)
Outlier physicians who did not meet defined standards of care in the ABIM Diabetes Practice Improvement Module had lower certification and maintenance of certification examination scores. (Hess BJ et al, 2011)
Higher scores on the ABIM Internal Medicine Maintenance of Certification (MOC) examination are associated with better performance scores on Medicare quality indicators for diabetes and mammography screening. (Holmboe ES, Wang Y et al 2008)
Audit and feedback, which enables health care professionals to be evaluated on their practice, can be effective in improving professional practice (Jamtvedt G et al 2008)
Quality Improvement (QI) projects can work; the introduction of two central venous catheter-care practice bundles into 29 PICUs across the US resulted in significant and sustained reductions in infection rates. (Miller MR et al 2010)
Asthma severity appeared to be modestly lowered in patients whose physicians completed an ABIM Asthma Practice Improvement Module. (Simpkins J et al 2007)
3. MOC LEARNING METHODS ARE GROUNDED IN ADULT LEARNING RESEARCH
Active educational activities are more effective at changing physician behavior than passive ones; and because knowledge translation focuses on health outcomes and changing behavior, activities set in the actual site of practice and its social, organizational and policy environment are more effective than those set in learning situations. (Davis D et al 2003)
ABIM Practice Improvement Modules appear to identify meaningful clinical gaps in physician knowledge and skill needed to apply quality improvement methods to change practice systems and improve those measures. (Duffy FD et al 2008)
Two important principles for medical CME derived from adult learning research are: (1) the learning activity must have high relevance to what the physician actually does, and (2) the CME activity needs to be interactive, not passive. (Holmboe ES, Cassel C 2007)
Much recent research has focused on improving the microsystems of patient care, but the clinician’s knowledge base is still a crucial component of quality care. (Holmboe ES, Lipner R et al 2008)
CME educational programs work best when the interventions are interactive, multiple approaches are used, and the teaching is designed for a small group of physicians from a single discipline. (Mansouri M et al 2007)
Pay for Performance programs, if properly designed aligned with other features, can produce substantive and sustained improvement in patient care. (Mandel KE et al 2007)
Research indicates that continuing medical education (CME) strategies that enable and reinforce change are more likely than more passive activities to influence behavior. Three components that are most effective include: (1) an assessment of learning needs, that will help the physician recognize a need to change his or her behavior, knowledge base or skills, (2) an interactive process for the physician-learners with opportunities to practice the skills learned, and (3) sequenced and multifaceted educational activities. (Mazmanian PE et al 2002)
Interactive educational experiences can result in moderately large changes in professional practice; passive, didactic sessions are unlikely to do so. (O’Brien MA et al 2009)
ABIM Practice Improvement Modules were successfully used to teach medicine residents Quality Improvement (QI) skills. (Oyler J et al 2008)
Clinical quality and patient experience are distinct but related domains that may require separate measurement and improvement initiatives. (Sequist TD et al 2008)
4. A GROWING BODY OF EVIDENCE CORRELATES PHYSICIAN COMMUNICATION SKILLS WITH PATIENT OUTCOMES
See (Annotated Bibliography by www.healthcarecomm.org 2013).
5. THE PUBLIC EXPECTS PHYSICIANS TO ENGAGE IN CONTINUOUS LEARNING AND ASSESSMENT
A recent ABMS consumer survey found 95% of Americans say it is important to them that their doctors participate in a program to maintain their board certification. (ABMS 2010 Consumer Survey)
The public expects, in return for the privilege of self-regulation, that physicians undergo a rigorous, periodic examination of knowledge. (Brennan TA et al 2004)
A recent survey found that parents report a preference for board certified physicians and expect them to participate in MOC. (Freed GL et al 2008)
6. NATIONAL ORGANIZATIONS AND GOVERNMENTAL BODIES EXPECT PHYSICIANS TO ENGAGE IN CONTINUOUS LEARNING AND ASSESSMENT
A survey of hospitals revealed that between 2005 and 2010 a larger proportion now require board certification for all pediatricians at the point of initial privileging (24% vs 4%), although more hospitals now make exceptions to their board certification policies than they did before (99% vs 41%). (Freed GL et al 2013)
Working with the U.S. Congress led to the recognition of the ABMS MOC program as one of the ways physicians can meet the continuous learning requirements of the Centers for Medicare & Medicaid Services (CMS) Physician Quality Reporting System (PQRS). (ABMS MOC: Myths & Facts)
The Blue Cross Blue Shield Association (BCBSA), the trade association for the independent, locally operated BCBS Plans in the United States, is working with ABMS to explore the use of claims data to inform ABMS MOC program assessment activities. (ABMS MOC: Myths & Facts)
In partnership with the AMA, ABMS convened the Physician Consortium for Performance Improvement (AMA-PCPI) with a focus on developing practice measures for ABMS MOC quality improvement. (ABMS MOC: Myths & Facts)
7. SPECIALISTS SUPPORT MOC
Physicians find the ABIM MOC program personally (62%) and professionally (68%) valuable. (ABIM Certification & MOC – What the Research Shows)
84% of physicians who completed an ABIM self-assessment of knowledge module agree that it helped them identify further areas of study, and 86% agreed that it enhanced their competence in how to improve patient care. (ABIM Certification & MOC – What the Research Shows)
82% of physicians would recommend the ABIM Practice Improvement Module to a colleague. 75% indicated that they changed their practice as a result of the module. (ABIM Certification & MOC – What the Research Shows)
A study of pediatricians who had allowed their own certificates to expire still overwhelmingly believed that physicians who provide direct patient care should maintain their certification. (Freed GL et al 2008)
A majority of physicians in a small pilot study found that self-assessment using the ABIM’s diabetes practice improvement module was a valuable experience. (Holmboe ES, Meehan TP et al 2006)
In general, physicians value the MOC process for its effort to improve quality of care and patient safety. Physicians who have participated in MOC differ from those who have not participated in several ways: they are more likely to claim that MOC is required, to participate because it is required, to have more positive attitudes about maintaining certification, and to believe that the program requirements are appropriate. (Lipner RS et al 2006)
8. DATA ON PHYSICIAN PARTICIPATION IN MOC
ABMS Member Boards certify nearly 800,000 physicians in the United States. Currently, more than 375,000 physicians participate in an ABMS MOC program. (ABMS MOC: Myths & Facts)
It is projected that by 2020, 93 percent of board-certified physicians will hold time-limited certificates and be engaged in ongoing MOC activities. (ABMS MOC: Myths & Facts)
The average cost of participation in an MOC program across the 24 Member Boards is $500 per year. (ABMS MOC: Myths & Facts)
9. GAPS IN OUR KNOWLEDGE ABOUT MOC
Despite many positive and hopefully positive signs about the MOC program, significant uncertainty remains. There are no data employing dermatologist or dermatology patients to suggest that MOC is beneficial. The data regarding the association between certification and higher performance on quality measures may reflect a selection bias in that physicians able to deliver higher quality care become certified, with MOC adding little additional value. It is unclear whether participation in MOC actually leads to measureable improvements in quality dermatologic care. Successful pursuit of MOC may simply reflect "jumping though the hoops.”
The same is true for the contention that MOC has more to offer than simple CME. Data exist to suggest that physician performance and knowledge base deteriorate over time, but the data supporting this notion sampled physicians at one point in time and do not reflect actual trends in an individual's practice over time. It is possible that younger physicians are simply better at documentation of the parameters necessary to look good in quality measures, rather than delivering better care.
See also the recent JAAD CME article by Stratman E et al 2013.
ABIM Certification & MOC - What the Research Shows
ABMS 2010 Consumer Survey
ABMS Fact Sheet
ABMS MOC: Myths & Facts
Annotated Bibliography by www.healthcarecomm.org 2013
Anson JA (letter) 2013
Brennan TA et al 2004
Choudhry NK et al 2005
Davis D et al 2003
Day SC et al 1988
Duffy FD et al 2008
Eva KW 2002
Freed GL et al 2013
Freed GL et al 2008
Galliher JE etl al 2014
Hagen MD et al 2006
Hawkins RE et al 2013
Hess BJ et al 2012
Hess BJ et al, 2011
Holmboe ES, Cassel C 2007
Holmboe ES, Lipner R et al 2008
Holmboe ES, Meehan TP et al 2006
Iglehart JK et al 2012
Jamtvedt G et al 2008
Lipner RS et al 2013
Lipner RS et al 2006
Mandel KE et al 2007
Mansouri M et al 2007
Mazmanian PE et al 2002
Miller MR et al 2010
O'Brien MA et al 2009
O'Neill TR et al 2013
Oyler J et al 2008
Peterson LE et al 2014
Peterson LE et al 2014b
Sequist TD et al 2008
Simpkins J et al 2007
Stratman E et al 2013
Turchin TA et al 2008
Anson JA. Letter to the American Society of Anesthesiologists Newsletter, January, 2013 (77;1:2013)
Brennan TA, Horwitz RI, Duffy FD, Cassel CK, Goode LD, Lipner RS. The role of physician specialty board certification status in the quality movement. JAMA 2004;292(9):1038-43.
Davis D, Evans M, Jadad A, Perrier L, Rath D, Ryan D, et al. The case for knowledge translation: shortening the journey from evidence to effect. BMJ 2003;327;33-5.
Davis, D. A., Thompson, M. A., Freemantle, N., Wolf, F. M., Mazmanian, P., & Taylor-Vaisey,
A. (1999). Impact of formal continuing medical education. Do conferences, workshops,
rounds, and other traditional continuing education activities change physician behavior or
health care outcomes? Journal of the American Medical Association, 282(9), 867-874.
Duffy FD, Lynn LA, Didura H, Hess B, Caverzagie K, Grosso L, et al. Self-assessment of practice performance: development of the ABIM practice improvement module (PIM).J Contin Educ Health Prof 2008;28(1):38-46. (03)
Freed GL, Dunham KM, Gebremariam A. Changes in hospitals' credentialing requirements for board certification from 2005 to 2010. J Hosp Med 2013;8:298-303.
Freed GL, Dunham KM, Althouse LA. Characteristics of general and subspecialty pediatricians who choose not to recertify. Pediatrics 2008;121(4): 711-7.
Galliher JM, Manning BK, Petterson SM, Dickinson LM, Brandt EC, Staton EW et al. Do professional development programs for maintenance of certification (MOC) affect quality of patient care? J Am Board Fam Med 2014;27:19-25.
Hagen MD, Ivins DJ, Puffer JC, Rinaldo J, Roussel GH, Sumner W, et al. Maintenance of certification for family physicians (MC-FP) self assessment modules (SAMs): the first year. JABFM 2006;19(4):398-403.
Hawkins RE, Lipner RS, Ham HP, Wagner R, Holmboe ES. American board of medical specialties maintenance of certification: theory and evidence regarding the current framework. J Contin Educ Health Prof 2013;33:s7-s19.
Hess BJ, Weng W, Holmboe ES, Lipner RS. The association between physicians' cognitive skills and quality of diabetes care. Acad Med 2012;87(2):157-63.
Hess BJ, Weng W, Lynn LA, Holboe ES, Lipner RS. Setting a fair performance standard for physicians' quality of patient care. J Gen Intern Med 2011;26(5):467-73.
Holmboe ES, Cassel C. Continuing medical education and maintenance of certification: essential links. The Permanente Journal 2007;11(4):71-5.
Holmboe ES, Lipner R, Greiner A. Assessing quality of care: knowledge matters. JAMA 2008;299(3):338-40.
Holmboe ES, Meehan TP, Lynn L, Doyle P, Sherwin T, Duffy FD. Promoting physicians' self-assesmsent and quality improvement: the ABIM diabetes practice improvement module. J Contin Educ Health Prof 2006:26(2):109-19.
Holmboe ES, Wang Y, Meehan TP, Tate JP, Ho SY, Starkey KS, et al. Association between maintenance of certification examination scores and quality of care for medicare beneficiaries. Arch Intern Med 2008;168(13):1396-403.
Iglehart JK, Baron RB. "Ensuring physicians' competence - is maintenance of certification the answer? N Engl J Med 2012;367(26):2543-9.
Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD. Audit and feedback: effects on professional practice and health care outcomes (Cochrane review). The Cochrane Library 2008, Issue 4.
Lipner RS, Hess BJ, Phillips RL. Specialty board certification in the United States: issues and evidence. J Contin Educ Health Prof 2013;33:s20-s35.
Lipner RS, Bylsma WH, Arnold GK, Fortna GS, Tooker J, Cassel CK.Who is maintaining certification in internal medicine - and why? A national survey 10 years after initial certification. Annals of Internal Medicine, 2006;144(1), 29-36.
Mandel KE, Kotagal UR. Pay for performance alone cannot drive quality. Arch Pediatr Adolesc Med 2007;161(7):650-5. (An ABP MOC-approved Part IV activity.)
Mansouri, M., & Lockyer, J. (2007). A meta-analysis of continuing medical education
effectiveness. Journal of Continuing Education in the Health Professions, 27(1),
Mazmanian PE, Davis DA. Continuing medical education and the physician as learner: guide to the evidence. JAMA. 2002;288:1057-60.
Miles PV. The future of maintenance of certification: a reaction to the paper by Kevin B. Weiss, MD. JABFM 2010;23(Suppl):542-5.
Miles PV. Maintenance of certification: the profession's response to physician quality. Ann Fam Med 2011;9(3):196-7.
Miller MR, Griswold M, Harris JM II, Yenokyan G, Huskins C, Moss M. Decreasing PICU catheter associated bloodstream infections: NACHRI's quality transformation efforts. Pediatrics 2010;125(2):206-12.
O'Brien T, Freemandle N, Oxman AD, Wolf F, Davis DA, Herrin J. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2001;(2):CD003030.
O'Neill TR, Puffer JC. Maintenance of certification and its association with the clinical knowledge of family physicians. Acad Med 2013;88:780-7.
Oyler J, Vinci L, Arora V, Johnson J. Teaching internal medicine residents quality improvement techniques using the ABIM's practice improvement modules. J Gen Intern Med 2008;23(7):927-30.
Peterson LE, Blackburn B, Phillips RL, Puffer JC. Improving quality of care for diabetes through a maintenance of certification activity: family physicians' use of the chronic care model. J Contin Educ Health Prof 2014;34:47-55.
Peterson LE, Blackburn B, Puffer JC, Phillips RL. Family physicians' quality interventions and performance improvement for hypertension through maintenance of certification. J Healthcare Qual 2014;00;1-11.
Sequist TD, Schneider EC, Anastario M, Odigie EG, Marshall R, Rogers WH, Safran DG. Quality monitoring of physicians linking patients' experiences of care to clinical quality and outcomes. J Gen Intern Med 2008;23(11):1784-90.
Simpkins J, Divine G, Wang M, Holmboe ES, Pladevall M, Williams LK. Improving asthma care through recertification: a cluster randomized trial. Arch Intern Med. 2007;167:2240-8.
Stratman E, Kirsner RS, Horn TD. Maintenance of certification in dermatology: what we know, what we don't. J Am Acad Dermatol 2013;69:1-11.
Turchin A, Shubina M, Chodos AH, Einbinder JS, Pendergrass ML. Effect of board certification on antihypertensive treatment intensification in patients with diabetes mellitus. Circulation 2008; 117:623-28.