During the fall of 2006, 24 hospitals associated with the Lake Erie Consortium for Osteopathic Medical Training (LECOMT) completed the competency development program from Medical Professional Performance Systems http://mpps.orgtalk.com/demo . Specialty areas participating included Family Medicine, Emergency Medicine, Internal Medicine, Orthopedics, Psychiatry, and General Surgery. The assessment tool is a multi-rater system that asks qualified raters to evaluate residents on the seven American Osteopathic Association competencies - the six ACGME competencies and osteopathic philosophy and osteopathic manipulative medicine competencies.1,2 Each rater completes 67 items on a ten-point Likert scale based on the Dreyfus and Dreyfus skill acquisition model.3 Evaluators identify priority areas for resident growth and development. Residents perform a simultaneous self-evaluation. The assessments are completed at a password-protected website. Each resident receives personalized results across 20 competency dimensions; results include individualized curricula for growth and development specific to the resident’s current skill level. Residents and faculty members create an action plan based on an assessment of priorities for that specific resident.

From November 1 to December 15, 2006, 263 raters evaluated 226 residents. An average of five raters, typically including an attending physician, nurse, and resident peers, evaluated each resident. Mean competency domain scores, averaged across items and residents, ranged from 6.16 to 7.95 out of 10; individual resident domain scores ranged from 1.17-10. Across the entire consortium, residents were rated lowest in the areas of Systems-Based Practice and Practice-Based Learning and Improvement. They were rated highest on Professionalism and Interpersonal and Communication Skills. The specific dimensions that were seen as highest priorities for development included: developing patient care plans, improving basic medical knowledge, practicing cost-effective medicine, and understanding the complexities of the local health-care system.

After receiving their reports, residents met with faculty to develop individual action plans that included specific activities, and use of resources to help guide their development. The faculty reported that most residents came to the meeting with some initial ideas about their action plans and used their time with faculty to refine their activities for improvement in the targeted competencies. In the spring, some of the hospitals will conduct a second evaluation to determine the effectiveness of action plans. All programs will participate in a second full cycle in the fall of 2007. Program costs are $100 per resident annually.

References

Lockyer J. Multisource feedback in the assessment of physician competencies. J Contin Educ Health Prof. 2003;23:4–12.

Gallagher M (chairman). Report of the Core Competency Task Force, 2003. Chicago: American Osteopathic Association, 2003.

Batalden P, Leach D, Swing S, Dreyfus H, Dreyfus S. General competencies and accreditation in graduate medical education. Health Affairs. 2002;21(5):103-111.