ACGME Outcome Project

Outcome Project Think Tank

The RRC Outcome Project Think Tank is an ad hoc advisory group whose purpose is to facilitate implementation of outcomes assessment according to ACGME program requirements. Because members either have been or currently are program directors as well as members of RRCs, they recognize the need for practicality and usefulness in whatever methods are eventually recommended for implementation by the RRCs.

Working with ACGME staff, Think Tank members reviewed current literature on specific competencies and their assessment. Their suggestions for Interpersonal and Communication Skills, Professionalism, and Practice-based Learning and Improvement (listed below) are based on these reviews, on their experience in the field, and on their own practical knowledge of "what works." The fact that several methods are suggested for each competency reflects the principle that multiple measures ultimately provide the most valid and reliable evidence as to whether competency has been achieved.

The Think Tank's suggestions will be useful as the RRCs begin to formulate specialty-specific expectations for a dependable system of assessing the competencies. Think Tank suggestions can also guide programs as they develop their own assessment approaches to the competencies.

Suggestions

The Think Tank suggests use of the following methods to assess:

Interpersonal and Communication Skills

What to Assess	How to Assess
Resident's overall interpersonal and communication skills during a rotation	Global performance rating AND 360^o Assessment, to be completed by the resident, professional associates, and patients (unless patient care is not the primary activity of the specialty)
Communication in difficult situations, e.g., breaking bad news, dealing with a non- compliant patient, a frightened patient, or a patient whose ethnicity differs from the resident	Focused observation and evaluation

Professionalism

What to Assess	How to Assess
Ethics knowledge	Vignettes with multiple choice and open-ended responses
Professional behaviors	360° Assessment, to be completed by the resident, professional associates, and patients (unless patient care is not the primary activity of the specialty)
The learning environment	Questionnaire completed by the residents

Practice-based Learning and Improvement

What to Assess	How to Assess
Processes and behaviors Reflection on practice and practice analysis Utilization of current literature A plan for change/improvement	Portfolio with checklist assessment OR Discussion-based assessment (e.g., chart stimulated) with • Checklist/encounter card (preferred) OR • Global performance rating
Outcomes, i.e., performance improvements	Record-based assessment with • Checklist/encounter card (preferred) OR • Global performance rating OR Discussion-based assessment with • Checklist/encounter card (preferred) OR • Global performance rating