1. UPDATE INTERNAL REVIEW PROTOCOL
   
   Update the internal review protocol to include language from the institutional requirements concerning the general competencies. At a minimum, the protocol should include the following:
   
   
   1. The internal review is to assess whether each program has defined, in accordance with the relevant Program Requirements, the specific knowledge, skills, and attitudes required and provides educational experiences for the residents to demonstrate competency in the following areas: patient care skills, medical knowledge, interpersonal and communication skills, professionalism, practice-based learning, and systems-based practice.
      
      
   2. The internal review is to provide evidence of the program's use of evaluation tools to ensure that the residents demonstrate competence in each of the six areas.
      
      
   3. The internal review is to appraise the development and use of dependable outcome measures by the program for each of the general competencies.
      
      
   4. The internal review is to appraise the effectiveness of each program in implementing a process that links educational outcomes with program improvement.
      
      
2. UPDATE INTERNAL REVIEW QUESTIONNAIRES
   
   If the institution uses a program director's questionnaire, resident's questionnaire, or has instructions on what to include in an internal review report, update these documents to include questions on the six competencies as well. It will be important for the program directors to respond to the following so that it may be documented in the internal review report:
   
   
   1. Can the program director provide adequate evidence of a curriculum, complete with goals and objectives, that is used by the program for teaching the following six general competencies: patient care skills, medical knowledge, interpersonal and communication skills, professionalism, practice-based learning, and systems-based practice?
      
      
   2. Has the program director developed adequate tools to evaluate resident competencies in the six areas based on the goals and objectives?
      
      
   3. Can the program director provide a list of the evaluation tools he/she is using for each of the six competencies (see attached example) and can he/she provide documented evidence of these tools to the internal review committee?
      
      
      
      d. Can the program director provide any evidence of developing or using dependable measures to access the residents' competence in each of these areas? Note: Dependable measures may not be developed until the program has had an opportunity to evaluate its residents over a period of time using various evaluation tools and analyzing the results.
      
      
   4. Can the program director provide any evidence of a process developed to link educational outcomes with program improvement? Note: Evidence of program improvement may not be available at first until the program has had an opportunity to develop and analyze its outcome measures over a period of time.
      
      
3. INTERVIEWS WITH RESIDENTS DURING THE INTERNAL REVIEW
   
   When interviewing residents during the internal review, seek verification as to the existence of a curriculum with goals and objectives for teaching the competencies, their involvement in the curriculum, and the kinds of tools used by the program to evaluate them. Do their responses match the program director's?
   
   
4. INITIAL DOCUMENTATION FOR THE INTERNAL REVIEW REPORT
   
   As part of each internal review report (conducted after July 1, 2002) there should be
   
   
   1. verification of the existence of a curriculum with goals and objectives provided for several of the general competencies;
      
      
   2. a summary or list of the types of evaluation tools used by the program for evaluating the competencies (see example attached);
      
      
   3. comments on the program's status in the development and use of dependable measures to assess resident competency in the six areas;
      
      
   4. comments on the program's status in developing a process that links educational outcomes with program improvement;
      
      
   5. verification or confirmation from the residents as to the existence of a curriculum with goals and objectives for teaching the competencies, their involvement in the curriculum, and the kinds of tools used by the program to evaluate them.

EXAMPLE

AN INTERNAL REVIEW CHECKLIST FOR THE GENERAL COMPETENCIES

Can be completed as part of a Program Director's questionnaire if used in the internal review

or can be completed by the internal reviewers.

Example of review for an Internal Medicine Program

List the evaluation tools used by the program for the following General Competencies:

General Competencies		List Evaluation Tools Used or In Development by the Program (completed by program director in internal med. at Sample Hospital)
						Other tools designed by program?
Patient Care		Developing OSCE	Mini CEX	Patient Surveys	Procedure Logs
Medical Knowledge		Chart Stimulated Recall	Oral Exam	Written Exam-Multiple Choice
Interpersonal and Communication Skills		Developing 360 Degree	Patient Surveys	Standardized Patients
Professionalism		360 Degree	Checklist
Practice Based Learning		Resident Portfolios	Developing Oral Exam	Record Review
Systems Based Practice		Developing Resident Portfolios	Developing 360 Degree

The Program Director is to provide the internal review committee with

1. documented evidence of a curriculum with goals and objectives for the general competencies currently implemented;
   
   
2. documented evidence of the evaluation tools used that he/she has listed;
   
   
3. the status of developing and using dependable measures to assess a resident's competence in these areas; and,
   
   
4. the status of developing a process that links educational outcomes with program improvement.