Search Programs
and Institutions
Search Programs
and Institutions
Menu

Self-Study

Eight Steps for Conducting The ACGME Program Self-Study

The suggested eight-step sequence described here is intended to offer guidance to programs conducting their first self-study.

The self-study is an objective, comprehensive evaluation of the residency or fellowship program, with the aim of improving it. Underlying the self-study is a longitudinal evaluation of the program and its learning environment, facilitated through sequential annual program evaluations that focus on the required components, with an emphasis on program strengths and “self-identified” areas for improvement (“self-identified” is used to distinguish this dimension of the self-study from areas for improvement the Review Committee identifies during accreditation reviews).

To offer context for the self-study, there are two new concepts: 1) an exploration of program aims; and 2) an assessment of the program’s institutional, local and, as applicable, regional environment. Both are discussed in detail below. The focus on aims and the program’s environmental context is to enhance the relevance and usefulness of the program evaluation, and support improvement that goes beyond compliance with the requirements.

Additional Notes

Conducting the self-study for a dependent subspecialty program

The ACGME has placed added responsibility for oversight of subspecialty programs on the core program and sponsoring institution.

The self-study group for the core program should try to coordinate activities with the self-study groups for any dependent subspecialty programs, to take advantage of common dimensions, explore potential synergies, and reduce the burden that may be associated with conducting an independent self-assessment.

The 10-year site visits for subspecialty programs will be coordinated with the visit of their respective core program.

1. Assemble the Self-Study Group

Membership: The members of the Program Evaluation Committee (PEC) are the ideal core group for the Self-Study, as they are familiar with the Annual Program Evaluation process and the resulting action plans and improvement efforts. Including the program coordinator is also recommended.

Additional Participants: While the ACGME does not require additional participants in the Self-Study process, it may be beneficial to have other individuals offer their perspectives. This might include department leadership, a clerkship director, chief residents (both in the accredited years of training and beyond), or experts in education, curriculum design, or assessment. These individuals should be included if program leaders think that their contributions would be beneficial. The DIO may be able to provide suggestions for institutional experts to include.

CCC Representative: It may be beneficial to include a member of the Clinical Competency Committee (CCC) in the Self-Study group. The CCC possesses educational outcome data, which could provide key input into Self-Study discussions.

2. Engage Program Leaders and Constituents in a Discussion of Program Aims

The basic components of the Self-Study is an Annual Program Evaluation. Added components include setting program Aims and conducting an abbreviated strategic assessment of the program, focusing on strengths, areas for improvement, opportunities, and threats.

The first task of the Self-Study group is a discussion of program aims. Aims are program and institutional leaders’ views of key expectations for the program, as well as how the program differentiates itself from other programs in the same specialty/subspecialty. Aims may focus on the types of trainees recruited by the program, or on preparing graduates for particular careers (clinical practice, academics, research, or primary/generalist care). Aims may also include other objectives, such as care for underserved patients, health policy or advocacy, population health, or generating new knowledge.

Review this brief slide presentation on how to set and validate program aims

Program aims should be vetted with program and institutional leadership, and in some institutions, setting aims will be an institution-level initiative. In setting aims, programs should generally take a longer-term strategic view. However, aims may change over time. Factors such as a shift in program focus initiated by institutional or department leadership, changes in local or national demand for a resident workforce with certain capabilities, or new opportunities to train residents and fellows in a different setting may prompt revision of program aims.

3. Aggregate and Analyze Data from Your Annual Program Evaluations and the Self-Study to Create a Longitudinal Assessment of Program Strengths and Areas for Improvement

The core data for the Self-Study is information from successive Annual Program Evaluations, with a focus on program strengths and self-identified areas for improvement; how improvements are prioritized, selected, and implemented; and follow-up to assess whether interventions were effective.

Added data for the Self-Study should relate to ongoing improvement activities and the perspectives of program stakeholders, such as results of the annual ACGME Resident and Faculty Surveys, and relevant departmental or institutional data.

Review a list of high-value data suggested for use in the Annual Program Evaluation and the Self-Study

Data aggregation and evaluation should (1) address any active citations and areas for improvement from the program’s most recent review; (2) identify any additional areas where the program may not be in compliance with ACGME requirements; and (3) focus on improvement that goes beyond compliance with requirements, with particular attention to improvements relevant to the program’s aims.

Access these simple-to-use (optional) forms for aggregating data for a single year’s Annual Program Evaluation and for tracking improvements longitudinally across multiple Annual Program Evaluations.

4. Examine the Program’s Environment for Opportunities and Threats

The next step in the Self-Study process is to conduct an assessment of the program’s environment. The rationale for examining opportunities for and threats facing the program is to provide context for the Self-Study.

Opportunities: Opportunities are external factors that are not entirely under the control of the program, but if acted on, will help the program flourish. Opportunities take many forms, such as access to expanded populations for ambulatory care at a local health center, partnering with an institution with a simulation center, or availability of new clinical or educational technology through agreements with external parties.

Threats: Threats also are largely beyond the control of the program, and like opportunities, come in many forms. They could result from a change in support for resident/fellow education at the national level, from changing priorities at the institutional or state levels, or from local factors, such as erosion of a primary ambulatory system based on voluntary faculty. The benefit of assessing program threats is that plans can be developed to mitigate their effect.

Review this resource for how to conduct a SWOT analysis (an environmental assessment)

5. Obtain Stakeholder Input on Strengths, Areas for Improvement, Opportunities, and Threats to Prioritize Actions

These data should be confirmed and augmented by information from program stakeholders (residents/fellows, faculty members, others as relevant). In some cases, important information may include the perceptions of representatives from other specialties who interact with the program’s residents or fellows.

To collect this information, the program may use surveys, conduct meetings with residents/fellows, or organize a retreat. Feedback from recent graduates could also provide useful data on the program’s educational effectiveness. It is planned that programs will receive formative feedback on their Self-Study for the next five years. The only circumstance under there may be an accreditation impact is if the program does not conduct a Self-Study and submit the Self-Study summary to the ACGME.

Engagement of stakeholders (faculty members, residents, and others, as determined by program leaders) in ongoing conversations about what works and what does not work in the program is a critical component of the Self-Study. Stakeholders should also be engaged in a discussion of program aims and an assessment of program context, either as part of the Self-Study or Annual Program Evaluation, or as a stand-alone activity to jumpstart the program’s improvement process.

See this Guidance for the Plan-Do-Study-Act (PDSA) process for critical questions to ask at the four phases of the process.

Program leaders, the program coordinator, and others as needed, should assemble a “program improvement” file from prior Annual Program Evaluations and past action plans to use as a starting point for this program improvement effort.

6. Interpret the Data and Aggregate the Self-Study Findings

The next step is to interpret the aggregated data from the Self-Study. Specific elements will include:

  1. establishing the working set of program aims;
  2. listing key program strengths;
  3. prioritizing among self-identified areas for improvement to select those for active follow-up, and to help define specific improvement activities;
  4. discussing opportunities that may enhance the program, and developing plans to take advantage of them;
  5. discussing threats identified in the Self-Study, and developing plans to mitigate their impact
  6. conducting a five-year look-back using the data from Annual Program Evaluations; and,
  7. conducting a five-year look forward that also seeks to answer the question, “What will take this program to the next level?”

7. Discuss and Validate the Findings with Stakeholders

The Self-Study findings from Step 6, particularly the five-year look forward and the vision for the program, should be shared with faculty members and residents/fellows. This step should validate the findings and improvement priorities identified by the Self-Study group with these key stakeholders.

For a core program with dependent subspecialty programs, there should be a discussion about any common strengths, areas for improvement, and shared opportunities and threats for some or all of the dependent subspecialties. These may be important priorities for improvements, particularly those requiring institutional resources.

8. Develop a Succinct Self-Study Document for Use in Further Program Improvement as Documentation for the Program’s 10-Year Site Visit

The final step for the Self-Study group, or an individual designated by the group, is to compile a succinct Self-Study document that describes the process and key findings in the areas of program aims, the threats and opportunities assessment, and program strengths and areas for improvement.

The ACGME’s Self-Study Summary does not include information on program strengths and areas for improvement. The rationale for this is to allow programs to conduct a frank assessment of areas for improvement, and to be able to report on improvements at the time of their 10-Year Accreditation Site Visit.

In addition to completing the Self-Study summary to be sent to the ACGME, programs should maintain a document for their own records that lists the strengths and areas of improvement identified during the Self-Study.

Access the Self-Study Summary for core and large specialty programs or, for optional use by smaller subspecialty programs, the Self-Study Summary Short Form

At the time of the 10-Year Accreditation Site Visit (12 to 18 months later), the program will list its strengths and provide a written update describing improvements already been realized since the Self-Study.

A list of strengths, areas for improvement, and opportunities and threats shared among some or all of the dependent subspecialties should also be maintained. Some of this information will be required in the Summary of Achievements programs must complete prior to their 10-Year Accreditation Site Visit.

Ideally, the role of data collection, aggregation, and tracking of progress for these areas should be assigned to an individual or to a small group (with each individual responsible for a particular area of improvement).

For the 10-Year Accreditation Site Visit, the ACGME will not ask programs to provide any information on areas identified during the Self-Study that have not yet resulted in improvements.

Click here for Eight Steps to Prepare for the 10-Year Accreditation Site Visit

Contact Us:
Quick Links
Self-Study Documents
Accreditation
Council for
Graduate
Medical
Education
Accreditation Council for Graduate Medical Education
  • Suite 2000
    401 North Michigan Avenue
    Chicago, IL 60611
  • Telephone: 312.755.5000
    Facsimile: 312.755.7498
  • Data Systems Technical Support: 312.755.7474
  • Browser Compatibility
© 2000-2017 ACGME CMS Login