Effective October 2023, the ACGME has officially discontinued 10-Year Accreditation Site Visits for programs. While the program Self-Study is still required (Common Program Requirement V.C.2.), it will no longer be linked to or reviewed during a site visit.
The ACGME is finalizing its plans for the program Self-Study and developing its process for conducting accreditation site visits for programs with a status of Continued Accreditation. The ACGME will provide updated information by spring 2024. Note: Sponsoring Institution Self-Studies and 10-Year Accreditation Site Visits will proceed according to the Institutional Review Committee’s announced plan.
Programs can continue to use the following previously published information on the program Self-Study. Further guidance will be provided by the Institutional Review Committee and specialty Review Committees in the future.
The suggested steps described here are intended to offer guidance to programs conducting their Self-Study.
The Self-Study is an objective, comprehensive evaluation of the residency or fellowship program, with the aim of improving it. Underlying the Self-Study is a longitudinal evaluation of the program and its learning environment, facilitated through sequential annual program evaluations that focus on the required components, with an emphasis on program strengths and self-identified areas for improvement (“self-identified” is used to distinguish this dimension of the Self-Study from areas for improvement the Review Committee identifies during accreditation reviews).
To offer context for the Self-Study, there are two concepts: 1) an exploration of program aims; and 2) an assessment of the program’s institutional, local, and, as applicable, regional environment. Both are discussed in detail below. The focus on aims and the program’s environmental context is to enhance the relevance and usefulness of the annual program evaluation, and support improvement that goes beyond compliance with the requirements.
The Self-Study Group Participants: The members of the Program Evaluation Committee (PEC) are the ideal group for the Self-Study, as they are familiar with the Annual Program Evaluation process and the resulting action plans and improvement efforts. Including the program coordinator is also recommended.
Additional Participants: While the ACGME does not require additional participants in the Self-Study process, it may be beneficial to have other individuals offer their perspectives. This might include department leadership, a clerkship director, chief residents (both in the accredited years of training and beyond), or experts in education, curriculum design, or assessment. These individuals should be included if program leaders think that their contributions would be beneficial. The DIO may be able to provide suggestions for institutional experts to include.
CCC Representative: It may be beneficial to include a member of the Clinical Competency Committee (CCC) in the Self-Study group. The CCC possesses educational outcome data, which could provide key input into Self-Study discussions.
The basic component of the Self-Study is the Annual Program Evaluation. Other components include setting program aims and conducting an abbreviated strategic assessment of the program, focusing on strengths, areas for improvement, opportunities, and threats.
The first task of the Self-Study group is a discussion of program aims. Aims are program and institutional leaders’ views of key expectations for the program, as well as how the program differentiates itself from other programs in the same specialty/subspecialty. Program aims may focus on recruiting specific types of residents or fellows, preparing graduates for particular careers (clinical practice, academics, research, or primary/generalist care), improving care for underserved patients, engaging with health policy or advocacy, studying population health, or generating new knowledge.
Review this brief slide presentation on how to set and validate program aims.
Program aims should be agreed on by program and institutional leadership, and in some institutions, setting aims will be an institution-level initiative. In setting aims, programs should generally take a longer-term strategic view. However, aims may change over time. Factors such as a shift in program focus initiated by institutional or department leadership, changes in local or national demand for a resident workforce with certain capabilities, or new opportunities to train residents and fellows in a different setting may prompt revision of program aims.
The central data for the Self-Study is information from successive Annual Program Evaluations, with a focus on program strengths and self-identified areas for improvement; how improvements are prioritized, selected, and implemented; and follow-up to assess whether interventions were effective.
Added data for the Self-Study should relate to ongoing improvement activities and the perspectives of program stakeholders, such as results of the annual ACGME Resident and Faculty Surveys, and relevant departmental or institutional data.
Review the Guide to the Common Program Requirements for more information.
Data aggregation and evaluation should (1) address any active citations and areas for improvement from the program’s most recent review; (2) identify any additional areas where the program may not be in compliance with ACGME requirements; and (3) focus on improvement that goes beyond compliance with requirements, with particular attention to improvements relevant to the program’s aims.
Click here for a single year’s Annual Program Evaluation template.
Click here for tracking improvement longitudinally across multiple Annual Program Evaluations.
The next step in the Self-Study process is to conduct an assessment of the program’s environment. The rationale for examining opportunities for and threats facing the program is to provide context for the Self-Study.
Opportunities: Opportunities are external factors that are not entirely under the control of the program, but if acted on, will help the program flourish. Opportunities take many forms, such as access to expanded populations for ambulatory care at a local health center, a partnership with an institution with a simulation center, or availability of new clinical or educational technology through agreements with external parties.
Threats: Threats are also largely beyond the program’s control and come in many forms. They could result from a change in support for resident/fellow education at the national level, from changing priorities at the institutional or state level, or from local factors, such as erosion of a primary ambulatory system based on voluntary faculty. The benefit of assessing program threats is that plans can be developed to mitigate their effect.
Data should be confirmed and augmented by information from program stakeholders (residents, fellows, faculty members, others as relevant). In some cases, important information may include the perceptions of representatives from other specialties who interact with the program’s residents or fellows.
To collect this information, the program may use surveys, conduct meetings with residents/fellows, or organize a retreat. Feedback from recent graduates could also provide useful data on the program’s educational effectiveness.
Engagement of stakeholders in ongoing conversations about what does and does not work in the program is a critical component of the Self-Study. Stakeholders should also be engaged in a discussion of program aims and an assessment of program context, either as part of the Self-Study or Annual Program Evaluation, or as a stand-alone activity to jumpstart the program’s improvement process.
Program leaders, the program coordinator, and others as needed, should assemble a “program improvement” file from prior Annual Program Evaluations and past action plans to use as a starting point for this program improvement effort.
The next step is to interpret the aggregated data from the Self-Study. The Self-Study group will:
The Self-Study findings from the five-year look forward and the vision for the program should be shared with faculty members and residents and/or fellows. This step should validate the findings and improvement priorities identified by the Self-Study group with these key stakeholders.
For a specialty program with dependent subspecialty programs, there should be a discussion about any common strengths, areas for improvement, and shared opportunities and threats for some or all of the dependent subspecialties. These may be important priorities for improvements, particularly those requiring institutional resources. Programs should maintain a list of strengths, areas for improvement, and opportunities and threats shared among some or all of the dependent subspecialties.
Programs should maintain a document for their own records that lists the strengths and areas of improvement identified during the Self-Study process in a “program improvement” file.
The next step for the Self-Study group, or an individual designated by the group, is to compile a succinct Self-Study document that describes the process and key findings in the areas of program aims, threats and opportunities assessment, and program strengths and areas for improvement.
Click here for a sample Self-Study Summary template.
Ideally, the role of data collection, aggregation, and tracking of progress for these areas should be assigned to an individual or to a small group (with each individual responsible for a particular area of improvement).
The time between completion of the Self-Study with the Annual Program Evaluation and the following year’s Annual Program Evaluation will allow programs time to evaluate improvements made in areas the program.
During this evaluation, the program should assess and document progress in areas for improvement identified during the Self-Study.
The PEC or, if desired, the Self-Study group, should review the data collected for areas of improvement identified during the Self-Study.
When the PEC conducts this evaluation prior, a key area to be assessed pertains to the improvements made in areas identified during the Self-Study.
In most cases, aims will take a longer-term perspective. However, aims may change over time, and it is beneficial to reassess them as part of the Annual Program Evaluation. In addition, the program’s context—opportunities and threats—should be reassessed for changes in the environment.
It is important to then discuss program aims, improvements achieved, and other elements of the program’s strategic assessment with program stakeholders.
The information on aims and the environmental assessment should be shared and discussed with program leadership and stakeholders. This is another opportunity for faculty members, trainees, the program coordinator, and any other appropriate individuals to have an improvement-focused conversation about the program.