ACGME Outcome Project

Search
Site Map
Contact Us

References

Assessment: General | Interpersonal and Communication Skills | Professionalism

General

Goldman RL. The reliability of peer assessments. A meta-analysis. Eval Health Prof 1994; 17:3-21. Abstract: A meta-analysis of studies examining the interrater reliability of the standard practice of peer assessments of quality of care was conducted. Using the Medline, Health Planning and Administration, and SCISEARCH databases, the English-language literature from 1966 through 1991 was searched for studies of chance corrected agreement among peer reviewers. The weighted mean kappa of 21 independent findings from 13 studies was .31. Comparison of this result with widely used standards suggests that the interrater reliability of peer assessment is quite limited and needs improvement. Research needs to be directed at modifying the peer review process to improve its reliability or at identifying indexes of quality with sufficient validity and reliability that they can be employed without subsequent peer review.
Hodges B, Turnbull J, Cohen R, Bienenstock A, Norman G. Evaluating communication skills in the OSCE format: reliability and generalizability. Med Educ 1996;30:38-43. Abstract: In most objective structured clinical examinations (OSCEs), communication skills are assessed as an 'add-on' to history-taking stations, rather than in stations designed to assess communication skills in the broadest sense. This study investigated the feasibility of developing such stations. In part one, 60 clinical clerks and 36 residents were rated in four 10-min emotionally charged situations portrayed by standardized patients. Inter-rater reliability was demonstrated (r = 0.59-0.63) and a highly significant effect of educational level was found. Generalizability between communication stations was low (0.17-0.20). Several explanations for poor generalizability, including poor discrimination as a result of low score variance and the confounding effect of content knowledge, were addressed in part 2. Ninety-five final-year medical students participated in an OSCE in which six 10-min encounters examined the students' ability to manage difficult emotional situations such as fear, anxiety, mania, sadness, confusion and anger. Half the students encountered a patient with moderate emotional symptoms and half an extreme emotional state. For difficult stations, students' scores were lower and standard deviation higher, suggesting that manipulating difficulty increases score variance and potentially discrimination. However, a strong interaction was found between difficulty and station content, and communication scores were highly correlated with content. Scenarios which created major communication difficulties (such as mania) resulted in much larger differences in scores between the easy and difficult versions. Communication OSCE stations can be created with acceptable reliability including difficult cases which address communication skills beyond simple history taking. Nevertheless, a generalizable set of communication skills remains elusive.
Reprinted with permission by Blackwell Science, Inc.
Jankowski J, Crombie I, Block R, Mayet J, McLay J, Struthers AD. Self-assessment of medical knowledge: do physicians overestimate or underestimate? J R Coll Physicians Lond 1991;25:306-8. Abstract: The relationship between doctors' medical knowledge and their inability to assess correctly what they know was investigated. Sixty out of 65 hospital physicians sat a MRCP Part 1 multiple-choice examination. In addition to the factual questions, they had to estimate how certain they were that their answers were correct. We confirmed that factual knowledge increased with clinical experience from the grade of house officer through to that of senior registrar. The self-assessment of likelihood of being correct revealed that, on average, doctors underestimated their knowledge by 8%. However, those who had passed their MRCP examination within the past three years overestimated on average by 6%. We suggest that this inadequacy of self-assessment could have serious clinical implications, and should be assessed.
Matthews DA, Feinstein AR. A new instrument for patients' ratings of physician performance in the hospital setting. J Gen Intern Med 1989; 4:14-22. Abstract: A new instrument to elicit patients' appraisals of physician performance has been developed from a previously-derived taxonomy of desired physician attitudes and behavior. The instrument allows patients to give ratings for their physicians' discrete, observable items of behavior, and also for complex, multidimensional attributes. When the instrument was administered to 131 randomly chosen medical inpatients, the results showed that technical competence and interpersonal (or humanistic) qualities were equally valued, and that physicians received high ratings for most features of performance. Except for less satisfaction in younger patients, clinical and demographic characteristics had little impact on the ratings. The performance characteristics of the instrument appear satisfactory, and its potential applications and proposals for further research are discussed.
Reprinted with permission by Blackwell Science, Inc.
Murray E, Gruppen L, Catton P, Hays R, Woolliscroft JO. The accountability of clinical education: its definition and assessment. Med Educ 2000;34:871-9. Abstract: BACKGROUND: Medical education is not exempt from increasing societal expectations of accountability. Competition for financial resources requires medical educators to demonstrate cost-effective educational practice; health care practitioners, the products of medical education programmes, must meet increasing standards of professionalism; the culture of evidence-based medicine demands an evaluation of the effect educational programmes have on health care and service delivery. Educators cannot demonstrate that graduates possess the required attributes, or that their programmes have the desired impact on health care without appropriate assessment tools and measures of outcome. OBJECTIVE: To determine to what extent currently available assessment approaches can measure potentially relevant medical education outcomes addressing practitioner performance, health care delivery and population health, in order to highlight areas in need of research and development. METHODS: Illustrative publications about desirable professional behaviour were synthesized to obtain examples of required competencies and health outcomes. A MEDLINE search for available assessment tools and measures of health outcome was performed. RESULTS: There are extensive tools for assessing clinical skills and knowledge. Some work has been done on the use of professional judgement for assessing professional behaviours; scholarship; and multiprofessional team working; but much more is needed. Very little literature exists on assessing group attributes of professionals, such as clinical governance, evidence-based practice and workforce allocation, and even less on examining individual patient or population health indices. CONCLUSIONS: The challenge facing medical educators is to develop new tools, many of which will rely on professional judgement, for assessing these broader competencies and outcomes.
Reprinted with permission by Blackwell Science, Inc.
Quattlebaum TG. Techniques for evaluating residents and residency programs. Pediatrics 1996;98 Pt 2:1277-83. Abstract: A comprehensive, functioning evaluation system is an important component of a residency program. It should focus on the residency program as well as on the residents and should provide feedback to the residents, their teachers, and the program director. Such a system allows residents and their faculty advisors to receive timely, ongoing, formative feedback concerning resident progress. Likewise, it can help the faculty recognize the strengths and weaknesses of the training program by providing data that can lead to the curriculum changes needed to improve resident experiences. Additionally, it can alert the residency program director to residents whose performance is significantly below that of their peers, allowing intensive faculty and advisor help for these individuals. The practical aspects of providing feedback and implementing an evaluation system apply no less to community-based educational experiences than they do to inpatient and outpatient training areas.
Stewart J, O'Halloran C, Barton JR, Singleton SJ, Harrigan P, Spencer J. Clarifying the concepts of confidence and competence to produce appropriate self-evaluation measurement scales. Med Educ 2000;34:903-9. Abstract: INTRODUCTION: This paper reviews the literature on self-evaluation and discusses the findings of a small-scale alitative study which explored the terms 'confidence' and 'competence' as useful measures in a self-evaluation scale. Four pre-registration house officers took part in interviews and completed a provisional instrument to assess their perceived competence. FINDINGS: Competence confidence are useful terms for house officers expressing beliefs about their ability to perform their job but the terms should not be used synonymously. In our study, 'competent' represented what individuals knew about their ability and was based on the individual's previous experience of the task. 'Confident' described a judgement which influenced whether an individual was willing or not to undertake an activity. Confidence was not necessarily based on known levels of competence and therefore performance of tasks which were unfamiliar to the house officer also involvedthe assessment of risk. The authors give examples of task and skill scales which may be useful in the process of self-evaluation by preregistration house officers. CONCLUSIONS: The authors suggest that the process of assessing oneself is complicated, and by its very nature can never be objective or free from the beliefs and values individuals hold about themselves. Therefore self-evaluation instruments are best used to help individuals analyse their work practices and to promote reflection on performance. They should not be used to judge the 'accuracy' of the individual's evaluation.
Reprinted with permission by Blackwell Science, Inc.
Thomas PA, Gebo KA, Hellmann DB. A pilot study of peer review in residency training. J Gen Intern Med 1999;14:551-4. Abstract: OBJECTIVE: To explore the utility of peer review (review by fellow interns or residents in the firm) as an additional method of evaluation in a university categorical internal medicine residency program. DESIGN/PARTICIPANTS: Senior residents and interns were asked to complete evaluations of interns at the end-of-month ward rotations. MAIN RESULTS: Response rates for senior residents evaluating 16 interns were 70%; for interns evaluating interns, 35%. Analysis of 177 instruments for 16 interns showed high internal consistency in the evaluations. Factor analysis supported a two-dimensional view of clinical competence. Correlations between faculty, senior resident, and intern assessments of interns were good, although varied by domain. CONCLUSIONS: An end-of-year attitude survey found that residents gave high ratings to the value of feedback from peers.
Reprinted with permission by Blackwell Science, Inc.
Turnbull J, MacFadyen J, Van Barneveld C, Norman G. Clinical work sampling A new approach to the problem of in-training evaluation. J Gen Intern Med 2000;15:556-61. Abstract: OBJECTIVE: Existing systems of in-training evaluation (ITE) have been criticized as being unreliable and invalid methods for assessing student performance during clinical education. The purpose of this study was to assess the feasibility, reliability, and validity of a clinical work sampling (CWS) approach to ITE. This approach focused on the following: (1) basing performance data on observed behaviors, (2) using multiple observers and occasions, (3) recording data at the time of performance, and (4) allowing for a feasible system to receive feedback. PARTICIPANTS: Sixty-two third-year University of Ottawa students were assessed during their 8-week internal medicine inpatient experience. MEASUREMENTS AND MAIN RESULTS: Four performance rating forms (Admission Rating Form, Ward Rating Form, Multidisciplinary Team Rating Form, and Patient's Rating Form) were introduced to document student performance. Voluntary participation rates were variable (12%- 64%) with patients excluded from the analysis because of low response rate (12%). The mean number of evaluations per student per rotation (19) exceeded the number of evaluations needed to achieve sufficient reliability. Reliability coefficients were high for the Ward Form (.86) and the Admission Form (.73) but not for the Multidisciplinary Team (.22) Form. There was an examiner effect (rater leniency), but this was small relative to real differences between students. Correlations between the Ward Form and the Admission Form were high (.47), while those with the Multidisciplinary Team Form were lower (.37 and .26, respectively). The CWS approach ITE was considered to be content valid by expert judges. CONCLUSIONS: The collection of ongoing performance data was reasonably feasible, reliable, and valid.
Reprinted with permission by Blackwell Science, Inc.
Winckel CP, Reznick RK, Cohen R, Taylor B. Reliability and construct validity of a structured technical skills assessment form. Am J Surg 1994;167:423-7. Abstract: Current methods of evaluating technical competence of surgical residents are subjective and potentially unreliable. This study assesses the reliability and construct validity of a new format for the assessment of technical ability, the two part Structured Technical Skills Assessment Form (STSAF). Part I, which is completed while an operation is proceeding consists of approximately 120 essential components of the procedure. Part II, completed at the end of the operation, is a 10-point global rating form. Forty-one operations were evaluated using the STSAF, with multiple observers present at 26. Inter- rater reliability of both Parts I and II were high (.78 and .73, respectively). Statistically significant differences were noted between senior-resident and junior-resident performances, suggesting construct validity. The incorporation of structured guidelines to the assessment of technical skill leads to high inter-rater reliability and construct validity, which ultimately may result in improved and reproducible evaluations of surgical trainees.
Reprinted from Am J Surg,167,Winckel CP, Reznick RK, Cohen R, Taylor B, Reliability and construct validity of a structured technical skills assessment form, 423-7, Copyright 1994 with permission from Excerpta Media, Inc. Elsevier

Interpersonal and Communication Skills

Argent J, Faulkner A, Jones A, O'Keeffe C. Communication skills in palliative care: development and modification of a rating scale. Med Educ 1994;28:559-565. Abstract: A complex rating scale has been used to evaluate the verbal communication skills of health professionals before and after training. The scale has been modified in order to decrease the useage time, to improve its accuracy and to develop a rating tool available to teachers. This paper describes the process of the modification and discusses the consequent validation of the scale.
Reprinted with permission by Blackwell Science, Inc.
Bertakis KD, Azari R, Callahan EJ, Robbins JA, Helms LJ. Comparison of primary care resident physicians' practice styles during initial and return patient visits. J Gen Intern Med 1999;14:495-8. Abstract: New adult patients (n = 212) were randomly assigned to 58 primary care resident physicians. Physician practice styles during initial and return visits were analyzed using the Davis Observation Code. Compared with initial patient visits, return visits were shorter, but more work- intensive. Return visits displayed significantly less technically oriented behavior (including history taking, physical examination, and treatment planning) and fewer discussions regarding use of addictive substances; however, there was more emphasis on health behaviors and active involvement of patients in their own care. These physicians' practice style differences between initial and return patient visits suggest that physician-patient familiarity affects what happens during the medical interview.
Reprinted with permission by Blackwell Science, Inc.
Butterfield PS, Mazzaferri EL, Sachs LA. Nurses as evaluators of the humanistic behavior of internal medicine residents. J Med Educ 1987; 62:842-9. Abstract: The reliability of a 13-item questionnaire designed to assess the humanistic behaviors of internal medicine residents and the reliability of nurses as raters of those behaviors were examined. Twenty-five residents were evaluated by 10 or 11 nurses on two general medicine services and on cardiology and hematology-oncology services in a large, highly specialized department of internal medicine. Using an application of generalizability theory, which extends beyond classical test theory to establish estimates of multiple-error sources, the investigators calculated reliability-like coefficients for each of the services. The coefficients were .95 and .85 for the two general medicine services, .67 for cardiology, and .88 for hematology-oncology. These findings indicate that the questionnaire is a reliable instrument for assessing humanistic behavior and identifying reliable raters in groups of nurses.
Callahan EJ, Bertakis KD. Development and validation of the Davis Observation Code. Fam Med 1991;23:19-24. Abstract: Direct observation has demonstrated considerable power in the reliable and valid measurement of human behavior. A variety of direct methodologies have been applied to physician-patient interactions to answer different types of questions. This study describes the development and evaluation of a 20-item direct observation scale for physician-patient interactions, the Davis Observation Code (DOC). The study compared the rates of occurrence of four key physician behaviors measurable by both direct observation and chart audit: disease prevention, health education, health promotion, and compliance checking. Forty-nine videotaped physician-patient interactions were independently analyzed using the DOC. The medical record of each videotaped encounter was also reviewed. Reliability determined by inter- rater agreement regarding the presence/absence of each was acceptable for both direct observation and chart audit. Rates of occurrence of each target behavior differed between the two methods of review; chart audit consistently yielded lower rates. Nonparametric correlation analyses yielded phi values ranging from .12 to .49, suggesting low concurrent validity. Most of the discordance between the results of the videotaped observation and chart audit involved underreporting in the chart of observed behavior by the physician. Implications of the findings for health care delivery research are discussed.
Reprinted with Permission from the Society of Teachers of Family Medicine www.stfm.org
Coutts LC, Rogers JC. Humanism: is its evaluation captured in commonly used performance measures? Teach Learn Med 2000;12:28-32. Abstract: BACKGROUND: There is an increased awareness of the importance of humanistic behavior and its education in the medical school curriculum. Relatively little is known about correlations between humanism and other performance measures. PURPOSES: To determine the correlation between humanism and other commonly used performance measures, and to determine if more humanistic students perform better, the same, or worse than less humanistic students. METHODS: During the Family Medicine clerkship, standardized patients (SPs) used the Physician Humanism Scale to assess 428 students for humanism. Clinical preceptors, SPs, written assignments, and a national knowledge examination also assessed student performance. The humanism scores were correlated with the SP and non-SP performance measures. RESULTS: Humanism scores were significantly and positively correlated to all of the performance measures, but the correlation coefficients were low, ranging from .12 to .31. Students in the lowest quartile for humanism consistently scored lower for all of the performance measures, including both local and national exams (two-tailed significance < .018). Students with very low humanism scores still passed other performance evaluations. CONCLUSIONS: The correlation between humanism and other performance measures is quite low, indicating that a separate measure for humanism provides different and additional information that current performance measures do not include. More humanistic students perform better than their less humanistic peers, but current performance measures do not identify students with the lowest scores on humanistic behavior. This study supports the inclusion of humanism as an additional, independent performance measure.
de Monchy C, Richardson R, Brown RA, Harden RM. Measuring attitudes of doctors: the doctor-patient (DP) rating. Med Educ 1988;22:231-9. Abstract: An attitude scale was devised to discriminate between the extremes of doctor-centred, disease-oriented as opposed to patient-centred, problem- oriented (the DP scale). Four groups of subjects (214 in all) were tested with a Likert-type questionnaire based on this scale. Significant differences in attitude between the groups were found. Educational implications are discussed.
Reprinted with permission by Blackwell Science, Inc.
Finlay IG, Stott NC, Kinnersley P. The assessment of communication skills in palliative medicine: a comparison of the scores of examiners and simulated patients. Med Educ 1995;29:424-9. Abstract: The Diploma in Palliative Medicine was established in 1991 and included communication skills as a major part of the curriculum. In order to assess the efficacy of doctors' communication skills in the Diploma examination, an assessment tool was developed based on the modified Prevara Score. Simulated (actor) patients are used for the consultation in the examination; the doctors' performance is assessed independently by the examiner and by the actor. This provides an opportunity to consider the methods used for describing agreement between raters. There was high correlation between examiners' and actors' scores and high acceptability of the scoring method. However, satisfactory agreement in terms of the mean differences between scores and their standard deviation between examiners' and actors' scores was not achieved. We have found the simulated patient interview to be a useful teaching and assessment tool. The good correlation between the observer's (examiner) and the recipient's (actor-patient) perception of the doctor's interviewing skills provides evidence of the validity of the assessment. However the actor-patients' ratings tended to be higher and the two groups of assessors could not be used interchangeably. We conclude that actor-patients are of value in teaching and in assessing the communication skills of doctors but produce different scores to clinical examiners.
Reprinted with permission by Blackwell Science, Inc.
Flocke SA. Measuring attributes of primary care: development of a new instrument. J Fam Pract 1997;45(1):64-74. Abstract: BACKGROUND: The purpose of this study was to evaluate the measurement properties of an instrument developed to measure seven key aspects of the delivery of primary care from the perspective of patients visiting their family physician, and to report the association of these aspects with patient satisfaction. METHODS: A cross-sectional study design was used to examine the responses of 2899 patients visiting 138 family physicians' offices in Northeast Ohio. A 20-item research tool, the Components of Primary Care Index (CPCI), was created to measure the domains of primary care based on the new Institute of Medicine definition and on additional domains based on the literature. Patient satisfaction was measured with the Medical Outcomes Study 9-item visit rating form. The usual provider continuity (UPC) index was calculated as the proportion of visits to the index physician with relation to all physician visits for the past year by patient report. The CPCI was subjected to item and factor analysis. Scale scores were computed, and the association with patient satisfaction with the visit was tested by correlation. RESULTS: The factor analysis resulted in four stable and internally consistent factors. The factors were named: interpersonal communication, physician's accumulated knowledge of the patient, coordination of care, and patients' preference to see their regular physician. Each of the CPCI scale scores was significantly associated with patient satisfaction with the visit. The UPC index, length of time as a patient, and intensity of visits were not as strongly associated with the patient satisfaction measure. CONCLUSIONS: The CPCI provides a brief and reliable measure of four important aspects of the delivery of primary care. The components of primary care are associated with patient satisfaction with visits to family physicians. The CPCI could be used with other outcomes and to assess the effect of interventions and systems changes on the delivery of critical aspects of primary care.
Reprinted with permisson from Flocke SA, Measuring attributes of primary care new instruments, 45(1):64-74, ©1997 The Journal of Family Practice
Greco M, Francis W, Buckley J, Brownlea A, McGovern J. Real-patient evaluation of communication skills teaching for GP registrars. Fam Pract 1998;15:51-7. Abstract: BACKGROUND: Five thousand eight hundred and eighty-five patient- completed questionnaires were used to evaluate the effectiveness of an interpersonal skills module designed for a vocational training programme for GPs. OBJECTIVES: It was anticipated that patient-based assessments would detect a significant improvement in the interpersonal skills of GP Registrars who undertook the module. METHOD: A quasi- experimental design using an intervention and control group (comprising 68 GP Registrars) was used to monitor the outcomes of the interpersonal skills module. RESULTS: Patient ratings of interpersonal skills were significantly higher for those GP Registrars who participated in the interpersonal skills module. CONCLUSIONS: Patient-based assessments are a useful evaluation method for assessing the quality of the doctor- patient relationship.
Reprinted with permission from Greco M, Real patient evaluation of communication skils teaching for GP registrars', Fam Prac 1998, 15;(1):51-7, by Oxford University Press .
Hauck FR, Zyzanski SJ, Alemagno SA, Medalie JH. Patient perceptions of humanism in physicians: effects on positive health behaviors. Fam Med 1990; 22:447-52. Abstract: Although humanism has emerged as an important issue in medical education and practice, there is no standardized definition of humanism or an instrument that measures patients' perceptions of their physician's level of humanism. This study addressed these three issues: 1) A definition of physician humanism was developed based on the current literature; 2) an instrument was designed that measured patients' perceptions of humanism in their family physicians; and 3) health outcome variables were measured relative to these perceptions. The design was a cross-sectional survey of 185 randomly selected patients from two family practice sites. The questionnaire consisted of a humanism scale, in addition to items to assess patient satisfaction and patients' adherence to medical advice regarding exercise, diet, and smoking cessation. A positive association was found between perceived physician humanism and patient satisfaction. Greater success in patients' attempts to quit smoking was associated with higher physician humanism. The implications of these findings for evaluation and training of health care providers and patients' use of health care services are discussed.
Reprinted with Permission from the Society of Teachers of Family Medicine. www.stfm.org
Humphris GM, Kaney S. The Objective Structured Video Exam for assessment of communication skills. Med Educ 2000;34:939-45. Abstract: OBJECTIVES: (i) To design a new, quick and efficient method of assessing specific cognitive aspects of trainee clinical communication skills, to be known as the Objective Structured Video Exam (OSVE) (Study 1); (ii) to prepare a scoring scheme for markers (Study 2); and (iii) to determine reliability and evidence for validity of the OSVE (Study 3). METHODS: Study 1 describes how the exam was designed. The OSVE assesses the student's recognition and understanding of the consequences of various communication skills. In addition, the assessment taps the number of alternative skills that the student believes will be of assistance in improving the patient-doctor interaction. Study 2 outlines the scoring system that is based on a range of 50 marks. Study 3 reports inter-rater consistency and presents evidence to support the validity of the new assessment by associating the marks from 607 1st year undergraduate medical students with their performance ratings in a communication skills OSCE. SETTING: Medical school, The University of Liverpool. RESULTS: Preparation of a scoring scheme for the OSVE produced consistent marking. The reliability of the marking scheme was high (ICC=0.94). Evidence for the construct validity of the OSVE was found when a moderate predicted relationship of the OSVE to interviewing behaviour in the communication skills OSCE was shown (r=0.17, P < 0.001). CONCLUSION: A new video-based written examination (the OSVE) that is efficient and quick to administer was shown to be reliable and to demonstrate some evidence for validity.
Reprinted with permission by Blackwell Science, Inc.
Kaplan CB, Centor RM. The use of nurses to evaluate houseofficers' humanistic behavior. J Gen Intern Med 1990;5:410-14. Abstract: OBJECTIVES: 1) To determine whether nurse evaluations of humanistic behavior discriminate between houseofficers in an internal medicine training program, and 2) to compare nurse and attending physician evaluations. DESIGN: Prospective, six-month comparison of nurse and attending ratings of houseofficer humanistic behavior. PROCEDURE: Using a six-item, Likert-scale humanistic behavior rating form, nurses and ward attendings evaluated 76 PGY-1, PGY-2, and PGY-3 houseofficers over a six-month period. Nurses and attendings voluntarily evaluated houseofficers on all inpatient units in both university and Veterans Administration teaching hospitals. MEASUREMENTS AND MAIN RESULTS: Nurse ratings discriminated residents from one another throughout the six months of the study and over all units in both hospitals. Attending physician ratings were only moderately correlated with nurses' and were significantly more lenient. Exploratory analyses of the nursing evaluations revealed that female houseofficers received significantly more favorable evaluations than did men and that ward nurses were significantly more lenient than were critical care nurses. Nurse ratings did not differ by hospital, training year, or month of evaluation. CONCLUSIONS: Nurses can provide information about humanistic behavior that will allow program directors to discriminate among different levels of houseofficer behavior. Information from nurses differs from that provided by attending physicians. Nurse ratings are affected by gender and by the type of unit from which they are obtained.
Reprinted with permission by Blackwell Science, Inc.
Kurtz SM, Silverman JD. The Calgary-Cambridge Referenced Observation Guides: an aid to defining the curriculum and organizing the teaching in communication training programmes. Med Educ 1996;30:83-9. Abstract: Effective communication between doctor and patient is a core clinical skill. It is increasingly recognized that it should and can be taught with the same rigour as other basic medical sciences. To validate this teaching, it is important to define the content of communication training programmes by stating clearly what is to be learnt. We therefore describe a practical teaching tool, the Calgary-Cambridge Referenced Observation Guides, that delineates and structures the skills which aid doctor-patient communication. We provide detailed references to substantiate the research and theoretical basis of these individual skills. The guides form the foundation of a sound communication curriculum and are offered as a starting point for programme directors, facilitators and learners at all levels. We describe how these guides can also be used on an everyday basis to help facilitators teach and students learn within the experiential methodology that has been shown to be central to communication training. The learner-centred and opportunistic approach used in communication teaching makes it difficult for learners to piece together their evolving understanding of communication. The guides give practical help in countering this problem by providing: an easily accessible aide-memoire; a recording instrument that makes feedback more systematic; and an overall conceptual framework within which to organize the numerous skills that are discovered one by one as the communication curriculum unfolds.
Reprinted with permission by Blackwell Science, Inc.
Levinson W, Roter D. The effects of two continuing medical education programs on communication skills of practicing primary care physicians. J Gen Intern Med 1993;8:318-24. Abstract: PURPOSE: To evaluate and compare the effects of two types of continuing medical education (CME) programs on the communication skills of practicing primary care physicians. PARTICIPANTS: Fifty-three community- based general internists and family practitioners practicing in the Portland, Oregon, metropolitan area and 473 of their patients. METHOD: For the short program (a 4 1/2-hour workshop), 31 physicians were randomized to either the intervention or the control group. In the long program (a 2 1/2-day course), 20 physicians participated with no randomization. A research assistant visited all physicians' offices both one month before and one month after the CME program and audiotaped five sequential visits each time. Data were based on analysis of the content and the affect of the interviews, using the Roter Interactional Analysis Scheme. RESULTS: Based on both t-test analysis and analysis of covariance, no effect on communication was evident from the short program. The physicians enrolled in the long program asked more open-ended questions, more frequently asked patients' opinions, and gave more biomedical information than did the physicians in the short program. Patients of the physicians who attended the long program tended to disclose more biomedical and psychosocial information to their physicians. In addition, there was a decrease in negative affect for both patient and physician, and patients tended to demonstrate fewer signs of outward distress during the visit. CONCLUSION: This study demonstrates some potentially important changes in physicians' and patients' communication after a 2 1/2-day CME program. The changes demonstrated in both content and affect may have important influences on both biologic outcome and physician and patient satisfaction.
Reprinted with permission by Blackwell Science, Inc.
Linn LS, DiMatteo MR, Cope DW, Robbins A. Measuring physicians' humanistic attitudes, values, and behaviors. Med Care 1987;25:504-15. Abstract: This paper describes the reliability and validity of 10 easily administered and scored self-report measures of physicians' humanistic attitudes, values, and behaviors. This research also provides evidence that evaluations of physicians' humanistic behavior made by their outpatients, and non-physician staff with whom they worked, and the faculty physicians supervising them on inpatient ward rotations were positively and significantly correlated with one another. The potential usefulness of a multi-modal approach in evaluating humanistic physician attributes in which self-report measures are combined with direct feedback from all of those who interact with physicians is discussed.
Matthews DA, Feinstein AR. A new instrument for patients' ratings of physician performance in the hospital setting. J Gen Intern Med 1989;4:14-22. Abstract: A new instrument to elicit patients' appraisals of physician performance has been developed from a previously-derived taxonomy of desired physician attitudes and behavior. The instrument allows patients to give ratings for their physicians' discrete, observable items of behavior, and also for complex, multidimensional attributes. When the instrument was administered to 131 randomly chosen medical inpatients, the results showed that technical competence and interpersonal (or humanistic) qualities were equally valued, and that physicians received high ratings for most features of performance. Except for less satisfaction in younger patients, clinical and demographic characteristics had little impact on the ratings. The performance characteristics of the instrument appear satisfactory, and its potential applications and proposals for further research are discussed.
Reprinted with permission by Blackwell Science, Inc.
McLeod PJ, Tamblyn R, Benaroya S, Snell L. Faculty ratings of resident humanism predict patient satisfaction ratings in ambulatory medical clinics. J Gen Intern Med 1994;9:321-6. Abstract: OBJECTIVE: To determine whether patient satisfaction ratings can be predicted by faculty ratings or self-ratings of resident humanism. DESIGN: A prospective three-month collection of patient satisfaction ratings in two ambulatory care clinics and simultaneous acquisition of faculty ratings and self-ratings of resident humanism using ABIM questionnaires. SETTING: Two teaching hospital ambulatory care internal medicine clinics. PARTICIPANTS: Forty-seven internal medicine residents and 17 faculty internists were sent questionnaires for evaluation of humanism of individual residents. One thousand one hundred ninety-four consecutive outpatients cared for by the residents were eligible for patient satisfaction questionnaires. MEASUREMENTS AND MAIN RESULTS: Thirty-three residents and 13 faculty completed evaluations of resident humanism while 792 patients completed satisfaction questionnaires, which were used for analysis. The faculty ratings of resident humanism correlated strongly with patient satisfaction ratings, while the resident self-ratings did not. CONCLUSIONS: Faculty ratings of resident humanism were highly predictive of patient satisfaction with the care rendered by internal medicine residents in two ambulatory care clinics. This suggests that ambulatory care settings are useful for evaluation of noncognitive behavioral features of resident performance.
Reprinted with permission by Blackwell Science, Inc.
Pieters HM, Touw-Otten FW, De Melker RA. Simulated patients in assessing consultation skills of trainees in general practice vocational training: a validity study. Med Educ 1994; 28(3):226-233. Abstract: Although simulated patients are increasingly used in medical education, little research has been carried out on their validity. Validity in this case defines the relationship between performance with a simulated patient and performance with a real patient. One of the objectives of this study was to determine the validity of the use of simulated patients in assessing the consultation skills of trainees in vocational training at the Department of General Practice, University of Utrecht, The Netherlands. A check-list with a rating scale was used to assess the consultation skills of trainees at the department with simulated patients as well as in their training practices with real patients. The simulated and the selected practice cases were patients with complex multi-conditional problems like low back pain, headache and chest pain. The consultation skills were subdivided into four groups: the patient- centered approach, the non-somatic approach, communication skills and interpersonal skills. The measurement of skills, in particular of consultation skills, is very difficult. A description is given of the way the research group solved this problem. The analysis was performed by determining the sensitivity and predictive value of the assessment of a simulated encounter with a routine practice encounter. A difference existed in the assessed level of consultation skills in the simulated encounter compared to the level in the training practice. In simulation the level of consultation skills was higher than in day-to- day practice. This difference can reflect the difference between competence and performance. Competence is defined as what a doctor is capable of doing and performance as what a doctor actually does in day- to-day practice. This difference can reflext the difference between competence and performance Competence is defined as what a doctor is capable of doing and performance as what a doctor actually does in day-to-day practice. The assessment of behaviour in simulation reflected well those trainees who also performed indadequately with practice cases. The predictive value of inadequate performance in simulation is high. For feedback purposes this is an important issue. Because of the difference between competence (simutaltion) and performance ( real practice), the predicitive value of the behaviour rate adequate in simulation is not so high. Considering the difference between competence and performance, actor simulation is a valid method to assess consultation skills of trainees.
Reprinted with permission by Blackwell Science, Inc.
Roter DL, Hall JA, Kern DE, Barker LR, Cole KA, Roca RP. Improving physicians' interviewing skills and reducing patients' emotional distress. A randomized clinical trial. Arch Intern Med 1995;155:1877-84. Abstract: BACKGROUND: Despite high prevalence, emotional distress among primary care patients often goes unrecognized during routine medical encounters. OBJECTIVE: To explore the effect of communication-skills training on the process and outcome of care associated with patients' emotional distress. METHODS: A randomized, controlled field trial was conducted with 69 primary care physicians and 648 of their patients. Physicians were randomized to a no-training control group or one of two communication-skills training courses designed to help physicians address patients' emotional distress. The two training courses addressed communication through problem-defining skills or emotion- handling skills. All office visits of study physicians were audiotaped until five emotionally distressed and five nondistressed patients were enrolled based on patient response to the General Health Questionnaire. Physicians were also audiotaped interviewing a simulated patient to evaluate clinical proficiency. Telephone monitoring of distressed patients for utilization of medical services and General Health Questionnaire scores was conducted 2 weeks, 3 months, and 6 months after their audiotaped office visits. RESULTS: Audiotape analysis of actual and simulated patients showed that trained physicians used significantly more problem-defining and emotion-handling skills than did untrained physicians, without increasing the length of the visit. Trained physicians also reported more psychosocial problems, engaged in more strategies for managing emotional problems with actual patients, and scored higher in clinical proficiency with simulated patients. Patients of trained physicians reported reduction in emotional distress for as long as 6 months. CONCLUSIONS: Important changes in physicians' communication skills were evident after an 8-hour program. The training improved the process and outcome of care without lengthening the visits.
Reprinted with permission by Arch Intern Med 1995;155:1877-84, Copyrighted 1995 American Medical Association.
Simmons J, Roberge L, Kendrick SB, Jr., Richards B. The interpersonal relationship in clinical practice. The Barrett- Lennard Relationship Inventory as an assessment instrument. Eval Health Prof 1995;181:103-12. Abstract: The biomedical model that has long been central to medical practice is gradually being expanded to a broader biopsychosocial model. Relationship-building skills commensurate with the new paradigm need to be understood by educators and taught to medical practitioners. The person-centered, or humanistic, model of psychologist Carl Rogers provides a theoretical approach for the development of effective biopsychosocial relationships. The Barrett-Lennard Relationship Inventory (BLRI) was developed in 1962 as an assessment instrument for the person-centered model. In this article, the person-centered model and the use of the BLRI as an assessment instrument of this model are discussed. Current and potential uses of the BLRI are explored.
Tamblyn R, Benaroya S, Snell L, McLeod P, Schnarch B, Abrahamowicz M. The feasibility and value of using patient satisfaction ratings to evaluate internal medicine residents. J Gen Intern Med 1994;9:146-52. Abstract: OBJECTIVE: To determine the feasibility and value of using patient satisfaction ratings to evaluate the physician-patient relationship skills of medical residents. DESIGN: A cross-sectional survey was used to collect patient satisfaction information for medical residents from all patients attending the out-patient teaching clinics over a three- month period. The feasibility of patient rating was assessed by evaluating the reliability of resident satisfaction scores and potential sources of bias in rating. The value of using this method was assessed by evaluating the proportion of variance in ratings attributable to residents, the prevalence of residents with ratings substantially below average, and the effect of training on patient satisfaction. SETTING: The medical clinics of two teaching hospitals at McGill University. PARTICIPANTS: 91 medical residents and clinical clerks, 1,003 patients, and 1,219 visit ratings. RESULTS: An average of 12.2 satisfaction ratings were collected per resident, providing a reliability of 0.56 (intraclass correlation) for resident satisfaction score. Adjustment for differences in patient populations would be necessary to obtain comparable scores among residents. Fifteen percent of the residents had a substantially greater proportion of poor and fair satisfaction ratings than the study average. Residents were responsible for a substantial proportion of the variance in clinic waiting time and patient satisfaction rating. Training appeared to have no effect on improving satisfaction scores, except among women residents. CONCLUSIONS: Patient satisfaction ratings provide valuable information about a resident's ability to establish an effective physician-patient relationship. However, the number of ratings required to obtain a reliable estimate of resident skill may limit the feasibility of using patient ratings as part of residency evaluation.
Reprinted with permission by Blackwell Science, Inc.
Tate P, Foulkes J, Neighbour R, Campion P, Field S. Assessing physicians' interpersonal skills via videotaped encounters: a new approach for the Royal College of General Practitioners Membership examination. J Health Commun 1999;4:143-52. Abstract: The Royal College of General Practitioners' Membership examination, the only postgraduate qualification in family medicine in the United Kingdom, has developed a direct assessment of candidates' interpersonal skills performance using videotaped consultations of the actual doctor- patient encounters. At present about 1,200 doctors are examined each year. The methodology has been developed and piloted over a period of eight years. The central tenet of the methodology is a clear definition, which is known both to the candidate and to the examiner, of the clinical and consulting competencies that are required to be demonstrated in order to pass the examination. The candidate is required to provide evidence of his or her competence usually by selecting appropriate patient encounters that demonstrate the fulfillment of the required performance criteria, effectively producing a portfolio of his or her communicative competence. The methodology is intended to encourage the learning and teaching of communication skills by making it part of an important examination and clearly defining the competencies required to pass. Reliability has been demonstrated to be satisfactory and refinement of the marking processes is likely to improve this further.
Weaver MJ, Ow CL, Walker DJ, Degenhardt EF. A questionnaire for patients' evaluations of their physicians' humanistic behaviors. J Gen Intern Med 1993;8:135-9. Abstract: OBJECTIVES: To determine what behaviors patients perceive as reflecting a physician's humanistic qualities, to develop an instrument for patients to use to assess the humanistic behaviors of their own physicians, and to compare patient assessment of residents' humanistic behaviors with patient satisfaction and the assessment of attending physicians. DESIGN: Cross-sectional descriptive study, using patient interviews and questionnaires, and evaluations of residents by attending physicians. SETTING: Inpatient medical service in a tertiary care teaching hospital and in a primary care internal medicine clinic. PARTICIPANTS: Six medical interns and six medical residents, 119 medical patients in the hospital, and 111 patients in the internal medicine clinic. MEASUREMENTS AND MAIN RESULTS: The 25-item Physicians' Humanistic Behaviors Questionnaire (PHBQ) was developed from patients' statements about important humanistic behaviors. The mean PHBQ scores were 4.46 +/- 0.22 (mean +/- SD, on a scale of 1 to 5) in the clinic and 4.18 +/- 18 in the hospital (p = 0.003). The Spearman's rank correlations between the PHBQ and the Medical Interview Satisfaction Scale (MISS) were r = 0.8741 (p < 0.001) in the hospital and r = 0.8751 (p < 0.001) in the internal medicine clinic. The Spearman's rank correlation between the hospital PHBQ and the attending physician evaluations (for the six residents for whom the authors had complete data) was r = 0.5753 (p = 0.232). CONCLUSIONS: Patients can evaluate the humanistic behaviors of their physicians using the PHBQ. There is good correlation between the PHBQ and patient satisfaction, which supports the validity of the PHBQ. The relative lack of agreement between patients and attending physicians suggests different observations, criteria, or standards. The higher ratings from patients in the clinic compared with those from patients in the hospital suggest that residents' behaviors are different or that patients have different observations, criteria, or standards in the two settings. Therefore, a complete assessment of residents' humanistic behaviors may require sampling in both settings.
Reprinted with permission by Blackwell Science, Inc.
Winefield HR, Chur-Hansen A. Evaluating the outcome of communication skill teaching for entry-level medical students: does knowledge of empathy increase? Med Educ 2000;34:90-4. Abstract: BACKGROUND: While the literature shows the clinical value for medical practitioners of skill in communicating with patients in an empathetic manner, objective evaluations of methods to teach empathy are few. PURPOSES: This paper describes a method of teaching entry-level medical students the elements of effective communication with patients, in preparation for their first practical exercises. The paper focuses on how the outcomes of the teaching were evaluated with special attention to empathy. METHODS: Student evaluative ratings were collected after training, and students also completed a pencil-and-paper test of empathy, both before and after the training. While all data were anonymous, student pre- and post-training empathy scores could be compared to assess individual changes in knowledge of empathy after training. RESULTS: Most students (81%) felt better prepared to interview after the training. The pencil-and-paper measure of empathy has good reliability, both internal (alpha 0.83 and 0.91) and inter- rater (kappa 0.96). Overall, students made significant gains in their ability to make empathetic responses, although some (30%) showed no gains. CONCLUSIONS: Further research is required to identify students who fail to acquire skill in expressing empathy after undergoing training, and to validate the pencil-and- paper measure of empathy against real-life performance.
Reprinted with permission by Blackwell Science, Inc.

Professionalism

Barry D, Cyran E, Anderson RJ. Common issues in medical professionalism: room to grow. Am J Med 2000;108:136-42. Abstract: PURPOSE: Much of the respect and trust that society grants to physicians is based on the expectation of upholding professional values. We performed this study to assess responses to common challenges to medical professionalism and to ascertain physician satisfaction with training in professionalism. METHODS: A self- administered questionnaire containing six challenges to professionalism (acceptance of gifts, conflict of interest, confidentiality, physician impairment, sexual harassment, and honesty) with multiple-choice responses was mailed to 528 medical students and 779 house officers at the University of Colorado Health Sciences Center and to 900 randomly selected Colorado physicians. Information about previous exposure to the issue depicted in the scenarios and satisfaction with, and site of, previous education in medical professionalism was obtained. An independent panel selected the best or acceptable answers to the challenges. RESULTS: In all, 961 evaluable responses were received. More than 40% of physicians reported experience with four of the six challenges. The frequency of the best or acceptable answers to the six scenarios ranged from 12% to 86%. Best or acceptable responses were more common in physicians than in house officers, and in house officers than in medical students (P < 0.001). Practice setting and specialty type had only modest effects. The physician impairment scenario was the most challenging: Only 12% of physicians provided the best answer. Most (73%) respondents reported having 10 or fewer hours of formal course work in professionalism, and many (40%) were dissatisfied with their training in professionalism. CONCLUSIONS: While everyday challenges to professionalism are commonly encountered by trainees and practicing physicians, many practitioners are dissatisfied with their training in this area and were unable to provide an acceptable answer to these challenges.
Reprinted with permission by Elseiver Science, Inc.
Beckman H, Frankel R, Kihm J, Kulesza G, Geheb M. Measurement and improvement of humanistic skills in first-year trainees. J Gen Intern Med 1990;5:42-5. Abstract: The American Board of Internal Medicine (ABIM) has recently emphasized the development of humanistic skills in trainees. Using video technology, transition outpatient visits of first-year house officers in a primary care training program were evaluated for the presence or absence of nine humanistic skills before and after the initiation of an instructional program to reinforce the skills. Thirteen videotaped PGY- 1 encounters constituted the preintervention group and 16 videotaped PGY-1 encounters constituted the postintervention group. The preintervention group performed a mean of 1.38 skills while the postintervention group performed a mean of 3.56 skills, a statistically significant improvement (p less than 0.05). The authors conclude that an educational approach that focuses on specific elements of interactions facilitates the incorporation of skills associated with humane medical care.
Reprinted with permission by Blackwell Science, Inc.
Butterfield PS, Mazzaferri EL. A new rating form for use by nurses in assessing residents' humanistic behavior [see comments]. J Gen Intern Med 1991;6:155-61. Abstract: OBJECTIVE: To determine the reliability and validity of an evaluation form for assessing the humanistic behavior of internal medicine (IM) housestaff. The form is for use by nurses. DESIGN: Evaluations were gathered three times during the 1987-88 academic year. Generalizability coefficients (interpreted like traditional reliability coefficients) were generated to establish the form's reliability, while data from attending physicians and from housestaff evaluation committee members were used to help establish its validity. SETTING: Three hospitals in central Ohio: a large university tertiary care center, a large private hospital, and an urban community hospital. PARTICIPANTS: The nurse raters were volunteers solicited by their head nurses. The criteria governing their participation were two years of postgraduate experience in nursing and regular contact with residents, which was self- determined. All IM residents who had worked on a medicine inpatient service at least once during the months under study were included. A total of 493 nurses and 116 residents participated. RESULTS: Sixty-four percent of the generalizability coefficients were 0.90 or higher, and 82% were above 0.75, indicating stable, reliable ratings. The nurses' ratings were positively and significantly correlated with attending faculty's and evaluation committee members' ratings (r = 0.38, p less than 0.01; r = 0.49, p less than 0.001). CONCLUSIONS: The evaluation form and the nurses provided consistent, reliable information about medical residents' humanistic behavior; data from five to six nurses should provide statistically reliable ratings using this form. Also, nurses' data yielded information somewhat different from those provided by physicians, suggesting that the form is a useful instrument for assessing this dimension of residents' performance.
Reprinted with permission by Blackwell Science, Inc.
Culhane-Pera KA, Reif C, Egli E, Baker NJ, Kassekert R. A curriculum for multicultural education in family medicine. Fam Med 1997;29:719-23. Abstract: BACKGROUND AND OBJECTIVES: To deliver effective medical care to patients from all cultural backgrounds, family physicians need to be culturally sensitive and culturally competent. Our department implemented and evaluated a 3-year curriculum to increase residents' knowledge, skills, and attitudes in multicultural medicine. Our three curricular goals were to increase self-awareness about cultural influences on physicians, increase awareness about cultural influences on patients, and improve multicultural communication in clinical settings. Curricular objectives were arranged into five levels of cultural competence. Content was presented in didactic sessions, clinical settings, and community medicine projects. METHODS AND RESULTS: Residents did self-assessments at the beginning of the second year and at the end of the third year of the curriculum about their achievement and their level of cultural competence. Faculty's evaluations of residents' levels of cultural competence correlated significantly with the residents' final self-evaluations. Residents and faculty rated the overall curriculum as 4.26 on a 5-point scale (with 5 as the highest rating). CONCLUSIONS: Family practice residents' cultural knowledge, cross-cultural communication skills, and level of cultural competence increased significantly after participating in a multicultural curriculum.
Reprinted with Permission from the Society of Teachers of Family Medicine www.stfm.org
Dogra N, Stretch D. Developing a questionnaire to assess student awareness of the need to be culturally aware in clinical practice. Med Teach 2001;23:59-64. Abstract: This study aimed to establish whether students had an awareness of the requirement to consider cultural issues in caring for patients and to identify those issues which are most difficult for students, in order to aid course development. Data was collected using previously developed questionnaires which were modified and added to for the study undertaken at the University of Leicester Medical School in February and June 1998. Thirty fourth year and 75 second year medical students completed the questionnaires. Students accepted that they as doctors have a responsibility to be aware of the different cultures within their practice. Their responses supported the introduction of sessions on cultural and racial awareness as part of the Human Diversity Module. The sessions have been designed to facilitate students to explore their own perception and understanding of culture in a challenging, relevant and enjoyable way. The questionnaire developed in this study will be used to compare student attitudes pre- and post-teaching sessions to assess any change.
Reprinted with permission by Taylor and Francis, Inc.
Feldman MD, Zhang J, Cummings SR. Chinese and U.S. internists adhere to different ethical standards. J Gen Intern Med 1999;14:469-73. Abstract: OBJECTIVE: To determine whether internists in the United States and China have different ideas and behaviors regarding informing patients of terminal diagnoses and HIV/AIDS, the role of the family in end-of- life decision making, and assisted suicide. DESIGN: Structured questionnaire of clinical vignettes followed by multiple choice questions. SETTING: University and community hospitals in San Francisco and Beijing, China. SUBJECTS: Forty practicing internists were interviewed, 20 in China and 20 in the United States. MEASUREMENTS AND MAIN RESULTS: Of the internists surveyed, 95% of the U.S. internists and none of the Chinese internists would inform a patient with cancer of her diagnosis. However, 100% of U.S. and 90% of Chinese internists would tell a terminally ill patient who had AIDS, rather than advanced cancer, about his diagnosis. When family members' wishes conflicted with a patient's preferences regarding chemotherapy of advanced cancer, Chinese internists were more likely to follow the family's preferences rather than the patient's preferences (65%) than were the U.S. internists (5%). Thirty percent of U.S. internists and 15% of Chinese internists agreed with a terminally ill patient's request for sufficient narcotics to end her life. CONCLUSIONS: We found significant differences in clinical ethical beliefs between internists in the United States and China, most evident in informing patients of a cancer diagnosis. In general, the Chinese physicians appeared to give far greater weight to family preferences in medical decision making than did the U.S. physicians.
Reprinted with permission by Blackwell Science, Inc.
Green MJ, Mitchell G, Stocking CB, Cassel CK, Siegler M. Do actions reported by physicians in training conflict with consensus guidelines on ethics? Arch Intern Med 1996;156:298-304. Abstract: OBJECTIVE: To assess the extent to which actions reported by internal medicine trainees conflict with published guidelines on ethics. METHODS: A confidential survey was sent to a random sample (N = 1000) of associate members of the American College of Physicians (ACP). Questions were asked about ethical decision making in areas addressed by the guidelines in the ACP Ethics Manual. Quoted manual guidelines were provided, followed by 55 yes or no questions, such that a yes answer represented an action that conflicted with a guideline. There were two follow-up mailings to nonresponders. RESULTS: Forty percent (n = 397) completed the questionnaire; 17% indicated they were aware of the guidelines on ethics. On average, associates responded yes to 16% of questions where a yes response indicated they have acted outside guidelines on ethics one or more times. The mean number of responses (n = 55) that conflicted with a guideline was 7.6 per person (SD, 4.7 responses; range, 0 to 33 responses). Ninety-eight percent of respondents reported actions falling outside a guideline one or more times and 80% did so four or more times. The most frequently reported reason (965/3219 [30%]) from a list of four choices for acting outside a guideline was "I was aware of the guideline, but this did not represent an ethical dilemma to me." CONCLUSIONS: Few responding ACP associates indicated awareness of the ACP guidelines on ethics. Physicians in training nevertheless reported acting according to the presented guidelines most of the time, although nearly all respondents acted outside a guideline at least once, and some did so many times. Reported behaviors were sometimes inconsistent with consensus ethical standards that apply to internists. Physicians in training need to know more about ethical standards that apply to their own practice and should be aware when their actions deviate from ethical norms. Before acting outside guidelines on ethics, trainees should discuss their conflicts with others, such as attending physicians, clinical ethicists, or hospital ethics committees.
Reprinted with permission by Arch Intern Med 1996;156:298-304, Copyrighted 1996 American Medical Association
Green MJ, Farber NJ, Ubel PA, Mauger DT, Aboff BM, Sosman JM et al. Lying to each other: when internal medicine residents use deception with their colleagues. Arch Intern Med 2000;160:2317-23. Abstract: BACKGROUND: While lying is morally problematic, physicians have been known to use deception with their patients and with third parties. Little is known, however, about the use of deception between physicians. OBJECTIVES: To determine the likelihood that resident physicians say they would deceive other physicians in various circumstances and to examine how variations in circumstances affect the likelihood of using deception. METHODS: Two versions of a confidential survey using vignettes were randomly distributed to all internal medicine residents at 4 teaching hospitals in 1998. Survey versions differed by introducing slight variations to each vignette in ways we hypothesized would influence respondents' willingness to deceive. The likelihood that residents say they would use deception in response to each vignette was compared between versions. RESULTS: Three hundred thirty surveys were distributed (response rate, 67%). Of those who responded, 36% indicated they were likely to use deception to avoid exchanging call, 15% would misrepresent a diagnosis in a medical record to protect patient privacy, 14% would fabricate a laboratory value to an attending physician, 6% would substitute their own urine in a drug test to protect a colleague, and 5% would lie about checking a patient's stool for blood to cover up a medical mistake. For some of the scenarios, the likelihood of deceiving was influenced by variations in the vignettes. CONCLUSIONS: A substantial percentage of internal medicine residents report they would deceive a colleague in various circumstances, and the likelihood of using deception depends on the context. While lying about clinical issues is not common, it is troubling when it occurs at any time. Medical educators should be aware of circumstances in which residents are likely to deceive, and discuss ways to eliminate incentives to lie.
Reprinted with permission by Arch Intern Med 2000;160:2317-23, Copyrighted 2000 American Medical Association
Hebert PC, Meslin EM, Dunn EV. Measuring the ethical sensitivity of medical students: a study at the University of Toronto. J Med Ethics 1992; 18:142-7. Abstract: An instrument to assess 'ethical sensitivity' has been developed. The instrument presents four clinical vignettes and the respondent is asked to list the ethical issues related to each vignette. The responses are classified, post hoc, into the domains of autonomy, beneficence and justice. This instrument was used in 1990 to assess the ethical sensitivity of students in all four medical classes at the University of Toronto. Ethical sensitivity, as measured by this instrument, is not related to age or grade-point average. Sensitivity increases between the 1st and 2nd year and then decreases throughout the rest of undergraduate medical training, such that the 4th-year students identify fewer issues than those entering medical school. Students expressing a career choice of family medicine identify more issues than their peers. Several problems with the use of the instrument and the interpretation of the data were found. Nonetheless, these findings, if reproducible, are important and their meaning needs further discussion.
Klessig J, Robbins AS, Wieland D, Rubenstein L. Evaluating humanistic attributes of internal medicine residents. J Gen Intern Med 1989;4:514-21. Abstract: OBJECTIVE: Methods of assessing humanism in internal medicine residents have not been completely designed or evaluated. This study used patient satisfaction as a measure of humanism, and assessed the validity of using faculty physicians to evaluate residents' humanistic behavior. Residents' ability to assess themselves was also evaluated. SETTING: A university-affiliated internal medicine training program. SUBJECTS: Forty-seven internal medicine residents were evaluated by patients, faculty, and themselves. DESIGN: Faculty physicians were given standard faculty evaluation and patient satisfaction forms, and were asked to evaluate residents. These evaluations were compared with the patients' responses on the same satisfaction forms. Residents performed self- assessment using identical forms; these responses were compared with those of the faculty and patients. RESULTS: There was no correlation between patients' responses and those of the faculty or residents. There was a significant inverse correlation between resident and faculty responses, especially for the female residents (r = 0.71). CONCLUSION: These findings suggest the need for further study of the evaluation process, including what factors influence individuals to respond as they do. It appears that the use of one rating group is not sufficient to achieve an accurate assessment of residents' humanistic skills. The present status of the process of evaluating humanism is discussed.
Reprinted with permission by Blackwell Science, Inc.
Linn BS, Arostegui M, Zeppa R. Performance rating scale for peer and self assessment. Br J Med Educ 1975;9:98-101. Abstract: A performance rating scale was developed and tested on a class of junior medical students who rated themselves and four to ten of their peers. When 928 ratings were factor analysed, two strong factors, knowledge and relationship, emerged. Test-retest reliabilities were good. Validity was measured by correlation of ratings with grades, and though both sources of ratings correlated significantly with grades given by faculty, peer ratings were more highly related to grades than were self ratings. Students tended to rate themselves lower than they were rated by their peers. Grades are probably not the best estimate of performance, but are currently one of the most reliable. Use of the scale to judge performance of physicians in practice has not been tested. The question of how such evaluation of peer and self would relate to other measures of quality of care is raised.
Norman GR, Davis DA, Lamb S, Hanna E, Caulford P, Kaigas T. Competency assessment of primary care physicians as part of a peer review program [published erratum appears in JAMA 1994 Jan 12;271(2):106]. JAMA 1993; 270:1046-51. Abstract: OBJECTIVE--To design and test a program that assesses clinical competence as a second stage in a peer review process and to determine the program's reliability. DESIGN AND SETTING--A three-cohort study of Ontario primary care physicians. PARTICIPANTS--Reference physicians (n = 26) randomly drawn from the Hamilton, Ontario, area; volunteer, self- referred physicians (n = 20); and physicians referred by the licensing body (n = 37) as a result of a disciplinary hearing or peer review. MAIN OUTCOME MEASURES--Standardized patients, structured oral examinations, chart-stimulated recall, objective structured clinical examination, and multiple-choice examination. RESULTS--Test reliability was high, ranging from 0.73 to 0.91, and all tests discriminated among subgroups. Demographic variables relating to the final category were age, Canadian or foreign graduates, and whether or not participants were certified in family medicine. CONCLUSIONS--The study demonstrated the feasibility, reliability, and validity of a multicomponent examination in the peer review process.
Reprinted with permission by JAMA 1993; 270:1046-51, Copyrighted 1993 American Medical Association.
Ramsey PG, Wenrich MD, Carline JD, Inui TS, Larson EB, LoGerfo JP. Use of peer ratings to evaluate physician performance. JAMA 1993;269:1655-60. Abstract: OBJECTIVE--To assess the feasibility and measurement characteristics of ratings completed by professional associates to evaluate the performance of practicing physicians. DESIGN--The clinical performance of physicians was evaluated using written questionnaires mailed to professional associates (physicians and nurses). Physician-associates were randomly selected from lists provided by both the subjects and medical supervisors, and detailed information was collected concerning the professional and social relationships between the associate and the subject. Responses were analyzed to determine factors that affect ratings and measurement characteristics of peer ratings. SETTING AND PARTICIPANTS--Physician-subjects were selected from among practicing internists in New York, New Jersey, and Pennsylvania who received American Board of Internal Medicine certification 5 to 15 years previously. MAIN OUTCOME MEASURE--Physician performance as assessed by peers. RESULTS--Peer ratings are not biased substantially by the method of selection of the peers or the relationship between the rater and the subject. Factor analyses suggest a two-dimensional conceptualization of clinical skills: one factor represents cognitive and clinical management skills and the other factor represents humanistic qualities and management of psychosocial aspects of illness. Ratings from 11 peer physicians are needed to provide a reliable assessment in these two areas. CONCLUSIONS--These findings suggest that it is feasible to obtain assessments from professional associates of practicing physicians in areas such as clinical skills, humanistic qualities, and communication skills. Using a shorter version of the questionnaire used in this study, peer ratings provide a practical method to assess clinical performance in areas such as humanistic qualities and communication skills that are difficult to assess with other measures.
Reprinted with permission by JAMA 1993;269:1655-60, Copyrighted 1993 American Medical Association.
Rezler AG, Schwartz RL, Obenshain SS, Lambert P, Gibson JM, Bennahum DA. Assessment of ethical decisions and values. Med Educ 1992;26:7-16. Abstract: The development and pilot testing of the Professional Decisions and Values Test (PDV) is described. The PDV is designed to assess how ethical conflicts are dealt with by medical and law students and which moral values motivate them. Data from two consecutive classes of entering medical and law students are presented and their action tendencies and ethical values are compared. The findings support the construct validity of the test. Regarding reliability, stability over time is present for action tendencies but not for values. Perhaps the ethical values of entering medical and law students do not become stable until later. Change in ethical values can be studied with the PDV for groups, not individuals, during the first year of professional education.
Reprinted with permission by Blackwell Science, Inc.
Roter DL, Larson S, Fischer GS, Arnold RM, Tulsky JA. Experts practice what they preach: A descriptive study of best and normative practices in end-of-life discussions. Arch Intern Med 2000;160:3477-85. Abstract: BACKGROUND: Advance directives (ADs) are widely regarded as the best available mechanism to ensure that patients' wishes about medical treatment at the end of life are respected. However, observational studies suggest that these discussions often fail to meet their stated goals. OBJECTIVES: To explore best practices by describing what physicians who are considered expert in the area of end of-life bioethics or medical communication do when discussing ADs with their patients and to explore the ways in which best practices of the expert group might differ in content or style from normative practice derived from primary care physicians' discussions of ADs with their patients collected as part of an earlier study. DESIGN: Nonexperimental, descriptive study of audiotaped discussions. SETTING: Outpatient primary care practices in the United States. PARTICIPANTS: Eighteen internists who have published articles in the areas of bioethics or communication and 48 of their patients. Fifty-six academic internists and 56 of their established patients in 5 practice sites in 2 locations- Durham, NC, and Pittsburgh, Pa. Eligible patients were at least 65 years old or suffered from serious medical illness and had not previously discussed ADs with their physician. Expert clinicians had discretion regarding patient selection, while the internists chose patients according to a predetermined protocol. MEASUREMENTS: Coders applied the Roter Interaction Analysis System (RIAS) to audiotapes of the medical visits to describe communication dynamics. In addition, the audiotapes were scored on 21 items reflecting physician performance in specific skills related to AD discussions. RESULTS: Experts spent close to twice as much time (14.7 vs 8.1 minutes, P<.001) and were less verbally dominant (P<.05) than other physicians during AD discussions. When length of visit was controlled statistically, the expert physicians gave less information about treatment procedures and biomedical issues (P<.05) and asked fewer related questions (P<. 05) but tended toward more psychosocial and lifestyle discussion and questions. Experts engaged in more partnership building (P<.05) with their patients. Patients of the expert physicians engaged in more psychosocial and lifestyle discussion (P<.001), and more positive talk (P<.05) than patients of community physicians. Expert physicians scored higher on the 21 items reflecting AD-specific skills (P<.001). CONCLUSIONS: Best practices as reflected in the performance of expert physicians reflect differences in measures of communication style and in specific AD-related proficiencies. Physician training in ADs must be broad enough to include both of these domains. Arch Intern Med. 2000;160:3477-85.
Reprinted with permission by Arch Intern Med. 2000;160:3477-85, Copyrighted 2000 American Medical Association.
Singer PA, Cohen R, Robb A, Rothman A. The ethics objective structured clinical examination. J Gen Intern Med 1993;8:23-8. Abstract: OBJECTIVE: To develop objective structured clinical examination (OSCE) stations to assess the ability of physicians to address selected clinical-ethical situations, and to evaluate inter-rater agreement in these stations. DESIGN: Two ten-minute OSCE stations were developed using video-taped encounters between attending physicians and standardized patients. One scenario involved a daughter requesting a do- not-resuscitate (DNR) order for her competent mother without the mother's knowledge; the other involved a competent elderly woman requesting not to be re-intubated if her congestive heart failure worsened. The scenarios were evaluated using foreign medical graduates taking an OSCE. Each candidate was scored on his or her interaction with a standardized patient in the two OSCE stations by two independent observers. PARTICIPANTS: Eight attending physicians from the Division of General Internal Medicine at the Toronto Hospital were used to develop the OSCE stations, and 69 foreign medical graduates taking the University of Toronto Pre-Internship Program OSCE were used to evaluate the stations. RESULTS: The inter-rater reliability coefficients for the DNR and intubation scenarios were 0.79 (95% CI 0.69-0.87) and 0.75 (95% CI 0.62-0.84), respectively. For the DNR station, the scores of the two examiners, on a scale of 0 to 10, agreed exactly for 34 candidates (50%), within one mark for 59 candidates (87%), and within two marks for 65 candidates (96%). For the intubation station, the scores of the two examiners agreed exactly for 27 candidates (40%), within one mark for 56 candidates (84%), and within two marks for 63 candidates (94%). CONCLUSIONS: The authors produced ethics OSCE stations with face and content validity and satisfactory inter-rater agreement. Ethics OSCE stations may be suitable for evaluating the ability of medical students and residents to address selected clinical-ethical situations.
Reprinted with permission by Blackwell Science, Inc.
Singh SP, Baxter H, Standen P, Duggan C. Changing the attitudes of 'tomorrow's doctors' towards mental illness and psychiatry: a comparison of two teaching methods. Med Educ 1998;32:115-20. Abstract: The General Medical Council's document 'Tomorrow's Doctors' (1993, GMC, London) recommended major changes in the undergraduate curricula of UK medical schools. In Nottingham, the fourth-year psychiatric attachment became shorter in duration, and interactive, problem-oriented, workshop- based learning replaced lectures. We compared the efficacy of this new teaching style in changing medical students' attitudes towards psychiatry and mental illness with that of old-style, didactic, lecture- based teaching. On the first and last days of their psychiatric attachment, 110 fourth-year-medical students (45 old curriculum; 65 new curriculum) completed two self-administered attitudinal measures: the Attitude to Psychiatry Questionnaire (ATP-30) and the Attitude to Mental Illness Questionnaire (AMI). We found that students had favorable attitudes towards psychiatry and mental illness before the attachment. These attitudes became more positive after the attachment in students from both curricula, with no significant difference between the groups and no gender difference. Students found patient contact rewarding, become more accepting of community care, and had greater appreciation of the therapeutic potential of psychiatric interventions. The interactive, student-centred, problem-oriented teaching of the shortened new curriculum appeared as effective in changing medical student' attitudes as a longer attachment with traditional teaching.
Reprinted with permission by Blackwell Science, Inc.
Sulmasy DP, Geller G, Levine DM, Faden R. Medical house officers' knowledge, attitudes, and confidence regarding medical ethics. Arch Intern Med 1990;150:2509-13. Abstract: As part of a trial of ethics education in a university-based, categorical, internal medicine training program, we surveyed all medical house officers at our institution regarding their knowledge of medical ethics, their attitudes and beliefs about selected issues in medical ethics, and their confidence in dealing with ethical problems. In a multivariate linear regression model, house officer knowledge scores were negatively correlated with postgraduate year, and positively correlated with age and with reporting a Jewish religious identity. A multivariate linear regression model predicting house officer confidence in dealing with ethical issues revealed a positive correlation with self-reported quality of ethics training in medical school and with being in the experimental group of house officers receiving ethics education. Attitudes and beliefs were largely uncorrelated with training or demographic characteristics. These results have implications for ethics education of both medical students and residents.
Reprinted with permission by Arch Intern Med 1990;150:2509-13, Copyrighted 1990 American Medical Association.