Mini S MD 6 CEX Guidelines

User Manual: Mini s MD-6

Open the PDF directly: View PDF PDF.
Page Count: 24

DownloadMini S MD-6 Mini-CEX Guidelines
Open PDF In BrowserView PDF
The Mini-CEX
A Quality Tool
In Evaluation
Guidelines and Implementation Strategies
from Program Directors

WORK-IN-PROGRESS

American Board of Internal Medicine
Clinical Competence Program
September 2001 – June 2002

Visit the Mini-CEX page on the ABIM website

and contribute your comments and experiences.

Also visit  to learn more about the ACGME Outcome Project
and  to learn more about related initiatives underway by
program directors in internal medicine.

If you have any questions or need Mini-CEX Forms
and/or Guidelines, please contact:
ABIM Clinical Competence Program
510 Walnut Street, Suite 1700
Philadelphia, PA 19106-3699
(215) 446-3469
email: smccrea@abim.org

Contents
The Mini-CEX Project: List and Contact Information on Participating Programs . . . . . . . . . . . . . . . . . . . . . . . . . . .i

Report in Brief:
The Mini-Clinical Evaluation Exercise Pilot Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

Guidelines for Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

Pilot Programs: Experiences in Brief with Mini-CEXs
Abington Memorial Hospital, Abington, Pennsylvania — David Smith, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Easton Hospital, Easton, Pennsylvania — David Kemp, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
Elmhurst Hospital Center/Mt. Sinai School of Medicine, New York — Rand David, MD . . . . . . . . . . . . . . . . . . . . . .6
Englewood Hospital and Medical Center, New Jersey — Steven Reichert, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6
George Washington University Medical Center, Washington, DC — Dragica Mrkoci, MD . . . . . . . . . . . . . . . . . . . .7
Howard University, Washington, DC — Sheik Hassan, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Jefferson Medical College, Philadelphia — Gregory Kane, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
Monmouth Medical Center, New Jersey — Sara Wallach, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8
National Naval Medical Center, Bethesda — Eric Holmboe, MD (currently at Waterbury Hospital) . . . . . . . . . . . . .9
Penn State Milton S. Hershey Medical Center, Hershey — Edward Bollard, MD and Richard Simons, MD . . . . . . .9
Pennsylvania Hospital, Philadelphia — Michael Buckley, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Robert Wood Johnson Medical School, New Brunswick — Nayan Kothari, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Seton Hall University School of Graduate Medical Education, New Jersey — William Farrer, MD . . . . . . . . . . . . .10
St. Vincent's Catholic Medical Center of New York, Staten Island Region
— Susan Grossman, MD and Cynthia Wong, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11
SUNY Downstate, Brooklyn — Jeanne Macrae, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
Temple University Hospitals, Philadelphia — Brenda Horwitz, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12
University Health Center of Pittsburgh — Frank Kroboth, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
University of Texas Medical Branch, Galveston — Stephen Sibbitt, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13
VA Medical Center, New York — Richard Rees, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Washington Hospital Center, Washington, DC — Frederick Williams, MD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14
Yale Primary Care Internal Medicine Residency, New Haven — Stephen Huot, MD, PhD . . . . . . . . . . . . . . . . . . . .15

Examples of Memos to Faculty and Residents on Mini-CEXs and Mini-CEX Form . . . . . . . . . . . . . . . . . . . . . . . .16

Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20

PARTICIPATING PROGRAMS - ABIM MINI-CEX PILOT
For more information on how programs implemented mini-CEXs, please contact any of the participants in the pilot.

Edward Bollard, MD
Penn State College of Medicine
Milton S. Hershey Medical Center, Hershey
Tel:
(717) 531-8390
Email: ebollard@psu.edu

Frank Kroboth, MD
University Health Center of Pittsburgh
Montefiore University Hospital
Tel:
(414) 692-4882
Email: krobothf2@msx.upmc.edu

Michael Buckley, MD
Pennsylvania Hospital, Philadelphia
Tel:
(215) 829-5410
Email: buckleym@pahosp.com

Jeanne Macrae, MD
SUNY Health Center at Brooklyn
Tel:
(718) 270-6707
Email: jmacrae@downstate.edu

Rand David, MD
Elmhurst Hospital Center
Mt. Sinai School of Medicine, New York
Tel:
(718) 334-2490
Email: radavid@aol.com

Dragica Mrkoci, MD
George Washington University Medical Center
Washington, DC
Tel:
(202) 994-4321
Email: domdzm@gwumc.edu

William Farrer, MD
Seton Hall University, New Jersey
Tel:
(908) 994-5455
Email: wfarrer@trinitas.org

Richard Rees, MD
Veterans Medical Center, New York
Tel:
(212) 686-7500 X-3865
Email: richard.rees@med.va.gov

Susan Grossman, MD
St. Vincent's Catholic Medical Center of New York
Staten Island Region
Tel:
(718) 818-2416
Email: susan_grossman@nymc.edu

Steven Reichert, MD
Englewood Hospital & Medical Center, Englewood
Tel:
(201) 894-3528
Email: steven.reichert@ehmc.com
Stephen Sibbitt, MD
University of Texas Medical Branch, Galveston
Tel:
(409) 772-2653
Email: ssibitt@utmb.edu

Sheik Hassan, MD
Howard University Hospital, Washington, DC
Tel:
(202) 865-6249
Email: shassan@howard.edu
Eric Holmboe, MD
Waterbury Hospital, Waterbury, Connecticut
Tel:
(203) 573-6573
Email: eholmboe@msn.com

Richard Simons, Jr., MD
Penn State College of Medicine
Milton S. Hershey Medical Center, Hershey
Tel:
(717) 531-4303
Email: rsimons@psu.edu

Brenda Horwitz, MD
Temple University Hospital, Philadelphia
Tel:
(215) 707-3397
Email: docbren@aol.com

David Smith, MD
Abington Hospital, Philadelphia
Tel:
(215) 481-2024
Email: davidgary@aol.com

Stephen Huot, MD, PhD
Yale Primary Care Internal Medicine Residency, New Haven
Tel:
(203) 785-5644
Email: stephen.huot@yale.edu

Sara Wallach, MD
Monmouth Medical Center, New Jersey
Tel:
(732) 923-6540
Email: swallach@sbhcs.com

Gregory Kane, MD
Jefferson Medical College, Philadelphia
Tel:
(215) 955-3892
Email: gregory.kane@mail.tju.edu

Cynthia Wong, MD
St. Vincent's Catholic Medical Center of New York
Staten Island Region
Tel:
(718) 818-2417
Email: none

David Kemp, MD
Easton Hospital, Philadelphia
Tel:
(610) 250-4517/4518
Email: dkemp@eastonhospital.org

Frederick Williams, MD
Washington Hospital Center, Washington, DC
Tel:
(202) 877-8290
Email: fkw1@mhg.edu

Nayan Kothari, MD
Robert Wood Johnson Medical School, New Brunswick
Tel:
(732) 745-8585
Email: kotharnk@umdnj.edu

i

THE MINI-CLINICAL EVALUATION EXERCISE PROJECT:
MINI-CEX REPORT IN BRIEF

The Mini-CEX Project: What Are the Goals?
The ABIM Mini-CEX Pilot Project was designed to accomplish three goals: 1) determine the feasibility of
using mini-CEXs as routine, seamless evaluations (minimum of four per PGY-1) for residents; 2) assess
the measurement characteristics of the mini-CEX; and 3) consider new Board policy.

What Is the Purpose of the Mini-CEX?
The mini-CEX is designed around both the skills that residents most often need in actual patient
encounters and the educational interactions that attending physicians routinely have with residents during teaching rounds. Conceptualized as a 15-20 minute snapshot of a resident/patient interaction, preliminary data indicated that the mini-CEX provides a valid, reliable measure of clinical performance
based on multiple encounters (four per year) by different examiners. 1,2
The mini-CEX was designed to assess the clinical skills, attitudes, and behaviors of residents that are
essential in providing high quality patient care. In conjunction with the ABIM Clinical Competence
Program, the Board’s earlier work in developing the mini-CEX and the implementation of the ACGME
Outcome Project, this pilot was launched to explore the feasibility of incorporating mini-CEXs among
the 393 currently accredited internal medicine residency programs training over 24,000 residents each
year. Other studies have shown the limitations and barriers associated with the traditional clinical evaluation exercise. 3-6 The mini-CEX is an efficient, effective tool for evaluating residents. Combined with the
other assessment strategies used by program directors, mini-CEXs serve to enrich residency programs’
already longstanding evaluation systems.

How Did the Project’s Work Scope Evolve?
The project was launched in Fall 1998, initially with 13 program directors who over the next four months
developed and tested the new mini-CEX form in their respective programs, and crafted various strategies
to both inform and educate faculty and residents about the use and value of mini-CEXs within the educational environment. In April 1999, the participants finalized the format and guidelines for using miniCEXs, and were reassured of total flexibility in implementation (settings and evaluators). The settings
spanned inpatient services (CCU/ICU, ward), ambulatory, emergency department, and other (admission
or discharge), and the evaluators included attending physicians, supervising physicians, and chief residents. Eight additional programs joined the pilot in Spring/Summer 1999.
Twenty-one residency programs, primarily located in the northeast, participated in the June 1999 September 2000 implementation phase. For each program, the goal was to achieve four documented
mini-CEXs on each PGY-1. Participants received a supply of the new mini-CEX "pocketbook of forms",
general guidelines to facilitate implementation and strategies developed early in the project.

1

During the course of the project, six meetings with participating program directors and staff from the ABIM
and ECFMG were convened in Philadelphia to provide an important venue from which to develop the form,
format, and strategies for implementing mini-CEXs and to monitor progress. The final meeting was held in
September 2000 to report the pilot results to participants, discuss potential policy implications, and develop a
plan for broad dissemination of related resource materials to program directors.

How Were the Mini-CEX Forms and Rating Scale Developed?
As part of the pilot, convenient pocket-size booklets of duplicate forms were developed and tested by participating program directors (Fall 1998 - Spring 1999) for use by evaluators. The concise duplicate forms were
designed to prompt immediate feedback to residents and provide documentation for the program file. The
form included the nine-point rating scale (used in ABIM tracking since 1988) which designates 1-3 unsatisfactory, 4-6 satisfactory, 7-9 superior, and defines 4 as "marginal." Two new areas were identified by the participants — counseling skills and organization/efficiency — and were added to the form. With the July 2001
implementation of the ACGME/ABMS general competencies for internal medicine, mini-CEXs are well positioned evaluation tools for measuring several elements of patient care, medical knowledge, communication and
interpersonal skills, practice-based learning and improvement, and professionalism as noted below in parentheses.

Mini-CEX: Competencies Assessed and Descriptors
Medical Interviewing Skills
(Patient Care)

Facilitates patient’s telling of story; effectively uses
questions/directions to obtain accurate, adequate
information needed; responds appropriately to affect,
non-verbal cues.

Physical Examination Skills
(Patient Care)

Follows efficient, logical sequence; balances screening
diagnostic steps for problem; informs patient; sensitive
to patient’s comfort, modesty.

Humanistic Qualities/Professionalism
(Professionalism)

Shows respect, compassion, empathy, establishes trust;
attends to patient’s needs of comfort, modesty, confidentiality, information.

Clinical Judgment
(Medical Knowledge)

Selectively orders/performs appropriate diagnostic
studies, considers risks, benefits.

Counseling Skills
(Communication and Interpersonal Skills)

Explains rationale for test/treatment, obtains patient’s
consent, educates/counsels regarding management.

Organization/Efficiency (Patient Care;
Practice-Based Learning and Improvement)

Prioritizes; is timely, succinct.

Overall Clinical Competence

Demonstrates judgment, synthesis, caring, effectiveness,
efficiency.

What Do the Findings Show?
The pilot was conducted in 21 programs and included 421 PGY-1s in internal medicine, 316 evaluators,
and 1228 encounters. In September 2000, at the project’s final meeting program directors received an individual program report which provided summary data for their respective programs and for comparison
purposes the aggregated data for all participating programs.

2

Encounters: At the level of encounters (n=1228), the mini-CEX confirms a range of patient problems
seen by the resident and observed by the evaluator, varied settings (inpatient, ambulatory, ED, other) and
two types of visits (new, follow-up). The time (median) committed to encounters was 15 minutes for
observation and five minutes for feedback. Residents’ performance ratings increased per encounter as
they advanced throughout the year, although the complexity (low, moderate, high) of patient problems
remained the same.
Residents: At the level of residents (n=421), the seven components of competence (medical interviewing
skills, physical examination skills, humanistic qualities, professionalism, clinical judgment, counseling
skills, organization/efficiency and overall clinical competence) were highly correlated, a finding that supports previous studies. Four encounters per resident produced acceptable confidence intervals for
aggregated ratings of five or better. The mini-CEX format was viewed positively by the residents (particularly the opportunity for feedback) and their satisfaction was not associated with performance ratings.
Evaluators: At the level of evaluators (n=316), there were some differences in stringency of evaluation
that bear further investigation. As with the residents, the mini-CEX format was well liked by the evaluators. Evaluator satisfaction was positively correlated with the duration of the encounter, the complexity
of the patient problem, and the competence of the resident.

What Conclusions Are Drawn From The Mini-CEX Project?
The ABIM pilot provided additional evidence that mini-CEXs assess residents in a much broader range
of clinical situations than the traditional CEX, have better reproducibility, and offer residents greater
opportunity for observation and feedback by more than one faculty member and with more than one
patient. In September 2000, participating program directors recommended that the Board establish policy requiring mini-CEXs (four per PGY-1) as an assessment method. In October 2000, the ABIM Board
of Directors endorsed this recommendation in principle and encouraged further discussion with APDIM
to determine what impact new policy would have on the internal medicine community. At the February
2001 APDIM Council meeting, the results of the Mini-CEX Project were presented. The APDIM Council
was reassured by the feasibility and reliability of mini-CEXs, and agreed with its added value as an assessment tool. However, the APDIM Council did not believe it is the appropriate time to require that programs adopt mini-CEXs for evaluation of PGY-1s. Accordingly, the Board strongly recommends (but
does not require) use of mini-CEXs to evaluate residents as an efficient, effective assessment tool for program directors and faculty. The ABIM remains committed to sponsoring workshops and national presentations and to providing resource materials to help program directors and faculty promote seamless
implementation of mini-CEXs in their programs.
For more information visit the Mini-CEX page on the ABIM website
.
REFERENCES
1.

Norcini JJ, Blank LL, Arnold GK, Kimball HR: The Mini-CEX (Clinical Evaluation Exercise): A Preliminary Investigation, Annals of Internal
Medicine, 1995;123:795-799

2.

Norcini JJ, Arnold GK, Blank LL, Kimball HR: Examiner Differences in the Mini-CEX, Advances in Health Sciences Education, 1997;2:27-33

3.

Holmboe ES, Hawkins RE: Methods for Evaluating the Clinical Competence of Residents in Internal Medicine: A Review, Annals of Internal
Medicine, 1998;129:42-48

4.

Kroboth FJ, Hanusa BH, Parker S, Coulehan JL, Kapoor WN, Brown FH, Karpf M, Levey GS: The Inter-rater Reliability and Internal
Consistency of a Clinical Evaluation Exercise, Journal of General Internal Medicine, 1992;7:174-179

5.

Noel GL, Herbers JE, Caplow MP, Cooper GS, Pangaro LN, Harvey J: How Well Do Internal Medicine Faculty Members Evaluate the Clinical
Skills of Residents? Annals of Internal Medicine, 1992;117:757-765

6.

Hauer KE: Enhancing Feedback to Students Using the Mini-CEX, Academic Medicine, 2000;75:524

3

GUIDELINES FOR IMPLEMENTING MINI-CEXS
What Is the Mini-CEX? Mini-CEXs focus on the core skills that residents demonstrate in patient
encounters. They can be easily implemented by attending physicians as a routine, seamless evaluation of
residents in any setting. The mini-CEX is designed as a 15-20 minute snapshot of a resident/patient
interaction. Based on multiple encounters over time, for example four during the year, this method provides a valid, reliable measure of residents’ performance. Program directors can encourage attending
physicians to perform one mini-CEX per resident during each rotation. Residents may also request their
attending physicians to conduct mini-CEXs on them during rotations.
How Are Mini-CEXs Documented? The ABIM Mini-CEX Forms Packet includes 10 forms in duplicate.
After completing the form, the evaluator provides the “original” to the program director and the “copy”
to the resident. A nine point rating scale (1-3 unsatisfactory, 4-6 satisfactory, 7-9 superior) is used; a rating of 4 is defined as “marginal” and conveys the expectation that with remediation the resident will meet
the set standard.
How Many Mini-CEXs ? On average, a minimum of four mini-CEXs per resident over the year.
Why Are Mini-CEXs Valued? Mini-CEXs are designed to enhance assessment, promote education, and
provide an effective evaluation tool. The advantages include the opportunity for residents to be observed
interacting with a broad range of patients in a variety of settings, to be evaluated by a number of different faculty members, and to have greater flexibility in both the settings and timing in which evaluation
occurs.
What Is Needed? A snapshot of clinical performance, the mini-CEX is also more efficient; optimally taking between 15-20 minutes. To strengthen the generalizability of the results of mini-CEXs and to provide
valid, reliable measures of performance, interaction is needed with a range of different patients (four on
average) in a variety of settings (e.g., inpatient, clinic, CCU, other) in which a focused medical interview
and physical examination can be conducted.
Settings

• Inpatient Services (CCU, ICU, Ward)
• Emergency Department

• Ambulatory
• Other including patient admission
and/or discharge

Evaluators

• Attending Physicians
• Chief Residents

• Supervising Physicians
• Senior Residents

Communication • Convey written and verbal expectations for mini-CEXs to evaluators and evaluatees.
• Reinforce goals and values of mini-CEXs to faculty and residents at conferences,
division meetings, pre-rotation briefings, and through written guidelines.
COMPETENCIES DEMONSTRATED DURING MINI-CEXs

Medical Interviewing Skills: Facilitates patient’s telling of story; effectively uses questions/directions to
obtain accurate, adequate information needed; responds appropriately to affect, non-verbal cues.
Physical Examination Skills: Follows efficient, logical sequence; balances screening/diagnostic steps for
problem; informs patient; sensitive to patient’s comfort, modesty.
Humanistic Qualities/Professionalism: Shows respect, compassion, empathy, establishes trust; attends to
patient’s needs of comfort, modesty, confidentiality, information.
Clinical Judgment: Selectively orders/performs appropriate diagnostic studies, considers risks, benefits.
Counseling Skills: Explains rationale for test/treatment, obtains patient’s consent, educates/counsels
regarding management.
Organization/Efficiency: Prioritizes; is timely; succinct.
Overall Clinical Competence: Demonstrates judgment, synthesis, caring, effectiveness, efficiency.
4

MINI-CEXS: EXPERIENCES AND
IMPLEMENTATION STRATEGIES OF PROGRAM DIRECTORS
On the following pages, a concise summary of each of the 21 internal medicine residency programs participating in the ABIM Mini-CEX Project describes both the experiences and implementation strategies.
A complete listing of the programs and program directors including telephone numbers and email
addresses is provided at the beginning of this publication as a resource to facilitate contact by other program directors who may consider implementing mini-CEXs in their programs.

ABINGTON MEMORIAL HOSPITAL, Abington, Pennsylvania
Program Director: David Smith, MD
Overview: Abington Memorial Hospital is a community hospital in suburban Philadelphia. The training
program in internal medicine sponsors 45 (19 USMG, 3 USIMG, 23 IMG) residents with a teaching faculty of seven full-time and 178 part-time/volunteer physicians. In conjunction with the MCPHahnemann University School of Medicine, residents also experience a ten-station series of objective
structured clinical examinations (OSCEs) using simulated patients. The program has participated in two
ABIM Mini-CEX Pilot Projects and values this strategy as both educational and evaluative.
Implementation: The novel idea of taking snapshots of a resident's performance as opposed to the "feature" length movie of the full CEX had an immediate appeal to our faculty. The increased reliability of
this new format stimulated a greater discussion about how we need to expand the use of this model as
both an educational and evaluative experience. Our program incorporates four mini-CEXs for each resident and a series of OSCEs. A sample of our residents' videotaped encounters serve as the material for
review by a behavioral medicine specialist with special attention paid to communication skills. The
remaining tapes are reviewed by a clinical faculty member with the resident. These joint sessions have
been extremely productive and educational.
A second activity deals with the evaluation and training of our faculty in precepting residents in their
primary care offices. Using sample videotaped mini-CEX encounters, faculty review the encounter and
rate the resident. Faculty who provide ratings that are either significantly higher or lower than the mean
participate in a debriefing exercise designed to standardize the criteria for performance ratings.
EASTON HOSPITAL, Easton, Pennsylvania
Program Director: David Kemp, MD
Overview: Easton Hospital is an acute care regional medical center providing patient care services for
more than 300,000 residents of Easton, Pennsylvania and the surrounding area. The internal medicine
residency program has eight categorical positions in each of the three years, and 21 full-time faculty. The
program is affiliated with the MCP-Hahnemann School of Medicine and the Philadelphia College of
Osteopathic Medicine. Residents are predominantly international medical graduates. Approximately one
half of our graduates enter fellowship training; the remainder enter practice as primary care general
internists.
Implementation: Mini-CEXs are now used exclusively, as we discontinued the traditional CEX one year
ago. Each PGY-1 resident is evaluated by a series of six mini-CEXs spaced throughout the year. We seek
to have a minimum of four different evaluators per resident and evaluations are performed in a variety of
venues (inpatient, continuity clinic, emergency department, private office). Both residents and faculty
alike have accepted the mini-CEX as a definite improvement in the evaluation of clinical skills, with
particular value in the time for feedback.

5

ELMHURST HOSPITAL CENTER-MOUNT SINAI SCHOOL OF MEDICINE, Elmhurst, New York
Program Director: Rand David, MD
Overview: The Mount Sinai School of Medicine (Elmhurst) Program is situated in a municipal community
hospital and integrated with rotations to its affiliate, the Mount Sinai School of Medicine. Elmhurst is a large
academic voluntary teaching hospital with 42 internal medicine residents and 152 full-time faculty. A special
focus of the program has been on ambulatory care curricular development for housestaff, primarily related to
the impact of language as a barrier to health care for underserved urban Hispanic populations.
Implementation: Now approaching our third year of using mini-CEXs, our program administers this evaluation tool quarterly for all PGY-1s. The majority of patient encounters have been with patients in the ambulatory
setting. Mini-CEXs are scheduled in advance with the faculty preceptors in the medical primary care continuity
practice. The first patient visit of the day often works best, as the faculty member is just beginning to precept.
The mini-CEX takes an average of 15-20 minutes, as it is a focused exercise, unlike the traditional CEX which
requires a much greater time commitment. The level of satisfaction with the mini-CEXs has been extremely
high, both with residents and faculty. The residents appreciate that it is the only time they are critiqued by direct
observation. This gives them a sense of security that what they are doing clinically is correct.
Personal Perspective: Although it has been validated that satisfactory performance on four mini-CEXs provides
us with the substantiation that a resident does indeed have the necessary skills, I found its usefulness elsewhere.
Even with residents who pass each component on the form, the preceptors' comments have added to my
understanding of our residents' competency in new ways, such as issues ranging from the overuse of clinical jargon during patient communication to awkwardness in conducting a physical examination. Overall, mini-CEXs
have been successfully incorporated into our evaluative process and appear through their efficiency and effectiveness to be here for some time to come.
ENGLEWOOD HOSPITAL & MEDICAL CENTER, Englewood, New Jersey
Program Director: Steven Reichert, MD
Overview: Englewood Hospital is a 300-bed community hospital in suburban north New Jersey and is a member of the Mount Sinai School of Medicine Graduate Medical Education Consortium. Our internal medicine
residency program trains 40 residents in categorical internal medicine, has six full-time faculty and is the only
residency program sponsored by the hospital. A small core faculty is augmented by voluntary teaching from
community practitioners. Our residents are largely international medical graduates, many of whom pursue subspecialty training. They spend the majority of their time at Englewood and have weekly continuity clinics at the
hospital.
Implementation: To date, we have performed mini-CEXs only in the medical clinic and only with interns. A
small core faculty group and the chief residents have taken responsibility for completing a minimum of four
mini-CEXs for each intern. We assume responsibility for reaching the goal of four instead of having the residents seek out examiners. We start the process in August (July is too unfair given the adjustment to new work
surroundings) and target having the first four finished before the end of December when PGY-2 contracts are
issued. Each resident must complete four mini-CEXs without a marginal (rating of 4) score. Any score of 4 or
less in any component requires remedial mini-CEXs. Those with multiple marginal scores may be required to
pass a full CEX. A summary report is tabulated for each intern which includes average scores, notable comments from examiners, and remediation plans if needed. In the future, we plan to implement mini-CEXs on the
medical wards by requiring supervising PGY-3 residents to perform one mini-CEX on each intern during their
ward chief month. We also are encouraging the ED and ICU attendings to perform mini-CEXs. We have thus
far been unsuccessful but are hopeful that these areas will provide valuable data as well.
We have had no problems administering the mini-CEX. Mini-CEX pocketbooks of forms are kept in the medical clinic where we perform all of our mini-CEXs. All exams are returned either directly or via inter-office mail
to a secretary who records them both in a logbook and in the residents' files. The data are collated on an Excel
spread sheet and all mini-CEX forms are personally reviewed by the associate program director to look for
insightful comments on performance.
6

GEORGE WASHINGTON UNIVERSITY MEDICAL CENTER, Washington, DC
Program Director: Dragica Mrkoci, MD
Overview: The Internal Medicine Residency Program at The George Washington University Medical Center
sponsors 85 residents and has 80 full-time faculty. There are 10-15 preliminary interns, but the majority of our
residents are in the categorical internal medicine program. There is also a primary care track and 5 of our residents each year elect to enter it and receive more training in ambulatory medicine.
Our residents have an excellent opportunity to rotate through four different hospitals: The George Washington
University Hospital, Holy Cross Community Hospital, VA Medical Center and NIH. Starting with PGY-1, our
residents have a weekly continuity outpatient clinic, where they have the opportunity to see 4-8 patients per session under supervision of a precepting physician. Every clinic starts with 30 minutes of didactics, an integral
part of the curriculum.
Implementation: The mini-CEX is part of the evaluation system for our PGY-1 residents. Our goal is to use
mini-CEXs exclusively and to discontinue traditional CEXs. To achieve this goal we have targeted a minimum
of 4 mini-CEXs in different clinical settings.
To date we have performed almost 100 evaluations and more than half of these were accomplished in the outpatient clinic, about 40% were done in the inpatient setting and 10% in the emergency department. The miniCEXs also revealed a diverse patient population for whom our residents provide care, as well as a spectrum of
complex patient problems. The major responsibility for completing mini-CEXs was placed predominately upon
the residents, however, preceptors and attending physicians were also frequently reminded of their responsibilities. Both evaluators and residents expressed their high satisfaction with mini-CEXs and consider it a major
improvement in evaluation of clinical skills.
HOWARD UNIVERSITY, Washington, DC
Program Director: Sheik Hassan, MD
Overview: The Internal Medicine Residency Program at Howard University Hospital sponsors 81 residents of
whom nine are in the preliminary year, and has 61 full-time faculty. The hospital is located in Washington, DC
and is just a short distance from the National Library of Medicine, the White House, the Smithsonian, and
many other places of national interest. The hospital and the department of medicine have a qualified and dedicated faculty that is readily accessible to the trainees. While the residents spend most of their time in training at
the University Hospital, they also rotate at several other sites, thus maximizing the diversity of patient encounters during their training. There is graded responsibility as training evolves, and during the third year residents
serve as consultants and spend most rotations on electives. At the end of training, residents are well prepared to
enter practice, research, subspecialty training, or other options in internal medicine. The department also sponsors fellowship programs in cardiology, pulmonary, gastroenterology, endocrinology, infectious diseases, hematology, and oncology.
Implementation: The program has been using mini-CEXs for almost three years and we have found it to be a
useful evaluative tool. Likewise, the trainees have reported their experiences to be not only evaluative but also
informative.
Evaluators: The entire full-time faculty is asked to participate, but a core group of faculty has been identified to
assist when there are "delinquencies." Our board-certified chief resident has been a valuable resource as well.
Setting: Mini-CEXs are easily implemented during rotations on inpatient wards, ED, ambulatory clinics, and
critical care units.
Time: A few faculty members spend more time than necessary conducting mini-CEXs, although with increasing frequency the amount of time per exercise has decreased. The actual time most faculty now spend is down
to about 20 minutes.
Value: Our goal is a minimum of six mini-CEXs for first-year residents. The mini-CEX provides actual direct
observations; identifies residents who need remediation; promotes and facilitates immediate feedback; and
helps our program in meeting the ACGME and RRC-IM requirements.
7

JEFFERSON MEDICAL COLLEGE, Philadelphia, Pennsylvania
Program Director: Gregory Kane, MD
Overview: Thomas Jefferson University Hospital has a three year training program in internal medicine
with approximately 200 teaching beds. While residents spend most of their time in training at Jefferson,
they also rotate through Methodist Hospital and the Wilmington VA Hospital. There are 119 internal
medicine residents, 92 full-time and 101 part-time/volunteer faculty. The department sponsors fellowship programs in cardiology; clinical cardiac electrophysiology; endocrinology, diabetes and metabolism;
gastroenterology; hematology/oncology; infectious disease; nephrology; pulmonary disease/critical care
medicine and rheumatology.
Implementation: At Jefferson Medical College we have routinely utilized mini-CEXs to emphasize the
direct observation of our interns' clinical skills. Our mini-CEXs are performed in the clinic where they
are often formally scheduled and also on the inpatient ward services and in the ICU. Our faculty may
take the opportunity to observe medical interviewing or physical diagnosis skills, but are equally likely to
observe the resident handling an important discussion with the patient or family or discussing the details
of a discharge. We feel that mini-CEXs give us an opportunity to comment upon clinical skills firsthand
and to emphasize the growth and importance of these skills throughout the remainder of the training
program. Overall, mini-CEXs are an important component of our entire evaluation process, giving us
direct observation of clinical skills to supplement block evaluations, ambulatory evaluations, and conference evaluations that occur regularly throughout the three years of training.
MONMOUTH MEDICAL CENTER, Long Branch, New Jersey
Program Director: Sara Wallach, MD
Overview: The Internal Medicine Residency Program at Monmouth Medical Center consists of 36 categorical residents and seven full-time faculty. Monmouth Medical Center (an affiliate of the Saint
Barnabas Health Care System) is a 500-bed community teaching hospital. Its academic affiliate is MCPHahnemann University School of Medicine in Philadelphia. Residents complete their entire training at
the Monmouth site but attend their continuity clinics at an offsite ambulatory center. They also rotate
through community physician offices. As there is no fellowship training at Monmouth Medical Center,
residents work directly with attending physicians when completing their subspecialty rotations. They are
required to rotate in geriatrics, emergency medicine, adolescent medicine, consultative medicine, cardiology, pulmonary medicine, gastroenterology, nephrology, hematology/oncology, and neurology. We have
specialized curricula in psychosocial medicine and palliative care, coupled with a weekly board preparation course.
Implementation: The majority of our mini-CEXs are conducted at our ambulatory site or as part of the
resident ward service. The attendings assigned to each clinic session are responsible for performing the
mini-CEXs and the ward faculty is assigned to do inpatient mini-CEXs. The program director and the
program coordinator monitor the numbers and the settings of each mini-CEX and when performance is
deficient, make specific appointments for additional mini-CEXs. Along with the immediate faculty feedback, results of these mini-CEXs are discussed at the resident's semiannual review with the program
director or her designee. In our program, unique opportunities for conducting mini-CEXs include geriatrics, medical consultation, and emergency medicine where the resident works on a one-to-one basis
with an attending physician. We are currently piloting a psychosocial mini-CEX which will be implemented on psychosocial rounds by our faculty psychologist. Mini-CEX instructions have been included
in our faculty development curriculum with the intention of extending it to subspecialty and ambulatory
offices.

8

NATIONAL NAVAL MEDICAL CENTER, Bethesda, Maryland
Division Director: Eric Holmboe, MD (currently at Waterbury Hospital, Waterbury, Connecticut)
Overview: This is a military program located in Bethesda, Maryland and affiliated with the Uniform Services
University of Health Sciences. It has 37 USMG residents, 35 full-time and 30 part-time faculty. Residents are
responsible for 106 internal medicine teaching beds. The program sponsors subspecialty fellowship programs
in cardiology; endocrinology, diabetes and metabolism; gastroenterology; hematology/oncology; infectious disease; and pulmonary disease/critical care medicine.
Implementation: Mini-CEXs are currently used in two settings, the interns’ longitudinal clinics and their consult medicine rotations. Twenty-one interns rotate one half-day each week in a longitudinal primary care clinic.
At the start of each clinic session, 1-2 attendings of a total of 11, join an intern for their first patient visit of the
clinic session. Feedback is usually given at the end of the clinic session. Mini-CEXs work very well at the
National Naval Medical Center where they are used in the outpatient setting by two attendings; the goal is six
per intern. On the whole it is seamless and both attendings and residents are very happy with it. We have
received considerable positive feedback since its implementation, and attendings have found it particularly helpful in identifying interns’ strengths and weaknesses. One resident commented about the mini-CEX that "no one
ever watched me do this before." Mini-CEXs have given the residents interesting insight into what they are
doing and saying to patients. There have been no problems keeping track of the forms.
PENN STATE MILTON S. HERSHEY MEDICAL CENTER, Hershey, Pennsylvania
Program Directors: Edward Bollard, MD and Richard Simons, MD
Overview: Our residency program has approximately 60 categorical internal medicine residents and 12 preliminary internal medicine residents. The program also offers a primary care track for those individuals who are
committed to careers in general internal medicine. The majority of the clinical training takes place at Penn
State's Milton S. Hershey Medical Center which is an academic, tertiary care hospital located in Hershey,
Pennsylvania. However, residents also gain experience caring for patients at a community hospital, York
Hospital, and a Veterans hospital, the Lebanon Veterans Administration Medical Center. In addition, residents
have a variety of outpatient experiences at both hospital-based clinics as well as in community-based internists'
offices.
The 75 full-time faculty in the department of medicine at Penn State's Hershey Medical Center are dedicated to
educating and training superior physicians in general internal medicine and medical subspecialties. The residency emphasizes the scientific basis and humanistic concerns necessary to achieve this goal. The program has
the flexibility to prepare residents for careers in the practice of general internal medicine and its subspecialties as
well as in academic medicine and clinical investigation.
Implementation: Mini-CEXs are a fundamental part of the comprehensive resident evaluation system.
Presently, each first year resident has four mini-CEXs: two inpatient encounters and two ambulatory encounters. We have found that mini-CEXs conducted during the interns’ morning work rounds are very useful in
terms of assessing their organizational skills and efficiency. The eventual goal in our program is to have four
mini-CEXs administered to all residents during each year of training.
PENNSYLVANIA HOSPITAL, Philadelphia, Pennsylvania
Program Director: Michael Buckley, MD
Overview: The Internal Medicine Residency Program at Pennsylvania Hospital currently trains 46 (22 USMG, 1
USIMG, 23 IMG) residents. It is a community hospital affiliated with the University of Pennsylvania School of
Medicine. The program is supported by 14 full-time and 125 part-time/voluntary faculty. Approximately 75%
of its graduates pursue traditional careers in internal medicine.
Implementation: The 10 members of the Residency Clinical Competency Committee are asked to each conduct
2-3 mini-CEXs. Each PGY-1 is informed that two of the four mini-CEXs need to be conducted by at least two
different attendings. The mini-CEX forms are reviewed with PGY-1s at which time the purpose and process of
the mini-CEX are also explained.
9

Settings: Inpatients – often patients the house officers know. Outpatients – new patients presenting
with acute problems in the residents’ clinic or the emergency room or occasionally in physicians’ private
practice offices
Barriers: Occasionally PGY-1s on the ICU rotation were a little more difficult to free up for an exam
outside of the unit, or the patients were too ill (comatose, intubated, etc.) for a reasonable history and
physical exam.
Feedback: The attending physicians unanimously feel that mini-CEXs are a very useful, essential exercise,
far superior in every way to the full CEX. They remain particularly excited about the opportunity for
longitudinal observation and feedback.
ROBERT WOOD JOHNSON MEDICAL SCHOOL, New Brunswick, New Jersey
Program Director: Nayan Kothari, MD
Overview: The Internal Medicine Residency Program at UMDNJ-Robert Wood Johnson Medical School in
New Brunswick, New Jersey is a university based training program with 113 residents and 96 full-time faculty.
Three hospitals (Robert Wood Johnson University Hospital, Saint Peter's University Hospital and Medical
Center at Princeton), the Cancer Institute of New Jersey and multiple outpatient clinical facilities serve as the
training ground for the program. The program's mission is to produce internists with the clinical and academic
skills needed to excel in a variety of primary care and subspecialty careers so there is a major focus placed on
the physician-patient relationship.
Implementation: Currently a core faculty group of 10 conducts the mini-CEX. We have developed a committment to achieve one mini-CEX per resident (PGY-1) each quarter.The flexibility and ease of the mini-CEX format allows it to be conducted in diverse settings that include general wards, ICU, clinic and ED. At the weekly
meetings of the core faculty, mini-CEX experiences are commonly shared and ideas exchanged. One of our four
chief residents is responsible for tracking the mini-CEX forms and experiences. Mini-CEXs provide a unique
evaluation tool and element to our program but are not perceived as “add-ons.” In the near future, mini-CEXs
will be an essential part of each year of residency training.
The program has also developed other tools to specifically teach and measure competencies. One innovation is
the 3S-ASK Project (Specialization in Selected Subjects - developing Attitudes, Skills and Knowledge). The
objective is to develop knowledge and skills in certain selected topics at the level of a consultant. Modules are
available in diabetes mellitus, breast care, office rheumatology/orthopedics and congestive heart failure. Now
under development is a peer review process run by the residents with practice-based learning and improvement
as a major objective.
SETON HALL UNIVERSITY SCHOOL OF GRADUATE MEDICAL EDUCATION
Trinitas Hospital, Elizabeth, New Jersey and St. Michael's Medical Center, Newark, New Jersey
Program Director: William Farrer, MD
Overview: The Seton Hall University School of Graduate Medical Education Internal Medicine Residency
Program has 75 residents, 30 of whom are based at Trinitas Hospital and 45 at St. Michael's Medical Center, and
19 full-time faculty. Residents rotate through both sites each year. All residents have one half day of continuity
clinic each week, plus block months of outpatient medicine in primary care faculty practices. There is a primary
care track as well as an AOA/AMA track. Both community hospitals, the two training sites, have rather different
patient and private vs. service mixes, making for a broad, enriching experience.
Implementation: Our first step before rolling out the mini-CEX was to meet with key faculty at both hospitals
to assure their buy-in to use this innovative tool. They also received background information on the mini-CEX
and were encouraged to make suggestions for implementation. Separate memos were sent to preceptors and
residents, explaining the goal of seamless implementation of mini-CEXs and the benefits. Both inpatients on
the residents' service and clinic patients were utilized.
Barriers: Some of our difficulties in getting mini-CEXs completed were: 1) clinic preceptors found it hard to be
away from the other residents for the 20 minutes needed to do the exercises, 2) it can be difficult to track down
residents as they rotate, and 3) faculty lethargy.
10

Solutions: Three were identified: 1) Clinic preceptors returned on a day they were not precepting to do the miniCEXs. They also focused on doing mini-CEXs on the first patient of the day, before other residents needed their
attention. 2) Check the monthly schedule to see which residents are available and set up a firm "appointment" for
the exercise. 3) Constant reminders and concerted effort to limit the number of responsible faculty members
involved.
Personal Perspective: Mini-CEXs are a major improvement over the previous "mega" CEX. We have learned many
surprising things about our residents' skills and deficiencies not evident at morning report, rounds, or via standardized tests such as the ACP-ASIM/APDIM/APM In-Training Exam. The immediate and specific feedback provided
to residents has much more impact than general suggestions given at evaluation sessions that are scheduled only a
few times a year. The ability to focus an entire session on such skills as counseling or giving bad news to a patient is
not only revealing, but an opportunity for on-the-spot mentoring. It is much easier to schedule these snapshot sessions than the grueling 90-minute CEX, less stressful for the residents and less soporific for the preceptors. Having
several different observers and several different patients also truly eliminates the "I was just having a bad day" phenomenon.
Based on our very positive experience piloting mini-CEXs during 1999-2000, we continue to use it as one of our
major clinical evaluation tools. Each PGY-1 resident has four mini-CEXs over the course of the year and each PGY-2
resident has two. For 2001-2002, we are employing a grid for resident and faculty scheduling. We will aim for quarterly exercises for the PGY-1s and one in each half for the PGY-2s. Our expectation is that this longitudinal approach
will allow us to better track our residents' progress and provide important insights into performance not previously
available to us.
ST. VINCENT'S CATHOLIC MEDICAL CENTER OF NEW YORK, Staten Island, New York
Program Directors: Susan Grossman, MD and Cynthia Wong, MD
Overview: The New York Medical College (Richmond) Internal Medicine Residency Program has as its primary
training site the St. Vincent's Catholic Medical Center of New York, Staten Island Region. This New York State designated primary care residency program sponsors 58 residents; 26 first-year residents, 13 categorical, and 13 preliminary, and has nine full-time and 120 part-time faculty. Residents spend 20% of their time in an outpatient continuity setting, either in the medical clinic located at the primary training site or in a community preceptor office.
Implementation: During our first year in the project, we gave PGY-1s the option of having four mini-CEXs in place
of the traditional CEX as part of their annual clinical competence evaluation. Several faculty development workshops were held with the full time inpatient and outpatient faculty to introduce them to the concept and implementation of mini-CEXs. Initially only two faculty members (Chief, General Internal Medicine and Program Director)
actually did the mini-CEXs. We found that the easiest way to incorporate them into the program was to do a miniCEX on some aspect of the patient interaction during attending rounds. Feedback to the resident was given immediately after rounds.
The response by the residents to mini-CEXs was overwhelmingly positive. We have found that their implementation
has stimulated us to do much more bedside teaching and to be aware of deficiencies in residents' clinical skills that
we would not have identified previously. It has also stimulated us to expand our clinical skills teaching in other formats, such as "auscultation sessions," and communicated to our residents the value we place on clinical competence.
This year we extended the mini-CEX option (4 per year) to PGY-2s and PGY-3s. We are now planning further faculty development workshops to discuss how to involve more faculty in conducting mini-CEXs. We also plan to give
the mini-CEX forms booklets to residents who are on subspecialty electives and require them to have at least one
mini-CEX done during these rotations.
Personal Perspectives: We have found mini-CEXs to be a tremendous eye-opener and an important tool and evaluation resource. A key feature of the mini-CEX is its flexibility. It can be used to observe residents across the entire
spectrum of activity, i.e., taking a history, physical examination, counseling, etc. Even though we had tried to emphasize bedside teaching, we did not realize how little our faculty were actually observing and evaluating residents' clinical skills until we participated in the Mini-CEX Project. We also learned that the skills that cause a resident to shine
on rounds, such as verbal case presentations and transmission of didactic material, do not necessarily correlate with
clinical skills.
11

SUNY DOWNSTATE MEDICAL CENTER, Brooklyn, New York
Program Director: Jeanne Macrae, MD
Overview: The SUNY Downstate Internal Medicine Residency Program comprises 138 residents who
rotate among three hospitals: Kings County, a large municipal hospital; University Hospital, the only private hospital in the program; and the Brooklyn campus of the Department of Veterans Affairs New York
Harbor Health Care System. In addition to our categorical program in medicine, we have a small preliminary track, programs in combined Medicine-Pediatrics and Medicine-Psychiatry, and a six-resident primary care track program. The program also has 150 full-time faculty.
Implementation: During the pilot phase of the mini-CEX project, we conducted mini-CEXs exclusively
on the inpatient medical floor at Kings County Hospital and we were successful in achieving the targeted
number although close monitoring and frequent reminders were required.
This academic year we are requesting that: 1) Clinic preceptors perform two mini-CEXs on each intern per
year (maximum eight mini-CEXs per year); 2) Medical floor attendings perform one mini-CEX per intern on the
team each month (maximum two mini-CEXs in a month); and 3) PGY-4 chief residents at Kings County perform
one mini-CEX per intern per month in the group they are supervising (maximum eight mini-CEXs in a month)

With these requests we should easily reach four mini-CEXs per intern per year even with suboptimal
compliance. The value of mini-CEXs in a large, complex program such as ours cannot be overstated for
the information they provide, the feedback they promote and the opportunity they present to see residents “in action.”
TEMPLE UNIVERSITY HOSPITALS, Philadelphia, Pennsylvania
Program Director: Brenda Horwitz, MD
Overview: Temple University Hospital is the major teaching center for Temple University School of
Medicine. The hospital is a 514 bed tertiary referral center located in North Philadelphia. It also serves
the primary care needs of the surrounding community. The Internal Medicine Residency Program at
Temple University Hospital trains 86 residents and emphasizes independence for housestaff, hands-on
responsibility for patient care, scholarly activities and a rich educational curriculum. Although the majority of clinical rotations take place at Temple University Hospital, residents also spend additional time at
Fox Chase Cancer Center and Abington-Memorial Hospital, a community hospital in the nearby suburbs. During the PGY-2 and PGY-3 years, residents may elect to participate in the primary care track
which provides a more intensive ambulatory experience. The program has 96 full-time faculty.
Implementation: Medical residents are required to complete four mini-CEXs before the end of their first
year of training. The mini-CEX has been implemented in a variety of venues, the most common being
the outpatient medical clinic and the inpatient medical service. However the emergency room, intensive
care units and subspecialty clinics are also viable options. It is suggested to the residents that two miniCEXs be completed in the outpatient setting. The responsibility for conducting the mini-CEX is placed
predominantly upon the residents; however, each year preceptors in the medical outpatient clinic are also
reminded of their responsibility to participate in mini-CEXs. Attending physicians from each of the subspecialty services are also recruited to help in this process. Performance on the mini-CEX is recorded in
the forms booklet which the residents are encouraged to keep readily available (for example, in their
white coats). The completed “original” mini-CEX form is submitted to the program administrator who
keeps a flow sheet of all completed exams. The duplicate copy is kept by the resident for his/her personal
file. The mini-CEX facilitates much needed feedback, real time observation of patient encounters, and
crucial documentation of performance.

12

UNIVERSITY HEALTH CENTER OF PITTSBURGH, Pittsburgh, Pennsylvania
Program Director: Frank Kroboth, MD
Overview: The University Health Center of Pittsburgh's Internal Medicine Residency Program consists of
multiple tracks at three major hospitals — Presbyterian University Hospital, the Oakland VA Medical
Center, and UPMC Shadyside Hospital. The University Categorical Track comprises 72 residents and has
212 full-time faculty. In addition, the Primary Care Track numbers 18, Women's Health Track 9,
Community Categorical Track 36, and Med-Peds Track 16. Lastly, there are 26 preliminary positions
between the campuses. Utilizing both the curricula of these tracks and the considerable research
resources of the medical center, we provide opportunities to trainees interested in either primary care or
in an academic career. Our Division of General Medicine also offers or coordinates a variety of graduate
degrees for fellowship trainees in all specialties.
Implementation: The program has presently adopted the mini-CEX as our preferred bedside resident
evaluation. Over the years, we have experimented with the mini-CEX in both in-patient and out-patient
settings. We also are utilizing the out-patient mini-CEX and have found it most helpful to present clinic
preceptors with the mini-CEX pocket forms in July. They can observe each of their interns throughout
the year, often by using the first patient or two of the half-day session for this purpose. We have also
asked our medical interviewing faculty to rate our interns' videotaped patient sessions after feedback has
been provided in the usual one hour session. This strategy worked well, with only minimal disruption of
this otherwise didactic exercise.
In our use of mini-CEXs on in-patients, we were especially interested in its applicability as an evaluative
tool for discharge day activities. We found, however, that the practicality of matching houseofficer and
faculty availability was sub-optimal. With the establishment of hospitalist services on the general medicine floors this year, we are hopeful that we will find the timing of these interactions to be much easier
and allow us to systemically evaluate the critically important activities that occur during discharge day.
UNIVERSITY OF TEXAS MEDICAL BRANCH, Galveston, Texas
Program Director: Stephen Sibbitt, MD
Overview: UTMB's Internal Medicine Residency Program comprises 102 categorical, preliminary, and
combined medicine-pediatrics resident physicians and 80 full-time faculty. UTMB is also home to the
only combined internal medicine-aerospace residency program in the United States. The internal medicine residents provide inpatient and outpatient care for patients from all over Texas, and they also rotate
through the only maximum-security prison hospital in Texas.
Implementation: Four steps describe the process.
1) Electronic and written notification regarding mini-CEXs to all clinical internal medicine and emergency
department faculty.
2) Electronic, written, and verbal notification regarding mini-CEXs to PGY-1 internal medicine residents.
3) Personal distribution of residents' mini-CEX packets which contain:
a) written instructions previously mailed to and discussed with PGY-1 residents
b) mini-CEX forms booklet
c)

pre-addressed envelopes to assist faculty evaluators in conveniently mailing their completed mini-CEX
evaluations directly to the Associate Program Director

4) Collection and Residency Committee review of mini-CEX evaluations on resident performances.
Value: The mini-CEX assures that residents are observed caring for patients, provided one-on-one feedback and
tracked longitudinally to measure performance and improvement.

13

VA MEDICAL CENTER, New York City, New York
Program Director: Richard Rees, MD
Overview: Up until June 2001, the NYU Medical Center (Veterans) Program was an independent residency
program consisting of 40 residents including 7 preliminary residents with 11 PGY-1s, and 36 full-time faculty.
Residents were primarily based at the VA Hospital, but rotated through Bellevue, Tisch (University) and Sloan
Kettering Hospital. As of July 2001, the Bellevue Program and the VA Program have combined. The combined
program has over 150 residents with 52 PGY-1s. The housestaff are primarily based at Bellevue but rotate to the
VA, Tisch and Sloan Kettering Hospitals.
Implementation: Our program has implemented the mini-CEX primarily in the outpatient area. Interns are
required to participate in four mini-CEXs during the year. The first focuses on history-taking and the second on
physical examination. Both of these are performed with new patients. We try to complete these by the end of
the first quarter in order to remediate early, if necessary. The third and fourth mini-CEXs are performed with
return patients and focus on the whole visit. The first three mini-CEXs are observed by the intern's clinic preceptor. The fourth is observed by another clinic attending to give the houseofficer a different perspective.
Feedback is given during the exercise, as appropriate for teaching purposes, and immediately afterward. We use
these evaluations to assess new houseofficers' basic capabilities early in the academic year with a focus on early
remediation as needed. Greater importance was also placed on mini-CEXs done by other than the precepting
attending as another source of feedback. Primary care attendings have become more comfortable with the flexibility of mini-CEXs and use them to focus on specific aspects of care as appropriate based on the individual
resident's strengths and weaknesses. For the current academic year, the evaluation committee is deciding how to
use mini-CEXs even more effectively and in what venues to standardize the process given the new large number
of housestaff and attending staff that will be involved in the process.
WASHINGTON HOSPITAL CENTER, Washington, DC
Program Director: Frederick Williams, MD
Overview: The Washington Hospital Center is the largest, busiest, and fastest-growing academic medical center
in the nation's capital. Each year we accept 18 categorical and 17 preliminary housestaff into our residency program. Our 907-bed hospital serves as a tertiary referral center for the District of Columbia, Maryland, and
Northern Virginia, but also the primary source of hospital care for the surrounding community. The newly renovated Ambulatory Care Center is a state-of-the-art facility and serves as the location for the resident continuity
practice experience. The principal goal of our program is to provide the best possible curriculum to prepare residents for a career in either primary care internal medicine or the subspecialty of their choice. We have been
highly successful in achieving that goal because of the total team commitment of our 55 full-time and 140 voluntary faculty, along with our nursing and ancillary support staff, residents, fellows and students, and a hospital
administration which is dedicated to ensuring optimal logistical support for all its academic programs.
Implementation: We have asked our entire full-time faculty to participate in mini-CEXs. Each of our faculty
spend two months a year as ward attendings, and during these months we request that they complete a miniCEX on each of the members of their team. We have also requested that the general medicine faculty who precept in the ambulatory setting complete a mini-CEX on each of their housestaff on a quarterly basis. We have
found that the mini-CEX is particularly easy to implement in the outpatient setting, and not an add-on.
Barriers: We purposefully set the number of targeted mini-CEXs high realizing that not all housestaff would
have an evaluation done every ward month and this has been the case. Many of our faculty who are more
accustomed to the traditional CEX have found it a difficult transition to limit the evaluation period to a short
encounter. In reviewing the evaluations accumulated there still seems to be a tendency toward grade inflation
with a very high percentage of the markings in the superior range.
Value: The mini-CEX has dramatically increased the number and source of evaluations we have on each house
officer. It also serves as a format that promotes direct observation and immediate feedback to our trainees.

14

YALE PRIMARY CARE INTERNAL MEDICINE RESIDENCY, New Haven, Connecticut
Program Director: Stephen Huot, MD, PhD
Overview: The Yale Primary Care Internal Medicine Residency Program has 68 residents, 20 in each year
of the three-year internal medicine program, and 8 preliminary interns. There are three main teaching
hospitals: Yale-New Haven Hospital, where residents spend approximately 40% of their time, is the site of
subspecialty training and two community hospitals, St. Mary's Hospital and Waterbury Hospital, both
located 16 miles from New Haven in Waterbury, Connecticut, are the sites for general medicine training.
The residents weekly continuity practice occurs in one of 2 hospital-based clinics in Waterbury that serve
an ethnically diverse urban population. Preceptors for the resident continuity clinics are academic general medicine faculty. Additional training in ambulatory medicine occurs through block rotations (12
weeks each year) and elective rotations including opportunities in International Health settings. All residents have rotations in private practice settings as part of the ambulatory block experience each year.
Implementation: We are now entering our third year of using the new mini-CEX in the Yale Primary
Care Program. We require that each intern have a minimum of 4 mini-CEXs as part of their continuity
clinic experience. All 4 cannot be done by the same faculty member. Copies of the completed mini-CEXs
are maintained in the residents' credentialing files, are included in the clinical competence committee
reviews, and are part of the semiannual review of performance that each intern has with their assigned
advisor. Beginning in July 2001, we require that one of the 4 mini-CEXs be direct observation of a pelvic
and breast exam. Two of the mini-CEXs must be completed by the end of September of the internship
year.
Faculty Development: In the areas of direct observation skills, questioning skills and feedback, faculty
development has been an important element of successfully implementing mini-CEXs and assuring their
maximum effectiveness. In addition, we discuss both barriers and ways to accomplish the mini-CEX
goals at our core faculty meetings and provide feedback to faculty advisors about completion of the 4
required mini-CEXs for each of their advisees.
Value: Faculty and housestaff now unanimously view the mini-CEX as an important evaluation and
feedback tool. Some interns report that these are the first times they have ever been directly observed
and given immediate feedback about their interactions with patients.
The mini-CEX is also being incorporated into the evaluation system for interns in our traditional internal medicine residency and in our combined medicine/pediatrics residency for the July 2001 - June 2002
academic year.

15

Example of Memo to Residents
DATE:

September 2001

TO:

Internal Medicine Residents

FROM:

(Name), Program Director
(Name), Associate Program Director
(Name), Chief Resident

RE:

MINI-CLINICAL EVALUATION EXERCISE (Mini-CEX)

Dear Resident,
The Mini-CEX is an efficient, effective evaluation tool that is designed to assess your clinical skills. As
part of our evaluation system, all residents are expected to participate in Mini-CEXs. The Mini-CEX is
a short, focused activity that provides the opportunity for you to be observed interacting with a
patient in any clinical setting (inpatient ward, ambulatory clinic, emergency department, etc.). During
this 15-20 minute observation, you may be evaluated in the following areas: medical interviewing
skills, professionalism, clinical judgment, counseling skills, organization/efficiency, and overall
clinical competency. The Mini-CEX is one of a number of strategies we use in evaluating your performance throughout residency.
Instructions:
1.

Complete four Mini-CEXs by (date). Each of your four patient encounters should be
evaluated by a different faculty member.

2.

Request attending or teaching faculty to observe you evaluate a patient.

3.

Provide faculty with Mini-CEX forms to document the exercise.

4.

Conduct an “observed” patient interview and evaluation that is focused and appropriate to
the patient’s complaint. Please be concise and organized.

5.

Ask the attending to complete the evaluation form and provide you with direct feedback.
Once the evaluation is completed, both you and the faculty member are required to sign the
form. You should retain the “yellow” copy. The “white” copy should be forwarded directly
to the Associate Program Director. Your Mini-CEX packet contains a supply of addressed
envelopes for this purpose. Please provide one of these envelopes to each faculty who
observes and evaluates your Mini-CEX.

6.

Obtain additonal copies of Mini-CEX forms from the Housestaff Office as needed.

Thank you for your commitment to improving the evaluation of your clinical competence.

16

Example of Memo to Faculty
DATE: September 2001
TO:

Internal Medicine Clinical Faculty

FROM: (Name of) Associate or Program Director
RE:

Mini-Clinical Evaluation Exercise (Mini-CEX)

Dear Faculty Member,
The Mini-Clinical Evaluation Exercise (Mini-CEX) is designed to introduce a streamlined format
that spotlights the assessment of residents’ clinical skills during training. Our internal medicine
residency program is now using this efficient, effective and tested evaluation strategy.
The goal of the Mini-CEX is targeted observation (15-20 minutes) by you of a resident interacting with a patient in any clinical setting (inpatient wards, ambulatory clinics, emergency department, etc.).
During a rotation, or in the ambulatory clinic, residents will ask you to observe and evaluate
their interactions with patients. During the next nine months our PGY-1s are expected to
complete four Mini-CEXs from four different faculty. Documentation, evaluation and feedback
are easily recorded on the concise Mini-CEX form that the resident will provide to you.
Instructions:
1.

Observe the resident interact with a patient (target 15 minutes).

2.

Complete the Mini-CEX form that the resident provides to you at the time of the patient
encounter. Instructions for filling out the form are self-explanatory.

3.

Provide the resident with direct feedback on performance (about 5 minutes).

4.

Sign the form and also obtain the resident’s signature. Give the “yellow” copy to the resident
and forward the “white” copy to me (an addressed envelope will be provided by the resident).

5.

Address any comments and/or concerns about the resident’s performance to my attention.

Thank you for your commitment to improving the evaluation of residents and providing
constructive feedback on their performance.

17

Example of Memo to Faculty

DATE:

September, 2001

TO:

(Name), Attending Physician

FROM:

(Name), Program Director

SUBJECT:

Using the Mini-Clinical Evaluation Exercise: Form and Format

The Mini-Clinical Evaluation Exercise (Mini-CEX) provides a streamlined form and format for
spotlighting the assessment of residents’ clinical skills during training.
The goal of the Mini-CEX is targeted observation (specifically 15-20 minutes) by the attending physician of a resident interacting with a patient during a rotation or in clinic.
Documentation, evaluation and feedback are easily recorded on the concise, duplicate MiniCEX form. During the course of a year, four mini-CEXs from different attending physicians
can provide a valid, reliable, and reproducible measure of a resident’s clinical performance.
The Mini-CEX is one of a number of strategies our program uses in evaluating residents’ performance throughout training. During your rotation, please conduct a Mini-CEX on the PGY-1s
on your team (or in your clinic). Enclosed for your use is the Mini-CEX Forms Booklet that can
be conveniently carried in your coat pocket. When you conduct a Mini-CEX, please complete
the form, return the white copy to me and provide the yellow copy to the resident.
I look forward to your comments and want to thank you for your commitment to improving
the evaluation of residents and providing constructive feedback on their performance.

18

THE MINI-CEX FORM
The forms are conveniently designed in a slim packet of 10 duplicate forms (one for the resident,
one for the program director) that easily fit into a coat pocket. Below is the description provided
on the inside cover of the packet (left) and the mini-CEX form (right).

19

Acknowledgements
Mini-CEX Project Staff — American Board of Internal Medicine (ABIM), Philadelphia

Linda Blank
Vice President
Clinical Competence and Communications

John Norcini, PhD
Senior Vice President
Psychometrics and Research

Hollice Lespoir
Manager, Mini-CEX Project

Gregory Fortna, MSEd
Senior Psychometrician

Nancy Grant
ABIM Visit Program Manager

Daniel Duffy, MD
Executive Vice President

Stephanie McCrea
Administrative Secretary
Questions regarding the ABIM Mini-CEX Project should be referred to
Mses. Linda Blank 215-446-3567  or Hollice Lespoir 215-446-3530

Project Consultants
Educational Commission for Foreign Medical Graduates (ECFMG), Philadelphia

Gerald Whelan, MD
Vice President for
Clinical Skills Assessment

Danette McKinley
Associate Psychometrician
William Burdick, MD
Assistant Vice President of
Clinical Skills Assessment Operations

Jack Boulet, PhD
Director, Test Development
and Research

Questions regarding the ECFMG Clinical Skills Assessment Program should be referred to
Drs. Gerald Whelan 215-823-2201  or Jack Boulet 215-823-2227


20



Source Exif Data:
File Type                       : PDF
File Type Extension             : pdf
MIME Type                       : application/pdf
PDF Version                     : 1.2
Linearized                      : Yes
Create Date                     : 2001:11:07 13:17:36
Producer                        : Acrobat Distiller 4.0 for Macintosh
Modify Date                     : 2001:11:07 13:17:37-08:00
Page Count                      : 24
EXIF Metadata provided by EXIF.tools

Navigation menu