How to assess communication milestones when standardized patients aren't an option
ACGME's 2026 Assessment Guidebook lists four options for communication skills. A workflow for weeks with no standardized patient, and what a transcript adds.
ClinicalSim Team
ClinicalSim
A residency program can assess interpersonal and communication skills milestones without adding standardized patient sessions by using the other methods the ACGME lists for that competency: direct observation in clinical care, multisource feedback, patient-reported feedback, and simulation that doesn't need an actor. The clinical competency committee weighs those records together twice a year, and the milestone judgment stays with the committee.
What the committee is rating
The ACGME Common Program Requirements for residency say residents "must demonstrate interpersonal and communication skills that result in the effective exchange of information and collaboration with patients, their families, and health professionals" (Section 4.8). The program must give an objective performance evaluation based on the Competencies and the specialty-specific Milestones, use multiple evaluators, and pass that information to the Clinical Competency Committee (Sections 5.1.b to 5.1.b.2). The committee reviews all resident evaluations at least semi-annually and determines each resident's progress on the specialty-specific Milestones (Sections 5.3.c and 5.3.d).
The one method those requirements name is direct observation: faculty members "must directly observe, evaluate, and frequently provide feedback on resident performance during each rotation or similar educational assignment" (Section 5.1). The document doesn't mention standardized patients. Specialty requirements can add their own rules, so read yours.
The Milestones themselves are not the assessment form. The ACGME's Milestones Guidebook says they "were never intended to serve as regular assessment tools such as end-of-rotation evaluations" and are "designed to guide a synthetic judgment of progress at least twice a year." So the program doesn't need a milestone level out of every encounter. It needs dated records of what a resident said and did, from enough sources that the committee can make that judgment.
The subcompetency names depend on your specialty
The Milestones 2.0 workgroup wrote three harmonized Interpersonal and Communication Skills (ICS) subcompetencies and said specialties "will integrate the new subcompetencies and milestones into their specialty-specific context as desired" (Morrison et al., ACGME). In the internal medicine, pediatrics, family medicine, emergency medicine, and surgery sets we checked, the three are patient- and family-centered communication, interprofessional and team communication, and communication within health care systems.
The level descriptions under those names differ between the internal medicine, pediatrics, and surgery sets, and some sets go further. The pediatric critical care set adds a fourth, "Communication Around Serious Illness, Including End-of-Life Care." Build the assessment plan from your own specialty's document, not from another program's summary of it.
When a standardized patient isn't an option
An SP session is unavailable in three situations. The program has no standardized patient (SP) budget. The SP program has nobody trained for the case the resident needs. Or the committee meets in three weeks and there is no SP time before then.
The ACGME's Assessment Guidebook shows what sits behind the second and third. It says a new SP can learn a new clinical problem in eight to 10 hours and an experienced SP in six to eight, that learning to score a checklist takes about twice as long, and that "development time can be considerable." It names cost and access to expertise, materials, and setting as "the most critical feasibility challenges in simulation." The staffing arithmetic behind that is in why standardized patient programs run out of capacity.
None of this argues against SPs. The same guidebook says SPs "can be especially useful for assessing the competencies of Patient Care and Interpersonal and Communication Skills for the one-on-one patient encounter," and that is still true on the weeks when none is free.
A workflow that doesn't need another SP session
1. Choose the conversations before the methods
Pick two or three conversations the committee wants evidence on for this resident or this class, such as a goals-of-care discussion, a consent discussion, or a consult request. What a CCC needs before it can act on a communication problem covers what each record has to carry.
2. Observe the conversations that happen on service
The ICS workgroup wrote that performance in this domain "requires direct observations of skills in the patient care setting, with scenarios, or using simulation." Direct observation comes first in that sentence, the Common Program Requirements already require it of faculty on every rotation (Section 5.1), and it costs no SP time. Ask each rotation for one dated observation of a named conversation, written as what the resident said, by the faculty member who heard it.
3. Use multisource feedback for team communication
The Assessment Guidebook lists multisource feedback for ICS "especially regarding interprofessional care." Nurses, pharmacists, and consultants hear how a resident hands off, escalates, and asks for a consult, which no patient encounter shows, staged or real.
4. Add patient-reported feedback where the clinic collects it
Patient-reported feedback and experience surveys are on the same ACGME list. They pool many visits and say little about any one conversation, so treat them as context for the other records.
5. Use simulation without an actor for the conversation that didn't come up
Some conversations can't be scheduled. A resident may finish a rotation without disclosing an error or leading a family meeting. The guidebook says simulation "can be an especially meaningful supplement to clinical experience when it involves a clinical scenario that is rare, but serious."
When no SP is available, the options are faculty or peer role-play and AI patients. Peer simulated patients have trial evidence in medical students and cost educator hours. A voice-based AI patient conversation costs no actor or room time, and the resident can repeat it.
6. Bring dated records to the committee, and say what is missing
File each record with its date, its setting, and whether the conversation was real or simulated. The workgroup warned that "single discrete assessments may not represent the actual capacity of the trainee to perform," so one simulated conversation shouldn't move a rating by itself. Where nothing was observed, say so instead of estimating.
What an AI patient report gives faculty
ClinicalSim is a voice-based AI patient platform, and we publish full recorded examples. In a goals-of-care conversation about a medically complex child, the learner talks for about seven and a half minutes with the mother of a boy in the PICU who may not return to his baseline.
The report scores the conversation against REMAP, a goals-of-care framework, one step at a time: 4 of 5 for reframing, 3 of 5 for responding to emotion, and 2 of 5 each for mapping the future, aligning with values, and planning. Under each score it prints the words the score rests on. For the emotion step it quotes the learner asking "what you're most worried about?" and the mother answering "I'm worried you're telling me he's dying," then says the learner "pivoted quickly to clinical explanation rather than naming and sitting with the emotion." Each step ends with wording the learner could try next time.
The same report scores the conversation against SPIKES. On the step where a clinician asks how much the family wants to know, the evidence line reads "No corresponding moment in the encounter." The report says nothing happened instead of finding a quote to fit.
A faculty member can read that and agree, disagree, or override it. The ACGME guidebook makes the same point about SP score reports: "Providing learners with only numeric ratings or scores will not be very useful for guiding ILPs," the individualized learning plans a resident works from.
What the report is not
The report is not a milestone rating. The committee determines progress on the Milestones, and a framework score from one simulated conversation is one record in the file beside direct observation and multisource feedback.
Nor is it an observation of patient care. The transcript shows what a resident said to an AI patient, not how that resident talks with patients on the ward.
The report also covers only part of the competency. A conversation with a patient or family member bears mostly on patient- and family-centered communication. Team communication and communication within health care systems need the other methods above.
The ACGME's Assessment Guidebook has a section on AI in assessment. It doesn't review any product, and it ends with a standard we agree with: "Across all domains, the central paradigm remains intelligence augmentation, supporting, rather than replacing, expert human judgment."
Four statements apply to every ClinicalSim report:
- ClinicalSim output is formative. It is evidence that informs program judgment, and it is not intended for employment, credentialing, privileging, licensure, or other high-stakes decisions.
- Consistency between model runs is not the same thing as agreement with expert human raters. Faculty-rater validation on a customer's own rubric is the gate before any high-stakes use, and that work has not been done.
- ClinicalSim extends a standardized patient program and does not replace it. SP encounters remain the gold standard for live coaching and high-stakes assessment.
- ClinicalSim is not endorsed by, affiliated with, or approved by the ACGME. Scores map to published frameworks; they do not establish compliance with any standard, which a surveyor determines against the institution's own policy.
Questions program directors ask
Can a residency program assess communication milestones without standardized patients?
Yes. The ACGME's Assessment Guidebook lists patient-reported feedback, multisource feedback, direct observation, and simulation for Interpersonal and Communication Skills, and marks simulation "(if available)." A program without SP access can build the committee's file from the first three and from simulation that needs no actor.
Does the ACGME require standardized patient sessions for the ICS milestones?
The Common Program Requirements for residency require faculty to directly observe residents (Section 5.1), an objective evaluation from multiple evaluators (Section 5.1.b), and a committee review at least twice a year (Section 5.3). They don't mention standardized patients. Specialty requirements can differ, so check your own.
Can an AI patient score be reported as a milestone level?
No. The clinical competency committee determines each resident's progress on the Milestones. A ClinicalSim report is formative evidence the committee can read alongside other assessments, and it does not establish a milestone level.
Are virtual patients an alternative to standardized patients?
No, not for what SPs do best. ClinicalSim extends a standardized patient program and does not replace it. AI patients add practice and a reviewable transcript when no SP is booked, and live coaching and high-stakes assessment stay with SPs and faculty.
What does standardized patient time cost?
Two published university rate cards put fully loaded standardized patient time at roughly $123 to $177 an hour (UCLA and the University of Utah rate cards, as published at August 2026). That is the SP-time component and not the whole bill. The comparison of AI clinical simulation and standardized patients explains why we don't publish a cost per encounter.
References
- Ragsdale JW, Elzie CA, Howley LD, Speicher M, Williams RL, Tanaka P, Holmboe ES, Hatlak K, Edgar L. Assessment Guidebook, revised edition. Accreditation Council for Graduate Medical Education. 2026.
- Edgar L, Hatlak K, Haynes IL, Holmboe ES, Hogan SO, McLean S. The Milestones Guidebook: competency-based medical education and Milestones development. Accreditation Council for Graduate Medical Education. 2025.
- ACGME Common Program Requirements (Residency), Sections 4.8, 5.1, and 5.3.c to 5.3.e, effective July 1, 2026. Accreditation Council for Graduate Medical Education. 2026.
- Morrison LJ, Joyce BL, Meyer LE, Moreland CJ, Puscas L, Regan L, Stephens MB, Roberts S, Edgar L. Strengthening interpersonal and communication skills assessment through harmonized milestones. Accreditation Council for Graduate Medical Education. Accessed November 15, 2026.
- Internal Medicine Milestones, second revision. Accreditation Council for Graduate Medical Education. 2020.
- Pediatrics Milestones, second revision. Accreditation Council for Graduate Medical Education. 2021.
- Family Medicine Milestones, second revision. Accreditation Council for Graduate Medical Education. 2019.
- Emergency Medicine Milestones, second revision. Accreditation Council for Graduate Medical Education. 2021.
- Surgery Milestones, second revision. Accreditation Council for Graduate Medical Education. 2019.
- Pediatric Critical Care Milestones, second revision. Accreditation Council for Graduate Medical Education. 2023.