Anonymized report from one partner health system, August 2026
What one health system learned from 82 practice conversations
From November 2025 to August 2026, residents, fellows, attendings, and nurse practitioners at one partner health system spoke with AI patients and family members on ClinicalSim. We scored every transcript against the system's own patient experience frameworks and quoted the clinician's words under every score.
This is the leadership view of those encounters. The health system and its framework names are withheld, ClinicalSim staff and outside users were removed from the data, and no clinician is named. The partner approved this anonymized version for publication.
- recorded practice conversations analyzed
- 82recorded practice conversations analyzed
- clinicians, anonymized
- 68clinicians, anonymized
- of the health system's own frameworks applied
- 3of the health system's own frameworks applied
The finding
Clinicians open these conversations well, and the close is where to practice next
All three frameworks point the same way. On the general communication model, scored on all 82 conversations, Listen averaged 3.83 and Explain 3.79, while Inquire what matters most averaged 3.67, with 18% of conversations below 3. The two service recovery models show the same shape on far fewer conversations: hearing the family averaged 4.22 (n=9) and 4.15 (n=13), and thanking the family for raising a concern averaged 2.00 (n=11).
The transcripts show clinicians listening. The practice opportunity is saying so out loud: reflecting the concern back, checking it was understood, asking what matters most, and thanking the family for raising it. Those are the steps that let a family leave knowing they were heard, and they can be taught and practiced.
| Element | Mean (1 to 5) | n | Below 3 |
|---|---|---|---|
| Receiving and explaining | |||
| SR-BHear | 4.22 | 9 | 0% |
| SR-AHear | 4.15 | 13 | 8% |
| CommListen | 3.83 | 82 | 17% |
| CommExplain | 3.79 | 81 | 6% |
| Confirming and asking | |||
| CommInquire what matters most | 3.67 | 82 | 18% |
| SR-BLearn (reflect the concern back and confirm) | 3.33 | 9 | 22% |
| SR-AThanks (for raising the concern) | 2.00 | 11 | 73% |
- Comm:
- the health system's general patient communication model, applied to all 82 conversations
- SR-A:
- its earlier service recovery model
- SR-B:
- the service recovery model replacing SR-A
Each element was scored 1 to 5 with a written rationale and a verbatim quote as evidence. "Below 3" is the share of scored conversations under the bar.
What strength sounds like
The skill is already inside the cohort
The strongest conversations come from the same clinicians. Every line below is verbatim from a recorded simulation, and no clinician is named.
"Does that kind of make sense?"
Clinician
Goals of care. The clinician had just reflected the mother's account back to her, the smile she had seen fade and the six PICU admissions, and asked whether the summary was right. The reflection is the skill; the question confirms it.
"that's exactly it."
The AI parent, in reply
Goals of care
"Nobody's said it like that before."
An AI parent in another encounter
Another conversation from the same cohort
What each participant sees
Feedback tied to a sentence the clinician said
Each clinician reads scored, quoted feedback right after the conversation, before any group report exists. These two examples are on one element, SR-B Learn: reflect the concern back and confirm understanding. Each shows the clinician's words and a summary of the feedback on them.
What went well
5/5
"It sounds like the Eli that you know and love is the one who's smiling at you, the one who is happy, the one who is always interactive with you."
Feedback summary: The clinician reflected the mother's account back with real specificity, checked understanding, and drew out her goals: comfort at home, being "his mom, not his nurse."
Feedback summarized from the report
What to practice next
2/5
"Is there anything we can do to help minimize his suffering in your mind?"
Feedback summary: The mother had just said she wonders whether Eli is suffering more than living. The question is a caring one. The next step is to say her words back to her and check they landed before moving on, because that is how a family learns they were heard.
Feedback summarized from the report
Because the score points to the clinician's own words, the clinician can see exactly what to change and run the case again right away. Open a complete participant report, transcript, and recording.
Scenarios and scope
Which conversations, and where each framework applied
The two service recovery models assume a family has raised a concern, complaint, or service failure. So before scoring, each conversation was judged in or out of scope for each framework, with the reason recorded, and no clinician was marked down for a behavior the conversation gave no chance to show. The general communication model applied to all 82.
First conversation with a parent after a drowning
46 conversations
The clinician makes first contact with a mother during her son's active resuscitation, before she has been told anything definite.
In scope: Comm 46/46, SR-A 1/46, SR-B 0/46
General pediatrics goals of care, before a lecture
28 conversations
The resident leads a meeting with the exhausted mother of a 7-year-old with severe cerebral palsy and six PICU admissions in the past year.
In scope: Comm 28/28, SR-A 12/28, SR-B 8/28
Informed consent for a central venous catheter (PICU)
3 conversations
A PICU fellow obtains consent from an anxious mother and uses teach-back to confirm she understood.
In scope: Comm 3/3, SR-A 0/3, SR-B 0/3
General pediatrics goals of care, after a lecture
2 conversations
The same case with the mother's key worry removed from the briefing, so the resident has to draw it out.
In scope: Comm 2/2, SR-A 0/2, SR-B 1/2
Unlabeled early sessions
2 conversations
The same drowning case, recorded before scenarios carried IDs.
In scope: Comm 2/2, SR-A 0/2, SR-B 0/2
Asthma discharge counseling, limited English proficiency
1 conversation
A mother with limited English proficiency says okay even when she doesn't understand, because she is embarrassed to ask questions in English.
In scope: Comm 1/1, SR-A 0/1, SR-B 0/1
Total in scope: Comm 82/82, SR-A 13/82, SR-B 9/82
What this report does not show
- Service recovery results rest on 9 and 13 in-scope conversations, because the current cases rarely give a family a reason to raise a concern. Those means shouldn't carry strong claims on their own. The general communication model, scored on all 82, can.
- Scenario mixes differed by cohort, so nothing here compares units or programs.
- A simulation is a proxy for the bedside, the same proxy in-person simulation already relies on.
- Consistency between model runs is not the same thing as agreement with expert human raters. Faculty-rater validation on a customer's own rubric is the gate before any high-stakes use, and that work has not been done.
- ClinicalSim output is formative. It is evidence that informs program judgment, and it is not intended for employment, credentialing, privileging, licensure, or other high-stakes decisions.
- ClinicalSim does not predict patient experience scores, readmissions, safety events, claims, or other clinical or business outcomes. Any comparison with institution-held outcome data requires a separate study plan.
What a next round looks like
From one report to a second round of practice
A retrospective report shows what the practice conversations contain. A second round tests whether practice changes the scores and whether faculty agree with them.
Confirm the standard
Agree on the framework or policy the institution wants to use. Rescoring every conversation against a revised rubric is quick, so the standard can be settled before the round begins.
Run the same cases with one defined group
Everyone in the group practices the same scenarios. Add cases in which a family raises a concern, since the current library gave a family that chance in only 9 of 82 conversations.
Return feedback to each participant
Each person gets the score, the transcript excerpt behind it, written feedback, and one or two things to practice next.
Practice again and compare
Participants repeat the cases, and the report compares the group's later attempts with its first ones without ranking individuals.
Add faculty validation
Faculty independently score a sample of the same transcripts, and agreement with the platform is measured before any broader use.
See how ClinicalSim works from the first case to the next round, or read how an engagement is structured.
Bring us the conversation your institution wants to understand
Tell us which standard your teams already teach. We'll show you what a report like this one could cover for your group.