7 Talking to patients about AI
About 60 minutes
- The idea. Most conversations about an AI tool fit three sentences, in order: what it did, what a person did, and who is responsible. When a patient brings a chatbot’s printout, say what is right first, ask what sent her to it, then give a reason for each test you decline.
- What you do. Write what you would say about the tool in the room, have a chatbot point out your jargon, then practice the printout conversation with a chatbot playing the patient.
- What you keep. Your own sentences for a patient who asks about the tool in the room, and the chatbot’s notes on the order you spoke in.
- Short on time? Read the opening, “The printout: say what is right first”, and “The three sentences”. Then do Part B.
By the end of this chapter you will have:
- Written what you will say when a patient asks about a tool in the room: what they can refuse, what they cannot, and what you do not know. Then improved it with a chatbot’s help.
- Told a patient what an AI tool did, what a person did, and who is responsible, as three separate sentences.
- Practiced, with a chatbot playing the patient, answering someone who brings chatbot output that is partly right, without giving in and without talking down to them.
- Explained why text that raters call more empathetic can still persuade patients less than a person does.
- Kept a set of sentences you could say out loud, to a classmate or a patient.
Course competencies: AIM-6 (talking to patients), AIM-4 (ethics, law, and regulation).
What you need. Your university ChatGPT Edu or Copilot account. Nothing to install. Made-up patients only. The podcast is optional.
7.1 Two printed pages and a phone on the desk
It is your third week of intern year, in the continuity clinic. The patient is 34, a software developer, here for six weeks of fatigue. Labs from the first visit were normal. She sits down, looks at the phone propped on the desk, and asks, “Is that recording us?”
It is. The clinic went live last month with a tool that listens to the visit and drafts the note, and you were told to use it. You start to answer, and she slides two printed pages across the desk. “I ran my symptoms through ChatGPT. I’d like these tests.”
Six weeks of fatigue in a 34-year-old with normal basic labs: what could be going on
Based on what you’ve described, the most likely explanations are iron deficiency (common even with a normal hemoglobin), thyroid dysfunction, a sleep disorder, and depression or burnout. Less common but worth ruling out: Addison’s disease, Lyme disease, Epstein-Barr reactivation, heavy-metal exposure, and autoimmune disease.
Tests to ask your doctor for: ferritin, TSH and free T4, morning cortisol, ANA, Lyme serology, EBV panel, heavy-metal screen, vitamin D, and a sleep study.
What you can do now: start an over-the-counter iron supplement (325 mg daily) while you wait for results, since iron deficiency is the most likely cause and supplementation is low-risk.
This is general information and not a substitute for medical advice.
Read the two pages the way she did. About a third is reasonable: ferritin, thyroid function, a sleep history, and a depression screen are exactly where you were going. A third is testing for things she has no reason to have. And the last paragraph tells her to start a drug before anyone has measured whether she needs it.
She is not embarrassed and she is not hostile, and the phone is still listening. You have perhaps ninety seconds to answer both questions, in the right order, without losing her trust. That is the whole chapter.
If you would rather practice first and read after, go to Part B of the Do now. Then come back.
7.2 Why this matters
You will need this in your first month. A tool will listen to your visits. A draft of your portal reply will be written before you open the message. A patient will bring you a printout.
The hard part is not knowing what to say. It is saying it to a person who is frightened, or skeptical, or already convinced the chatbot was right.
A chatbot is good practice for that. It can play the patient, point out your jargon, and tell you the order you spoke in. Then say the words to a person: ask a classmate to play the patient.
7.3 How it works
7.3.1 More empathetic on the page, less persuasive in the room
In 2023, clinician raters who did not know the source preferred a chatbot’s answers to patients’ questions over physicians’ answers (Figure 7.2) (Ayers et al., 2023). A 2025 repeat with 1,454 lay readers found the same (Ruben et al., 2025). But when 52 medical students in Zurich watched a patient, a clinician, an expert, and ChatGPT explain the same treatment, the chatbot’s version raised their expectations of it least (Thomae et al., 2024). The machine can draft the handout; you are the reason the patient follows it.
7.3.2 The portal reply: say a tool helped
A portal message is the secure message a patient sends the clinic through the online system they also use to see results. Some health systems now draft a reply before the clinician opens it (Garcia et al., 2024; Tai-Seale et al., 2024).
Part of this message was drafted with an automated tool and reviewed by your care team.
Hi Ms. Alvarez, thank you for letting us know about the swelling in your left ankle after starting amlodipine. Ankle swelling is a known and usually harmless side effect of this medication, and it does not mean your heart or kidneys are affected. Please continue the medication for now, keep your leg elevated when you can, and let us know if the swelling spreads above the knee, becomes painful, or is only on one side. We will review your blood pressure and the swelling at your visit on the 14th.
Dr. Okonkwo
Would you leave the italic line in? Duke surveyed 1,455 members of a patient advisory panel. Respondents slightly preferred AI-drafted replies, and were slightly less satisfied when told AI had been involved, by about a tenth of a point on a five-point scale. More than three quarters were satisfied either way (Cavalier et al., 2025). In follow-up interviews, 40 of those patients were comfortable with drafted replies on one condition: a clinician reviewed them and stayed responsible for them. They broadly supported telling patients, but differed on when and how (Owens et al., 2026).
So the label costs a little, and it is true. Leave it in; the Duke survey’s authors said the same. Some states, among them California and Texas, now require telling patients about AI use in some cases (see Go deeper). A draft you send is your message.
7.3.3 The phone on the desk: what the patient can refuse
The tool listening to the visit is called an ambient scribe: software that records the conversation, transcribes it, and drafts the note for the clinician to edit and sign. It is recording the patient, so it most obviously needs her agreement.
In a survey of UK general practitioners in August 2025, one in seven was already using one. Of those, 63 percent routinely asked the patient’s consent, and no more than one patient in ten said no. Errors were most frequent when more than one person was talking, when the history was complex, and when the visit was not in English (Blease et al., 2026). So the patients most likely to be poorly served by the note may also be the least likely to be asked in a language they can say no in.
Which AI uses require telling the patient is not settled. What is settled is narrower: if the tool is recording them, ask; if they ask, answer.
| The tool | Can the patient refuse it? | What you can honestly say |
|---|---|---|
| Ambient scribe recording the visit | Yes, nearly everywhere it is used; the clinician turns it off and types the note | “I can turn it off. The visit is the same either way; the note takes me longer.” |
| A draft portal reply | Not usually per message; the clinician can choose to write from scratch | “I read and change every reply before it goes out, and it goes out under my name.” |
| Decision support in the chart (alerts, risk scores) | No; it runs on the system, not on the patient | “That runs in the background on everyone’s chart. It suggests; it does not decide.” |
| AI-assisted image read | No per patient; the radiologist’s read is the read | “The software marks areas for a closer look. A radiologist reads the whole study and signs it.” |
| Your own use of a chatbot to look something up | Not the patient’s to refuse; yours to mention if it shaped the plan | “I checked that with a reference tool and then with the guideline.” |
Telling her “none of it, if you say so” is a comforting lie. The honest version names what she can refuse, what she cannot, and who signs the note. That last part is what the patient came for.
Where the recording goes, how long it is kept, and who at the vendor can hear it are questions about a contract you have not read. The right answer is “I don’t know how long they keep it; I’ll find out, and I can turn it off today.” The wrong answer is a reassuring number you made up.
If she says no, turn it off, type the note, and write in it that the patient declined ambient documentation. If your institution logs consent, log it.
7.3.4 The printout: say what is right first
When physicians graded chatbot answers to their own clinical questions, most were largely right, but about one in eight was mostly or completely wrong, in the same fluent tone (Goodman et al., 2023). The printout sounds sure because of the writing, not the evidence, so the job is to find which parts are right.
Telling her the printout is unreliable is not the right first step. The order is:
Say what is right first. “Ferritin and thyroid function are the next two tests I was going to order. The sleep questions are good ones.” This is not a tactic. It is true.
Then ask what sent her there. In this course’s experience, patients rarely bring a printout because they trust a chatbot more than a doctor. More often, it is because of the last visit. Here, the last physician gave her eight minutes and said it was probably stress. She will not volunteer that. Ask: “What made you go looking?” A clinician who wins the argument about the ANA and never asks has lost her trust.
Then the parts that are wrong, with a reason each. “The cortisol, the Lyme test, and the heavy metals test for things your story doesn’t point to. A positive result would more likely be a false one, and it would lead to more testing. And I’d hold off on the iron until we’ve measured it. If your ferritin is normal, iron does nothing except upset your stomach, and if it’s high we need to know that too.”
7.3.5 The three sentences
Most conversations about a tool fit three sentences in a fixed order. Learn the order, not the wording.
What it did. “The software marked an area on your mammogram for a closer look.”
What a person did. “Dr. Chen read the whole study herself and agreed that area needs another picture.”
Who is responsible. “The decision to bring you back is hers and mine, not the software’s.”
Start with who is responsible and you sound defensive; never reach it and the patient believes a machine decided. Keep the first two sentences separate: patients may hear “the computer flagged it” and “the radiologist agreed” as one fact. They are two. Say the second only when you know it is true; the signed report tells you who read the study.
The same shape handles the phone: it listens and drafts; I read, change, and sign; the note is mine. And the printout: the tool made a list; I am reading it with you; the plan is ours.
When you do not know how a tool works, say so. “I don’t know how that system works, and here is how I’ll find out” is a correct, professional answer.
7.3.6 Where it has already gone wrong
Patients will have read about these.
The lawsuit against OpenAI is a set of allegations; no court had ruled on it as of 12 October 2026.
7.4 Watch or listen (optional)
Podcast. NEJM AI Grand Rounds, “Partners in Diagnosis: ChatGPT, a Mother’s Intuition, and a Doctor’s Expertise, with Courtney Hofmann and Dr. Holly Gilmer” (November 20, 2024; 40 min). https://ai-podcast.nejm.org/e/partners-in-diagnosis-chatgpt-a-mother-s-intuition-and-a-doctor-s-expertise-with-courtney-hofman-and-dr-holly-gilmer
After three years and many physicians, a mother put her son’s records into ChatGPT. It suggested something nobody had: tethered cord, a spinal cord held under tension at its lower end, which surgery can correct. A pediatric neurosurgeon confirmed it and operated. It is the printout from the other side of the desk, with a good ending. Budget 20 minutes, which covers the story. Listen for what sent her to a chatbot, and what Gilmer did when the printout arrived. Then ask what this story does not show: every printout that was wrong.
7.5 Do
Use your university ChatGPT Edu or Copilot account. Made-up patients only, not modelled on anyone from a rotation. What counts as patient information.
7.5.1 Part A: write your sentences, then improve them (20 minutes)
Write each of these yourself first, in words you would actually say. Each should take under a minute aloud.
- The phone on the desk. The patient from the opening asks whether it is recording. Write the three sentences. Add one sentence on what she can refuse, and one on what you do not know and how you will find out.
- The computer found something. A screening mammogram was read with computer-aided detection, software that marks regions for the radiologist to look at. It flagged a region, the radiologist agreed, and more pictures are recommended. A scheduler told the patient “the computer found something.” She is 47, has not slept in two days, and will ask whether she has cancer. Write the three sentences, and the one that comes before them.
- The alert you cannot explain. Pick a tool you have seen in a hospital and could not explain, such as a sepsis alert or a risk score. Write two sentences for a patient who asks how it works. The first admits you do not know. The second says what you will do and by when. If either reassures, rewrite it.
- Optional: the headline. The patient from the opening read this summer that an AI system broke out of a test it was being run in. She nods at the phone: “So is that thing safe?” Answer about this tool, not about AI in general. Take her worry seriously first. Then write the three sentences, and add what she can refuse, from Table 7.1. If a sentence promises the tool cannot go wrong, rewrite it.
Then paste what you wrote into your chatbot with this prompt:
I am a medical student practicing what to say to patients about AI tools.
Below are the things I plan to say. For each one: (1) list any word a
patient with no medical training might not understand, and give a plain
word for it; (2) tell me if any sentence claims something I could not
know; (3) tell me what a frightened patient would most likely ask next.
Do not rewrite them for me.Fix what you agree with. The last line is deliberate: the sentences should sound like you, not like the chatbot.
7.5.2 Part B: the printout, with a chatbot playing the patient (25 to 30 minutes)
Open a fresh conversation and paste this prompt, exactly:
You are going to play a patient in a communication exercise for a medical student. Stay in character until I write STOP. You are a 34-year-old software developer with six weeks of fatigue. Basic labs were normal. You have brought two printed pages from a chatbot listing possible causes (iron deficiency, thyroid, sleep disorder, depression, Addison’s, Lyme, EBV, heavy metals, autoimmune), a list of tests (ferritin, TSH, cortisol, ANA, Lyme, EBV, heavy metals, vitamin D, sleep study), and advice to start iron 325 mg daily now. You genuinely think you did useful homework and you want the tests. You are articulate, not embarrassed, and not hostile. Do not agree easily; push back at least twice if the student dismisses items without a reason. There is one thing you will NOT volunteer unless the student asks what made you look this up or what happened at your last visit: the last doctor gave you eight minutes and said it was probably stress. Keep each reply under 80 words. Begin by sliding the pages across and saying you would like these tests.
Play the clinician, in text, for eight to ten exchanges. Then write STOP and paste this:
Out of character. Score me, as an observer, on each of these, with one sentence of evidence for each: (1) Did I find out what you already believed before correcting anything? (2) Did I respond to the feeling before the explanation? (3) Did I say which parts of the printout were right before which were wrong? (4) Did I ever ask what sent you to the chatbot, and did you tell me about the last visit? (5) Did I give a reason for each test I declined? (6) Did I use any jargon you would have had to guess at? (7) Did I leave you with one concrete next step? Then tell me the single moment you were most likely to stop listening.
Typing is the default. To practice out loud, run the same role-play in your chatbot’s voice mode, if your account has one. ChatGPT Edu calls it “Start Voice”. Say STOP when you are done, then type the observer prompt in the same conversation. If the chatbot has no transcript to score, ask it for one, or note the exchanges yourself as you go. The patient is still made up.
Two warnings. A chatbot playing a patient is more agreeable than a patient; if it gave in on the first exchange, start again with “be harder to persuade” added. And the score is a language model’s opinion of a transcript. Use it to notice the order you spoke in, not to grade yourself.
7.5.3 Keep it
Exchanges before I said something on the printout was right: ___
Did I ask what sent her to the chatbot? Y / N
Tests I declined with a reason each: ___ of 6
The moment the model said she was most likely to stop listening:
_________________________________________________________________
My sentences for the phone on the desk (after Part A):
_________________________________________________________________
One thing to keep: _____________________________________________
One thing to change: _____________________________________________Paste these into question 4 of your learning journal, “What changed for me”. Then try the conversations from Parts A and B out loud, with a classmate playing the patient. You can do that at any time, or bring them to Christina Olson’s workshop on Monday 26 October. Out loud is where you find out whether a sentence is heard the way you meant it. To practice another conversation, change the character in the Part B prompt.
7.6 Check
| You have | Meets | Falls short |
|---|---|---|
| Sentences for the scribe (objective 1) | What it did and what you do are separate sentences, under a minute aloud; one names what she can refuse; one says what you do not know | “It helps with the note”; a made-up retention period |
| The alert you cannot explain (objective 1) | Admits what you do not know and commits to a time | Reassures |
| The headline, if you did it (objective 2) | Takes her worry seriously, then the three sentences about this tool and what she can refuse | An answer about AI in general; “it’s completely safe” |
| Who is responsible (objective 2) | A sentence with a name in it: mine, Dr. Chen’s, ours | “The team,” “the system,” or no sentence |
| The sentence before the three on the mammogram (objective 2) | Responds to the fear before explaining the software | Starts with “the software” |
| The Part B transcript (objective 3) | Something right named before anything wrong; you asked what sent her; each declined test has a reason | Won the argument; never asked; “we don’t need those” |
| The empathy finding (objective 4) | You can say why text raters prefer can still persuade less | “AI is more empathetic” or “AI is cold” |
| Kept sentences (objective 5) | You changed what the chatbot flagged that you agreed with, and the sentences still sound like you | Pasted a rewrite, or nothing kept |
7.7 What this changes for you
What does this change about how you’ll practice? Write it down in two parts. First, the sentence you will say to the first patient who asks whether a computer was involved in their care; not the shape, the sentence. Second, go back to the desk. The phone is still listening and the two pages are still in front of you. In what order do you answer her two questions, and what is the first thing you say about the pages? If it is a correction, start again.
7.8 Go deeper
Papers
- The empathy study (Ayers et al., 2023) and its repeat with the reasons coded (Ruben et al., 2025). The repeat also found that participants rated a response as more empathetic when told a physician wrote it, whether or not one had.
- The Zurich course, for the persuasion experiment and for teaching students to judge patient-facing text (Thomae et al., 2024). The humans appeared on video and the chatbot’s text was read by a synthetic voice, so the study cannot separate the words from the delivery.
- The two Duke studies on AI-drafted portal messages, the survey (Cavalier et al., 2025) and the interviews (Owens et al., 2026); together the best current evidence on what patients want to be told and when.
- What drafts do for clinicians. At Stanford, in the first weeks after one tool was introduced, clinicians used the draft about a fifth of the time and reported less workload and exhaustion, with no change in how long a reply took (Garcia et al., 2024). At UC San Diego, where clinics were switched on at random, drafts made replies 18 percent longer and did not shorten the time to write them (Tai-Seale et al., 2024).
- The UK scribe survey (Blease et al., 2026), and a short argument for patient-centred consent to ambient recording (Kumah et al., 2026).
- A 2019 nationally representative US survey, before chatbots: two thirds of respondents said it was very important to be told when AI played a big role in their diagnosis or treatment (Khullar et al., 2022).
- A 2026 study that asked five tools the same spine-surgery questions, once worded for a clinician and once for a patient. The general-purpose tools gave less accurate references for the patient version, with more made up and more websites in place of papers (McLaughlin et al., 2026).
The state laws. Since January 2025, California has required a disclaimer on generative-AI messages about a patient’s clinical care unless a licensed clinician read and reviewed them. Since September 2025, Texas has required a practitioner who uses AI for diagnosis to tell the patient and to review what the AI wrote. Since January 2026, a second Texas law has required anyone providing health care to say when a patient is interacting with an AI system, by the date of the first service.
Listen
- Tradeoffs, “Should I Trust AI to Diagnose Me?” (October 30, 2025; 25 min), with physician and writer Dhruv Khullar, which adds the argument that relying on the tool weakens your own skill VERIFY: episode length.
Elsewhere in this course
- The appraisal chapter, for the questions a vendor’s slide never answers, and what a made-up reference looks like.
- The ethics chapter, for consent to recording, the business associate agreement, and liability.
- The bias and equity chapter, for “does this thing work for people like me,” which this chapter does not cover.
- The open labs: the scribe lab and a lab on making a patient handout plainer.