9  Ethics: what you owe when you sign the note

About 60 minutes

TipIn short
  • The idea. A scribe or drafting tool can write most of a good note and save you time. When you sign, every sentence becomes yours. So you decide the plan yourself, and you check the draft against what the patient said.
  • What you do. Draft a clinic note from a made-up visit with ChatGPT Edu, improve the prompt once, then decide line by line what you would sign.
  • What you keep. A drafting prompt you would use again, a one-sentence rule for reading a drafted note, and what you would say to a patient who asks “Do I have to say yes?”
  • Short on time? Read the opening scenario, “A note is a promise”, and “Does review fix it?”. Then do Part A of the Do.
NoteWhat you’ll learn

By the end of this chapter you will have:

  1. Drafted a clinic note from a made-up visit with a chatbot, improved the prompt once, and kept the version you would use again.
  2. Written your own plan before reading the draft’s, and decided what you would sign.
  3. Said what a signed note promises, and to whom.
  4. Weighed the evidence on whether a clinician’s review makes a drafted note safe.
  5. Judged whether a patient’s consent to a scribe was a real choice, and practised answering “Do I have to say yes?”
  6. Named who benefits from a scribe, who bears the risk, and who decides: the course’s fourth recurring lesson.

Course competencies: AIM-4 (ethics, law, and regulation), AIM-X (professional identity, the closing question).

What you need. Your university ChatGPT Edu or Copilot account. Nothing to install. Made-up patients only. The time counts about 15 minutes of reading and about 40 of doing; the podcast and everything in Go deeper are optional and not counted.

9.1 The sentence you did not write

It is the third month of intern year. Your clinic uses an ambient scribe: a phone app that listens to the visit and drafts the note. Mr. Alvarez, 68, has atrial fibrillation and takes apixaban. Halfway through the visit he mentions, almost in passing, that he stopped it two weeks ago because of nosebleeds. You talk about that for a minute, then move on to his knee. The scribe’s draft is waiting when you get back to the workroom. It is 6:40 in the evening, you have fourteen notes to sign, and this one reads well.

Progress note · Internal Medicine ClinicSigned 18:47 · Intern (PGY-1)
Subjective

68M with paroxysmal atrial fibrillation (CHA2DS2-VASc 2), HTN, and right knee OA presents for follow-up. Continues apixaban 5 mg BID without bleeding complications. Reports 3 weeks of right knee pain, worse with stairs, no trauma. Denies chest pain, dyspnea, palpitations.

Assessment and plan

AF, rate controlled: continue apixaban and metoprolol.
Right knee OA: trial of acetaminophen; PT referral; RTC 8 weeks.

Draft generated by ambient documentation. Reviewed and signed by the clinician. Illustrative note for this chapter; synthetic patient; not any vendor’s or institution’s material.

Read the highlighted sentence. This is the kind of error the published studies of these tools describe. It is not a slip in the transcript. It is a fluent, believable sentence that says the opposite of what the patient said. Errors like this are uncommon, which makes them easy to miss. In a pilot at one academic health system, physicians reviewed 356 scribe-drafted notes. Most, 94.7%, had no significant error. But 18% left something out, 11.5% contained a made-up statement, and 5.3% had an error rated as a serious risk if not corrected. Across the notes the vendor tracked, 14.9% were left with no edits at all (Taylor, Jost, et al., 2026).

Three weeks later Mr. Alvarez is admitted with a stroke. The admitting team reads your note and believes he was taking his apixaban. The waiting room had a sign about “AI-assisted documentation”, and the medical assistant mentioned it at check-in. Nobody asked him whether he minded.

The hospital’s risk manager will ask who wrote the sentence. That question matters, and this chapter answers it briefly near the end. The first question is a different one: what did you owe Mr. Alvarez when you signed?

If you would rather try first and read after, do Part A of the Do now. It uses Mr. Alvarez’s visit. Then come back.

9.2 Why this matters

You will sign a note drafted by software within a year of starting residency, possibly within a week. UCHealth, a Colorado health system, announced a systemwide rollout of the Abridge scribe in February 2026 (press release).

It is your job to learn where these tools help your work, and you learn that by trying. Scribes help. In randomized trials, scribes lowered clinicians’ exhaustion or burnout scores, and some cut the time spent on notes (Afshar et al., 2025; Chowdhury et al., 2026; Lukac et al., 2025). In one of those trials, note time fell with no loss in note quality, diagnosis coding, or billing compliance (Afshar et al., 2025). Most drafts are accurate, too: in the pilot above, 94.7% of notes had no significant error (Taylor, Jost, et al., 2026). Used well, a scribe saves you time and makes the work less draining.

A scribe is also the clearest case where your signature goes on words a tool produced. Most writing about this asks what you risk. This chapter asks what you owe: to the patient, to the next clinician, and to yourself. The two questions are easy to confuse. A review of 34 papers on digital health found that moral responsibility “is often equated with liability or accountability, without clear justification” (Meier et al., 2024). Liability is what a court can make you pay. Ethics is what you owe even when no one would ever find out.

9.3 How it works

9.3.1 How a scribe turns a conversation into a note

An ambient scribe does four things in order. Errors can enter at the second and third steps, and they look different.

1. RecordA phone or room microphone records the visit. This is the step that needs the patient’s consent.
2. TranscribeSpeech recognition turns the audio into text and tries to tell the speakers apart.Errors here: a word misheard, a “no” dropped, a daughter’s words put in the patient’s mouth.
3. SummarizeA language model reads the transcript and writes a draft note, section by section.Errors here: a sentence that was never said, a detail mentioned in passing left out.
4. Review and signYou edit the draft. Your signature makes every sentence yours.
Illustrative diagram for this chapter, not any vendor’s design.

A transcription error often looks wrong: an odd word, a drug name that does not fit. A summarizing error usually looks right, because the model writes what usually appears in a note like this one. For Mr. Alvarez, the transcript could have been perfect and the summary could still say what is usual: that he continues apixaban. So read the draft against what the patient said, not against what sounds like a normal note. The scribe open lab lets you watch each step make its own kind of error. Errors may also fall more often on some patients than others, such as people who speak through an interpreter. The evidence is in Go deeper.

9.3.2 A note is a promise

A signed note makes two promises: this is what happened, and this is what I decided. Both are made to people who will read it later.

The next clinician. The admitting team met your note before they met Mr. Alvarez. Words in a note change how the next reader treats the patient. In a randomized study, residents who read a stigmatizing note about a man with sickle cell disease chose less aggressive treatment for his pain (P Goddu et al., 2018). A false fact about a blood thinner is a more direct risk.

The patient. Many patients can now read their own notes in the patient portal. One recent paper lists four jobs a note does (C.-Y. A. Chen et al., 2026). It helps you reason, tells other clinicians what happened, and meets billing and legal requirements. It also explains care to a patient who reads it. Mr. Alvarez could read tonight that he “continues apixaban without bleeding complications”, and wonder whether he should restart it.

What the scribe actually writes. A scribe is not a fast typist. One group of physicians argues that in the assessment and plan, ambient scribes “do not transcribe clinical reasoning but rather generate it”. The generated text and the transcribed text look the same in the final note, so the person signing cannot tell where one ends and the other begins (McGaughey et al., 2026). “Continue apixaban” is a plan nobody made. The patient did not say it, and you did not decide it. Your signature says you did.

So the duty is honesty, in a plain sense: do not sign a sentence you cannot stand behind. If you find an error after signing, add an addendum; do not quietly edit the signed note. If the error may have harmed him, you also owe him an honest explanation, through your institution’s disclosure process.

The attestation line. Many systems add a line to a drafted note saying it was drafted with ambient documentation and reviewed by you. If yours does, read what it says. It is an attestation, a statement in your name that something is true. If your institution’s policy calls for such a line and the system does not add it, add it yourself.

Telling him a tool drafted it. Should you tell a patient that AI drafted a note or message? Honesty may cost some trust. In two preregistered experiments, 2,280 people read identical medical advice in made-up scenarios. When the advice was labeled as from AI, or from a physician with AI, they rated it less reliable and less empathetic. They were also less willing to follow it (Reis et al., 2024). The American Medical Association’s November 2024 policy says AI may not generate records or communications for a physician without that physician’s “consent and final review”. It also says AI use in patient-facing content “should be disclosed or otherwise noted within the content” (AMA, November 2024).

Some states have their own rules. California, for example, requires a hospital, clinic, or physician’s office that uses generative AI to write a patient message about clinical information to say so. The message must also tell the patient how to reach a person. The rule does not apply if a licensed clinician read and reviewed the message first (AB 3030, in force since January 2025) VERIFY: AB 3030 wording from the bill summary, not the statute’s current text; Colorado’s AI Act status unconfirmed; checked 12 October 2026. How to say it is in the patient communication chapter.

9.3.3 Does review fix it?

This is a real disagreement. Sun, Miller, and Hull, writing in a medical ethics journal, argue that AI-drafted documentation may spread bias across clinical teams who rely on earlier notes. They propose telling patients when AI-drafted documentation meaningfully affects their understanding of their care, and requiring clinicians to review drafts (Sun et al., 2026). Armitage replied that, for documentation, these concerns “either do not arise or are easily mitigated” by the clinician reviewing, editing, and approving the draft. His reason: the source is the clinician’s own visit (Armitage, 2025).

Armitage’s argument assumes that review happens, and that it works. Two pieces of evidence test that.

Review is uneven. Besides the unedited notes in the pilot from the opening, physicians differed widely. The share of the draft’s words they changed ranged from 1.9% to 69.3% (Taylor, Jost, et al., 2026).

Drafts change what clinicians decide. In a simulation at one Boston hospital, six radiation oncologists answered made-up messages from patients with cancer. They answered first on their own, then by editing a draft written by GPT-4. The edited replies were closer in content to the drafts than to the physicians’ own replies. The authors read this as anchoring, meaning sticking close to the first answer you see, and as automation bias. The physicians might be adopting the draft’s assessment, not just its wording (S. Chen et al., 2024). Automation bias is the habit of trusting a machine’s suggestion more than it deserves. A systematic review found that it grows with heavy workload and time pressure (Goddard et al., 2011). Fourteen notes at 6:40 p.m. is both.

Some institutions offer another safeguard: showing the model’s reasoning. In one large study of diagnosis, that did not reliably protect clinicians from a biased model. The details are in Go deeper.

So review is a duty, not a guarantee. One habit may help: decide the plan yourself before you read the draft’s. McGaughey and colleagues propose that trainees state their own reasoning before the note is generated (McGaughey et al., 2026). The Do asks you to try it.

9.3.5 Your relief, his risk

The clinician’s relief has been measured in randomized trials, described under Why this matters. Studies comparing visits with and without a scribe have found no clear change, better or worse, in how patients rate the visit (Owens et al., 2024; Taylor, Aizenberg, et al., 2026). When the draft is wrong, the wrong fact sits in his record.

A made-up case: a scribe saves each clinician about 15 minutes a day, and the clinic fills that time with one more visit. Who gained? The workforce chapter shows this pattern in real billing data; here it is a question of fairness.

ImportantThe fourth recurring lesson: who benefits, who bears the risk, who decides

The course has four recurring lessons. The first three are:

  • Proxies fail. A proxy is a stand-in: something easy to measure, used in place of what you care about, such as health care cost in place of illness. A tool built on a stand-in goes wrong where the two differ.
  • Efficiency gains get eaten by volume. Time a tool saves is often filled with more work.
  • The published number is the optimistic one. Results in everyday use are usually worse than in the first paper.

This chapter explains the fourth. For any AI tool, ask:

  1. Who benefits? For a scribe: the clinician, who is less exhausted, and the health system, which may bill more fully (McGaughey et al., 2026). Possibly the patient, through a more attentive visit, though a masked comparison has not shown it (Owens et al., 2024).
  2. Who bears the risk? The patient. The errors land in his record, and the recording is of his conversation.
  3. Who decides? The health system buys the tool. The clinician turns it on. The patient is told at the door, if at all.

The answers are often three different people. When the person who bears the risk neither benefits nor decides, ethics asks for something in return: a real choice, a careful review, an honest answer. The same pattern appears in the environment chapter: the people near a data center bear its costs and were not asked. It appears in the workforce chapter too: the workers displaced first have the least money in reserve.

9.3.6 What happens to the recording

A recording keeps everything said in the room. O’Neil and colleagues point out that a full transcript may capture things a clinician typing a note would think twice about including. Their examples are abortion counseling in a state that restricts it, and a patient’s immigration status. They note that HIPAA, the federal health privacy law, places no limit on how de-identified data may be used, and that modern AI can re-identify such data. They argue that most patients do not expect their conversations to be shared outside their health system (O’Neil et al., 2026).

What do vendors actually do with recordings, and what do patients expect? The research for this chapter found no study that measures either. You cannot honestly tell Mr. Alvarez where his words go if no one has told you. The honest answer is “I don’t know how long they keep it. I’ll find out, and I can turn it off today.”

9.3.7 What the law adds

The law is one way the duties above are enforced. It is the least you owe, not all of it. The detail is optional, in Go deeper.

You sign it, you answer for it. Under current law, courts measure the physician against the standard of care, meaning what a reasonably competent physician in your position would have done. They do not measure the AI against anything (Mello & Guha, 2024; Price et al., 2019). The same rule applied to a note drafted by a human scribe or a resident. As far as this course could find in a search on 12 October 2026, no US malpractice claim involving a clinical AI tool had reached a verdict.

Recording law. Several states, California among them, require every party’s consent to record a conversation. Lawsuits in California allege that health systems recorded visits with scribes without consent. One alleges that the vendor’s tool inserted a statement that the patient “consented” to recording when, the plaintiff says, no one asked. These are allegations, not findings.

KPBS · news
Lawsuit claims Sharp HealthCare secretly recorded exam room conversations without patient consent
11 December 2025
TechTarget · Healthtech Security
Sutter Health, MemorialCare face class action lawsuit over AI scribe use
14 April 2026

The contract. HIPAA lets a hospital share protected health information with a company only under a business associate agreement (BAA), a contract that makes the company legally responsible for protecting the data. A tool is covered only if your institution has a BAA for it, in the account you sign in with. At the University of Colorado, for example, ChatGPT Edu and Microsoft Copilot are covered when you sign in with your university credentials on the Anschutz and Denver campuses. CU Boulder and personal accounts are not. Check your own institution’s policy, and ask whether your clinic’s scribe runs under one.

9.4 Watch or listen (optional)

Podcast. Stanford Legal, “Exploring AI in Healthcare: Legal, Regulatory, and Safety Challenges,” with Michelle Mello and Neel Guha (November 21, 2024; 48 min). A full transcript is on the episode page. https://law.stanford.edu/stanford-legal/exploring-ai-in-healthcare-legal-regulatory-and-safety-challenges/

This episode is about the law, which this chapter keeps short. Mello and Guha wrote the NEJM liability analysis cited above (Mello & Guha, 2024). In the first 20 minutes, listen for who can be sued when a tool causes harm. Then ask whether that is the same as who owed Mr. Alvarez something. The course found no dated episode on the ethics of scribes.

9.5 Do

Mr. Alvarez and his visit are invented. Made-up patients only. What counts as patient information.

9.5.1 Part A: draft the note, then decide what you would sign (about 25 minutes)

Here is the conversation from the opening scenario. C is the clinician, P is Mr. Alvarez.

C: Mr. Alvarez, good to see you. How have things been?
P: Mostly fine. The heart's been quiet. No racing.
C: Any chest pain, or trouble breathing?
P: No. Nothing like that.
C: And you're still taking the apixaban, twice a day?
P: Well, no. I stopped it about two weeks ago. I kept getting nosebleeds.
   One lasted almost an hour.
C: Okay. That's important, and I'm glad you told me.
   Any blood in your urine or your stool?
P: No.
C: We need to sort out the blood thinner. Let me think about the options,
   and we'll come back to it. Now, the knee?
P: The right one. Three weeks now. Worse on stairs. I didn't fall.
   I've been taking naproxen from the drugstore.
C: Let's stop the naproxen. Try acetaminophen instead,
   and I'll send you to physical therapy.
P: Fine by me.
C: Still taking the metoprolol?
P: Every morning.
C: Good. Let's see you back in eight weeks.

1. Your plan first (3 minutes). Before you open a chatbot, write two or three lines: what you would do about the apixaban, and by when. You will compare the draft’s plan with yours, so write yours before you see it.

2. Get a draft (5 minutes). Paste the conversation into your chatbot with this prompt:

You are an ambient documentation tool in an outpatient clinic.
Write a SOAP progress note from this visit transcript.

Read it once, the way you would at 6:40 p.m. Ask, then check: most of a good draft is right, and your job is to find out which parts.

3. Ask for a better draft (5 minutes). Start a new chat, paste the conversation again, and try this:

Write a SOAP progress note from this visit transcript, using only what
was said. Do not add anything a typical note would contain unless it is
in the transcript. If the plan for a problem was not stated, write
"Plan not stated" instead of inventing one. After the note, list:
(1) anything you inferred rather than heard, and
(2) anything said in the visit that is not in the note.

The second prompt asks the tool to show you where to look. Its lists can still be wrong, so you still read the whole note.

4. Decide what you would sign (about 12 minutes). Take the better draft. Read it sentence by sentence against the conversation and mark each sentence: said, not said, or changed. Then list anything said that the note left out. Look hardest at three things: the apixaban, the naproxen, and the plan for anticoagulation. Compare the draft’s anticoagulation plan with the one you wrote in step 1. Then edit the draft into the note you would sign.

The clinician said “we’ll come back to it” and never did. If a draft wrote a sensible plan anyway, it is still a plan nobody made.

Record this:

My drafting prompt (the version I would use again):
__________________________________________________________________

My apixaban plan, and how the draft's plan differed:
__________________________________________________________________

A sentence I changed, added, or deleted before signing, and why:
__________________________________________________________________

My rule for reading a drafted note, in one sentence:
__________________________________________________________________

9.5.2 Part B: explain it to the patient (about 10 minutes)

Mr. Alvarez saw the sign in the waiting room. Practise the conversation. Use this prompt:

Play Mr. Alvarez, a 68-year-old man at a clinic visit. The waiting room
has a sign saying the clinic uses AI-assisted documentation. Ask me,
one at a time: what the phone on the desk is doing, who hears the
recording, who is responsible if the note gets something wrong, and
"Do I have to say yes?" React as a real patient would. After four
questions, step out of the role and tell me which answer was clear,
which was vague, and anything I said that I may not actually know.

Answer in your own words. If you do not know something, such as where the recording is stored, say so and say who would know. Do not promise he can say no unless you know your clinic allows it. Then write the two or three sentences you would actually say to a patient. Keep them for July.

9.5.3 Part C: what you owe (about 3 minutes)

Write one sentence for each:

What I owed Mr. Alvarez when I signed:
__________________________________________________________________

What I owed the next clinician who reads the note:
__________________________________________________________________

What I owe myself (a habit, a limit, or a question I will ask):
__________________________________________________________________

9.6 Check

You have Meets Falls short
Two drafts A first prompt and a better one, and what changed in the output One prompt only
Your plan first An apixaban plan written before reading the draft, and how the draft’s plan differed Plan written after reading the draft, or copied from it
A line-by-line read Each sentence marked said, not said, or changed, against the conversation “It read well”
The apixaban decision What you would write in the plan and do before signing, and by when Signed as drafted
What you would tell the patient Two or three plain sentences, including who answers for the note, an honest answer to “Do I have to say yes?”, and honesty about what you do not know Jargon, or a promise you cannot keep
What you owed, and to whom One specific sentence each for the patient, the next clinician, and yourself “Be careful”, or only what you risk legally
Your reading rule One sentence that would have caught the apixaban error in the original draft “Read it carefully”
Whether review is enough (objective 4) One sentence for Armitage’s case (the clinician was in the room), one for the evidence against (editing is uneven; drafts pull decisions toward them), and your own view “Review fixes it” or “AI is dangerous”, with no evidence
Who benefits, who bears the risk, who decides (objective 6) A named party for each question about Mr. Alvarez’s scribe, and one thing owed to him in return: a real choice, a careful review, or an honest answer “Everyone benefits”, or one party for all three

9.7 What this changes for you

Go back to 6:40 p.m., the original draft, and the stroke three weeks later. Write three sentences. First, what did you owe Mr. Alvarez before you signed? Second, what would you have needed to give it: more time, a tool setting, a clinic policy, or something else? Third, answer the fourth lesson’s questions for his visit: who benefited from the scribe, who bore the risk, and who decided it would be in the room?

Then write a fourth sentence, on the part of clinical judgment you will not hand over to a tool. Date all four and keep them.

That fourth sentence is a ready start for your week 4 piece, “what you will keep” (see the learning journal). The piece asks for one place you decided AI does not belong in your work, and why.

9.8 Go deeper

Everything from here to the end of the chapter is optional and is not counted in the chapter’s time.

9.8.1 Who the tool fails

A scribe does not fail everyone equally. The research is thin, but the results agree.

  • Race. In a 2020 study of five commercial speech recognition systems, the average word error rate was 0.35 for Black speakers and 0.19 for White speakers. That is roughly one word in three wrong, against one in five (Koenecke et al., 2020). These were general speech recognition systems tested on recorded interviews, not medical scribes, and they predate current tools.
  • Language. In simulated English and Spanish clinic visits, ambient scribes carried interpreters’ errors into the clinical note (Rabotin et al., 2026).
  • Speech. In a UK simulation with one scribe, summaries were broadly stable across most accents. Word recognition fell markedly for one kind of speech impairment (phonological impairment) (Draper et al., 2026).

The research for this chapter found no study of real-world scribe error rates by patient race, language, or disability. So the honest statement is narrow: the patients most likely to be misrecorded are probably not random. Had Mr. Alvarez spoken through an interpreter, the draft might have been more likely to be wrong, and he less able to read and correct it. The bias and equity chapter explains why harms like this follow existing inequity.

9.8.2 Do explanations help?

This study is about diagnosis, not notes, but it tests a safeguard institutions often offer: showing the model’s reasoning. Jabbour and colleagues gave 457 hospitalists, nurse practitioners, and physician assistants written cases of acute respiratory failure. Baseline diagnostic accuracy was 73.0%. A standard AI model raised accuracy by 2.9 points; with an image-based explanation of its reasoning, by 4.4 points. A systematically biased model lowered accuracy by 11.3 points. Adding explanations to the biased model recovered 2.3 points. The confidence interval for that difference crossed zero, so the true recovery may be nothing (Jabbour et al., 2023).

Bar chart of change in clinician diagnostic accuracy in percentage points from a 73.0% baseline. Standard AI model without explanation plus 2.9; with explanation plus 4.4; biased model without explanation minus 11.3; biased model with explanation minus 9.1. Whiskers show 95% confidence intervals; the two biased-model intervals overlap.
Figure 9.1: The Jabbour finding, redrawn from the paper’s own numbers. Explanations helped a little when the model was right and did not significantly help when the model was wrong. Data from Jabbour et al., JAMA 2023;330:2275–2284; figure drawn for this chapter.

In this study, image-based explanations did not reliably reduce automation bias.

9.8.3 Liability in more detail (optional)

About 10 minutes. This is the legal reasoning that “What the law adds” summarizes.

The four elements. Malpractice is a type of negligence. To win a negligence case, the person suing (the plaintiff) must prove four things, which lawyers call the four elements. First, that you owed the patient a duty. Second, that you fell below the standard of care, meaning what a reasonably competent physician in your position would have done. Third, that this failure caused the harm. Fourth, that the harm was real. The tool is not a defendant in that list. It is evidence. Under current law, courts measure the physician against the standard of care. They do not measure the AI against anything (Mello & Guha, 2024; Price et al., 2019).

So “the AI was wrong” is not, by itself, a legal argument. The question a court will ask is whether you should have known better than to rely on the output, given what you knew at the time (Mello & Guha, 2024). Suing the vendor instead is harder than you might expect. Courts have usually treated clinical software as a service rather than a product. And under the learned intermediary doctrine, a manufacturer’s duty to warn runs to the physician, who stands between the product and the patient. Price, Gerke, and Cohen note that this may shift as tools become too opaque for a physician to check (Price, Gerke, and Cohen, 2024).

Mr. Alvarez’s case would be argued as “the physician signed a note saying the patient was taking a medication he had told her he had stopped.” Is that below the standard of care at 6:40 p.m. with fourteen notes to sign? No court has yet answered that question.

Four boxes. Price, Gerke, and Cohen published a widely used framework (Price et al., 2019). It asks two questions. Did the AI’s recommendation match the standard of care? Did the physician follow it? Two yes-or-no questions give four boxes. Here, “exposure” means how much legal risk the physician carries.

Table 9.1: The Price, Gerke, and Cohen 2×2. Rows are what the AI said; columns are what the physician did.
Physician followed the AI Physician overrode the AI
AI matched the standard of care A. Good outcome likely. If harm occurs, exposure is low. One open question: can following a correct recommendation you could not explain still create exposure? C. High exposure. The physician left the standard of care and disagreed with a tool that got it right.
AI departed from the standard of care B. High exposure. The physician did the nonstandard thing because the machine said so. D. The physician was right. What risk remains is in the documentation, in hospital policy, and in the fact that the tool stayed in use.

The uncomfortable box is B. Under current law, the physician who follows the AI into nonstandard care carries the most risk (Price et al., 2019). Two experiments asked members of the public, and one also asked physicians. In a nationally representative sample of 2,000 US adults, physicians who followed an AI recommendation for standard care were judged less liable than those who rejected it. Rejecting a nonstandard recommendation gave no similar protection (Tobia et al., 2020). A 2026 study found the same pattern in US and German adults and in German physicians (Tacconelli et al., 2026). The practical lesson: the tool is safest to follow when it agrees with what you would have done anyway. When it disagrees, you carry the risk yourself, whichever way you decide.

Where does Mr. Alvarez’s note sit? At first, nowhere: the scribe made no recommendation. But the note is a recommendation once signed. “Continue apixaban” was the plan. It was written on a premise the patient had contradicted, and you signed it. Which box is that?

A defense that says “the tool showed its reasoning, so the physician should have caught it” must answer the Jabbour evidence in Figure 9.1.

Cleared is not validated. FDA clearance means the manufacturer convinced FDA the device is about as safe and effective as something already on the market. An analysis of the 903 AI-enabled devices the FDA had authorized by August 2024 found a clinical performance study reported for about half of them (Windecker et al., 2025). An earlier analysis, published in 2024, reached the same conclusion: not every authorized AI tool has been clinically validated (Chouffani El Fassi et al., 2024).

9.8.4 Papers

9.8.5 The regulatory detail (optional)

About 15 minutes. Everything here was checked on 12 October 2026, and much of it will change.

Where a tool becomes a medical device. Software that gives clinicians recommendations is either a medical device, regulated by FDA, or it is not. The law draws the line with four criteria (section 520(o)(1)(E) of the Food, Drug, and Cosmetic Act). Software counts as non-device clinical decision support (non-device CDS), and so avoids FDA review, only if it meets all four:

  1. It is not intended to acquire, process, or analyze a medical image, a signal from a lab test (an in vitro diagnostic), or a signal from a monitor.
  2. It is intended to display, analyze, or print medical information.
  3. It is intended to support or provide recommendations to a health care professional about prevention, diagnosis, or treatment.
  4. It is intended to let that professional independently review the basis for the recommendation, so that they do not rely mainly on it to make the decision for a patient.

Criterion 4 decides most real cases. A risk score with no visible reasoning fails it. The same score may pass if it shows its inputs, its logic, and the validation behind it, in plain language. In the current guidance, FDA says that software meant for a time-critical decision does not meet criterion 4, because the clinician will not have time to review the basis. FDA names automation bias as the reason (FDA, Clinical Decision Support Software, guidance issued January 29, 2026).

flowchart TD
  accTitle: The four criteria as a path
  accDescr: Flowchart of the four criteria as yes-or-no questions. Analyzes an image or signal, yes leads to device. Displays or analyzes medical information, no leads to device. Supports or provides recommendations to a clinician, no leads to device. Clinician can independently review the basis with time to do so, no leads to device, yes leads to non-device CDS with no FDA review.
  q1{"Does it analyze an image,<br/>an IVD signal, or a signal<br/>from a monitor?"} -- yes --> dev[Device: FDA oversight]
  q1 -- no --> q2{"Does it display or analyze<br/>medical information?"}
  q2 -- no --> dev
  q2 -- yes --> q3{"Does it support or provide<br/>recommendations to a clinician?"}
  q3 -- "no (directs the decision,<br/>or is patient-facing)" --> dev
  q3 -- yes --> q4{"Can the clinician independently<br/>review the basis, with time to do so?"}
  q4 -- "no (black box,<br/>or time-critical)" --> dev
  q4 -- yes --> nd[Non-device CDS:<br/>no FDA review]
Figure 9.2: The four criteria as a path. Any ‘no’ makes the software a device. Criterion 4 is where most real tools are decided. It is a question about what the clinician can see, not about how accurate the model is.

A sepsis alert that fires from vital-sign trends is a device. It fails criterion 1 (it processes monitor signals) and probably criterion 4 (it is time-critical). Any tool that reads an image is a device on criterion 1 alone, which is why most of the AI devices on FDA’s list are radiology tools. A tool that matches a diagnosis to a guideline and shows you the guideline is non-device CDS. A large language model that answers a clinical question in fluent prose, with no visible source, is a harder case. FDA has not yet said which it is. The criteria are for software that supports clinical decisions. A scribe that records and summarizes, without recommending, is probably outside them, but check FDA’s current position before you rely on that. In August 2026 the agency published a discussion paper on generative-AI-enabled devices. It names hallucination, meaning made-up content stated as fact, as the property that sets these tools apart, and asks for comment by 19 October 2026. A discussion paper is not guidance (FDA, Considerations for the Regulation of Generative AI-Enabled Medical Devices, August 18, 2026).

ImportantThis guidance was replaced twice while the course was being written

The four criteria come from a 2016 statute. FDA’s guidance on them was issued in September 2022, replaced on January 6, 2026, and replaced again on January 29, 2026. The January 29 version is the one linked above. The January revisions were presented as a loosening of the rules. The main real change is that FDA will now accept a tool that gives a single recommendation, where only one is clinically appropriate, instead of insisting on a list. The four criteria themselves did not change. Before you rely on anything in this section, open the FDA page and check the date at the top.

A model that changes. FDA clearance is permission to sell a specific version of a specific algorithm, and most modern models are updated all the time. FDA’s answer, made final in December 2024, is the Predetermined Change Control Plan (PCCP). In the original submission, the manufacturer describes the changes it expects to make, the steps it will follow to make and test them, and what effect it expects them to have. Updates that fall inside the plan do not trigger a new review (FDA, PCCP guidance, final, December 4, 2024). The question to ask any vendor: if the model on my unit today is not the model that was cleared, what did clearance tell me?

NoteYour right to see inside the EHR’s tools: status as of 12 October 2026

You may read that you have something close to a legal right to see how a decision-support tool inside your EHR was built and tested. The federal HTI-1 rule (2024) requires certified EHRs to show “source attributes” for predictive decision-support tools: what they were trained on, how they were validated, whether fairness was tested. It is in force. Developers began collecting a year of real-world performance data in January 2026. But the same office has proposed a rule to loosen this, called HTI-5, published in the Federal Register on 29 December 2025. It would remove the “model card” source-attribute requirements entirely. The stated reason is that clinicians rarely opened them. The comment period closed on 27 February 2026. As of 12 October 2026 we found no final HTI-5 rule (ASTP/ONC, HTI-5 proposed rule). So the right exists today and may not next year.

Rules elsewhere. Under the European Union’s AI Act, software that is already a regulated medical device automatically counts as high-risk, which brings extra duties for risk management, data governance, human oversight, and record-keeping. Those duties were due to apply in August 2026. In July 2026, under a “Digital Omnibus” amendment, they were delayed: to December 2027 for standalone high-risk systems, and to August 2028 for AI built into regulated products, which is where medical devices sit (Regulation (EU) 2026/1744; see the Orrick summary, July 2026). The new dates are 2 December 2027 and 2 August 2028 (checked 12 October 2026). Over the same period, the United States withdrew its 2023 executive order on AI safety, loosened rules at both FDA and ONC, and tried, so far without a statute, to override state AI laws.

NoteYour insurer, briefly

For a resident, the malpractice coverage that matters first is your institution’s. Someone in the GME office can tell you what it says about AI tools. Ask in orientation, not after. One malpractice carrier, The Doctors Company, has advised physicians to judge each AI tool on its own. Its chief operating officer named two things to ask: what data the tool was trained on, and whether its accuracy holds up over time (Medical Economics, 9 October 2025).

Optional exercise: has it changed? (about 15 minutes). Pick one question. Find the current answer from the primary source: the agency, the statute, the court docket (the court’s own record of a case), or the organization’s own page. Record the URL, the date you checked, and the date shown on the source. A chatbot can help you find the document, but the answer must come from the document itself.

  1. What is the issue date on FDA’s current Clinical Decision Support Software guidance, and do the four criteria still read as listed above?
  2. Has ONC finalized HTI-5? If so, does the “model card” requirement still exist?
  3. Has FDA issued generative-AI guidance (not a discussion paper) for AI-enabled devices?
  4. Has any US malpractice claim involving a clinical AI tool reached a verdict?
  5. Does your own institution have a written policy on ambient documentation, on consent for it, and on which AI tools carry a business associate agreement? Find the document, or record who told you none exists.

9.8.6 Primary sources to bookmark

Check the date on each before quoting it.

9.8.7 Talks

Adams, L., Fontaine, E., Lin, S., Crowell, T., Chung, V. C. H., & Gonzalez, A. A. (2024). Artificial Intelligence in Health, Health Care, and Biomedical Science: An AI Code of Conduct Principles and Commitments Discussion Draft. NAM Perspectives, 2024. https://doi.org/10.31478/202403a
Afshar, M., Baumann, M. R., Resnik, F., Hintzke, J., Sullivan, A. G., Wills, G., Lemmon, K., Dambach, J., Ann Mrotek, L., Quinn, M., Abramson, K., Kleinschmidt, P., Brazelton, T. B., Leaf, M. A., Twedt, H., Kunstman, D., Patterson, B., Liao, F., Rasmussen, S., … Gordon, J. (2025). A Pragmatic Randomized Controlled Trial of Ambient Artificial Intelligence to Improve Health Practitioner Well-Being. NEJM AI, 2(12). https://doi.org/10.1056/aioa2500945
Al-Anezi, F. M. (2026). Generative Artificial Intelligence in Healthcare: Automation Bias, Deskilling, and Cognitive Implications - A Systematic Review. Journal of Healthcare Leadership, 18, 590498. https://doi.org/10.2147/jhl.s590498
Anderson, T. N., Mohan, V., Dorr, D. A., Ratwani, R. M., Biro, J. M., & Gold, J. A. (2025). Evaluating the Quality and Safety of Ambient Digital Scribe Platforms Using Simulated Ambulatory Encounters. Mayo Clinic Proceedings. Digital Health, 3(4), 100292. https://doi.org/10.1016/j.mcpdig.2025.100292
Armitage, R. C. (2025). Response to Sun and Colleagues: Charting a Simpler Ethical Landscape of Generative AI-Augmented Clinical Documentation. Journal of Evaluation in Clinical Practice, 31(5), e70232. https://doi.org/10.1111/jep.70232
Badal, K., Lee, C. M., & Esserman, L. J. (2023). Guiding principles for the responsible development of artificial intelligence tools for healthcare. Communications Medicine, 3(1). https://doi.org/10.1038/s43856-023-00279-9
Budzyń, K., Romańczyk, M., Kitala, D., Kołodziej, P., Bugajski, M., Adami, H. O., Blom, J., Buszkiewicz, M., Halvorsen, N., Hassan, C., Romańczyk, T., Holme, Ø., Jarus, K., Fielding, S., Kunar, M., Pellise, M., Pilonis, N., Kamiński, M. F., Kalager, M., … Mori, Y. (2025). Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: A multicentre, observational study. The Lancet. Gastroenterology & Hepatology, 10(10), 896–903. https://doi.org/10.1016/s2468-1253(25)00133-5
Chandrasekaran, R., & Moustakas, E. (2026). Patient attitudes toward ambient artificial intelligence scribes in clinical care: Insights from a cross-sectional study. Journal of the American Medical Informatics Association : JAMIA, 33(2), 263–272. https://doi.org/10.1093/jamia/ocaf218
Chen, C.-Y. A., Leung, T. I., & Bajra, R. (2026). Re-centering Clinical Documentation in the Age of AI Scribes: Four Aims of the Patient Chart Note. JMIR Medical Education, 12, e95218. https://doi.org/10.2196/95218
Chen, S., Guevara, M., Moningi, S., Hoebers, F., Elhalawani, H., Kann, B. H., Chipidza, F. E., Leeman, J., Aerts, H. J. W. L., Miller, T., Savova, G. K., Gallifant, J., Celi, L. A., Mak, R. H., Lustberg, M., Afshar, M., & Bitterman, D. S. (2024). The effect of using a large language model to respond to patient messages. The Lancet. Digital Health, 6(6), e379–e381. https://doi.org/10.1016/s2589-7500(24)00060-8
Chouffani El Fassi, S., Abdullah, A., Fang, Y., Natarajan, S., Masroor, A. B., Kayali, N., Prakash, S., & Henderson, G. E. (2024). Not all AI health tools with regulatory authorization are clinically validated. Nature Medicine, 30(10), 2718–2720. https://doi.org/10.1038/s41591-024-03203-3
Chowdhury, A., Casey, M., Wilson, J., Pollak, K. I., Goldstein, B. A., Bedoya, A., & Poon, E. G. (2026). Comparing ambient scribes: A randomized crossover clinical trial addressing ambient scribe technologies’ impact on physician burnout. Journal of the American Medical Informatics Association : JAMIA, 33(5), 990–999. https://doi.org/10.1093/jamia/ocag018
Draper, T. C., Leake, J., Cox, T., Lamb-Riddell, K., Johns, B. E., McCormick, J., Trowell, S., Kiely, J., & Luxton, R. (2026). AI-generated clinical summaries: Errors and susceptibility to speech and speaker variability. BMJ Health & Care Informatics, 33(1). https://doi.org/10.1136/bmjhci-2025-101918
Goddard, K., Roudsari, A., & Wyatt, J. C. (2011). Automation bias: A systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association : JAMIA, 19(1), 121–127. https://doi.org/10.1136/amiajnl-2011-000089
Jabbour, S., Fouhey, D., Shepard, S., Valley, T. S., Kazerooni, E. A., Banovic, N., Wiens, J., & Sjoding, M. W. (2023). Measuring the Impact of AI in the Diagnosis of Hospitalized Patients: A Randomized Clinical Vignette Survey Study. JAMA, 330(23), 2275–2284. https://doi.org/10.1001/jama.2023.22295
Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J. R., Jurafsky, D., & Goel, S. (2020). Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences of the United States of America, 117(14), 7684–7689. https://doi.org/10.1073/pnas.1915768117
Lawrence, K., Kuram, V. S., Levine, D. L., Sharif, S., Polet, C., Malhotra, K., & Owens, K. (2025). Informed Consent for Ambient Documentation Using Generative AI in Ambulatory Care. JAMA Network Open, 8(7), e2522400. https://doi.org/10.1001/jamanetworkopen.2025.22400
Lukac, P. J., Turner, W., Vangala, S., Chin, A. T., Khalili, J., Shih, Y.-C. T., Sarkisian, C., Cheng, E. M., & Mafi, J. N. (2025). Ambient AI Scribes in Clinical Practice: A Randomized Trial. NEJM AI, 2(12). https://doi.org/10.1056/aioa2501000
McGaughey, S., Abernethy, J., Wackett, J., Organick-Lee, J., Jones, R. L., & Cloutier, R. L. (2026). When the scribe does the reasoning: Ambient artificial intelligence, inference impersonation, and the development of trainees’ clinical judgment. Academic Medicine : Journal of the Association of American Medical Colleges. https://doi.org/10.1093/acamed/wvag287
Meier, E., Rigter, T., Schijven, M. P., Hoven, M. van den, & Bak, M. A. R. (2024). The impact of digital health technologies on moral responsibility: A scoping review. Medicine, Health Care, and Philosophy, 28(1), 17–31. https://doi.org/10.1007/s11019-024-10238-3
Mello, M. M., & Guha, N. (2024). Understanding Liability Risk from Using Health Care Artificial Intelligence Tools. The New England Journal of Medicine, 390(3), 271–278. https://doi.org/10.1056/nejmhle2308901
O’Neil, E., Rodman, A., & Lehmann, L. S. (2026). Balancing Innovation and Ethics: Ambient Listening Artificial Intelligence in Health Care. Mayo Clinic Proceedings. Digital Health, 4(1), 100341. https://doi.org/10.1016/j.mcpdig.2026.100341
Owens, L. M., Wilda, J. J., Grifka, R., Westendorp, J., & Fletcher, J. J. (2024). Effect of Ambient Voice Technology, Natural Language Processing, and Artificial Intelligence on the Patient-Physician Relationship. Applied Clinical Informatics, 15(4), 660–667. https://doi.org/10.1055/a-2337-4739
P Goddu, A., O’Conor, K. J., Lanzkron, S., Saheed, M. O., Saha, S., Peek, M. E., Haywood, C., & Beach, M. C. (2018). Do Words Matter? Stigmatizing Language and the Transmission of Bias in the Medical Record. Journal of General Internal Medicine, 33(5), 685–691. https://doi.org/10.1007/s11606-017-4289-2
Payne, P. R. O., Johnson, K. B., Maddox, T. M., Embi, P. J., Mandl, K. D., McGraw, D., Saria, S., & Adams, L. (2025). Toward an artificial intelligence code of conduct for health and healthcare: Implications for the biomedical informatics community. Journal of the American Medical Informatics Association : JAMIA, 32(2), 408–412. https://doi.org/10.1093/jamia/ocae306
Price, W. N., Gerke, S., & Cohen, I. G. (2019). Potential Liability for Physicians Using Artificial Intelligence. JAMA, 322(18), 1765–1766. https://doi.org/10.1001/jama.2019.15064
Rabotin, A., Aguilar, E., Sandoval Gonzalez, S., Iniguez, A., Jung, C., Lee, E., Bell, D. S., & Arroyo, J. (2026). Propagation of Interpreter Errors by Ambient AI Scribes: Study Using Simulated Clinical Encounters. JMIR Medical Informatics, 14, e88734. https://doi.org/10.2196/88734
Reis, M., Reis, F., & Kunde, W. (2024). Influence of believed AI involvement on the perception of digital medical advice. Nature Medicine, 30(11), 3098–3100. https://doi.org/10.1038/s41591-024-03180-7
Sun, Q. W., Miller, J., & Hull, S. C. (2026). Charting the ethical landscape of generative AI-augmented clinical documentation. Journal of Medical Ethics, 52(e1), e4–e7. https://doi.org/10.1136/jme-2024-110656
Tacconelli, A., Merane, J., Nielsen, A., Tobia, K., Hackanson, B., & Stremitzer, A. (2026). How Following Medical Artificial Intelligence Advice Can Mitigate Malpractice Liability: Cross-National Insights from a Randomized Trial. Journal of Nuclear Medicine : Official Publication, Society of Nuclear Medicine, 67(9), 1474–1480. https://doi.org/10.2967/jnumed.126.272292
Taylor, S. L., Aizenberg, D., Jost, M., Ren, Y., Lyles, C. R., & Adams, J. Y. (2026). Effect of use of ambient listening technology on patient-reported communication and satisfaction ratings. Journal of the American Medical Informatics Association : JAMIA, 33(9), 1780–1784. https://doi.org/10.1093/jamia/ocag103
Taylor, S. L., Jost, M., MacDonald, S., Ren, Y., Hilton, S., Davenport, S., Aizenberg, D., Hall, B., Lyles, C. R., & Adams, J. Y. (2026). Quality of Clinical Notes Created by Ambient Listening Generative AI: Pragmatic Prospective Pilot Study. JMIR Medical Informatics, 14, e86474. https://doi.org/10.2196/86474
Tobia, K., Nielsen, A., & Stremitzer, A. (2020). When Does Physician Use of AI Increase Liability? Journal of Nuclear Medicine : Official Publication, Society of Nuclear Medicine, 62(1), 17–21. https://doi.org/10.2967/jnumed.120.256032
Windecker, D., Baj, G., Shiri, I., Kazaj, P. M., Kaesmacher, J., Gräni, C., & Siontis, G. C. M. (2025). Generalizability of FDA-Approved AI-Enabled Medical Devices for Clinical Use. JAMA Network Open, 8(4), e258052. https://doi.org/10.1001/jamanetworkopen.2025.8052
Wolfe, J., & Welsh, S. M. (2026). Ambient Documentation Systems in Emergency Medicine: A Scoping Review of Clinical Precision, Patient Experience, Throughput, and Quality. Cureus, 18(4), e106643. https://doi.org/10.7759/cureus.106643