Leitfaden

How Reliable Is AI Documentation Really? What to Check When You Review It

AI documentation takes the writing off your hands – but how reliable are the drafts it delivers? The honest answer: very accurate, but not error-free. In this article, you will learn where AI documentation makes mistakes, why they happen and what you should check specifically when reviewing.

20.11.2026
Redaktion
Reading time
output:  Moderner Krankenhausflur mit Empfang und Pflanzen.

Practice
Type
Specialty
Location
Website
About the practice

Challenge

In this article, you'll read:

The essentials at a glance

  • In studies, AI documentation is very accurate but not error-free – and its most common mistake is not making things up, but leaving things out.
  • Read every AI draft on the same day and look for gaps first, then for incorrect information.
  • If you review in a targeted way, you keep the time the AI saves you and stay fully in control of your own patient records.

Friday, 6 pm. The last treatment is over, and 25 drafts are waiting for approval. They are all neatly worded, well structured and, at first glance, complete. You read the entry for the filling on tooth 46: findings, anaesthesia, material, aftercare advice. Everything is correct.

What is missing, you cannot see. Right at the end, while you were doing the final check, the patient asked whether a ceramic filling on tooth 47 might be an option. You promised to give her a treatment plan for it at her next appointment. None of this is in the draft.

The draft is not wrong. It is incomplete. And that is exactly what you cannot tell by looking at it.

How reliable is AI documentation – what do the studies say?

In studies, AI documentation shows error rates of around one to three per cent – which makes it reliable, but not error-free. This is the conclusion of a review in npj Digital Medicine from 2025. However, the authors stress that modern systems bring their own types of error, such as fabricated content, critical omissions and misattributed statements.

How high the error rate is in any given case depends heavily on what is measured and how. Some studies only count factually incorrect information, others also count missing content and inconsistencies. The following overview shows what selected studies have found:

Table of studies on the reliability of AI documentation: error rates of around 1–3%, omissions as the most common error, 94.7% of notes without serious errors

So there is no single error rate. But there is a clear pattern: AI rarely makes things up. More often, it leaves things out.

Is AI less accurate than your own documentation?

This question cannot currently be answered reliably. Direct comparisons between AI drafts and manual documentation are rare, and the existing studies use very different methods.

What is clear, however, is that even your own documentation has gaps. Anyone who fills in notes from memory in the evening forgets details – that is only human. With AI documentation, every entry is looked at twice: the AI records what was discussed in the conversation, and you add or correct what it missed. This reduces the likelihood of an error ending up unnoticed in the patient record.

Why does AI make mistakes in documentation at all?

Errors in AI documentation arise at four points: when hearing, when attributing, when summarising – and with everything that is never said out loud in the treatment room. If you know this chain, you know what to look for when reviewing.

Table of the four sources of error in AI documentation – hearing, attribution, summarising and what is not said – with examples from a dental practice

The last stage is often underestimated. The dentists Falk Schwendicke and Tabea Flügge point out in zm that AI can only capture what is actually said in the conversation. What you only see, feel or think, it cannot document.

Conversations involving several people and languages are also particularly vulnerable. A study in JMIR Medical Informatics shows that more than half of an interpreter’s translation errors later reappeared in the AI notes. For your practice, this means: if a relative interprets for the patient, the draft deserves special attention.

What are AI hallucinations – and how common are they really?

AI hallucinations are content that sounds plausible but has no basis in the conversation. One example would be a documented examination that never took place. The topic causes a lot of uncertainty, but in documentation it is less common than often assumed: in the study by Asgari et al., it affected fewer than two per cent of sentences.

Rare does not mean harmless, though. Hallucinations are tricky because they are worded just as convincingly as everything else in the draft. A fabricated sentence does not come with a warning sign.

This is where Nelly’s AI Documentation comes in. It listens passively during the patient conversation, filters out the medically relevant content in real time and prepares a draft from it. You don’t have to dictate or rephrase anything. At the same time, what is said out loud in the conversation can be captured – what remains unsaid cannot.

Learn more here: https://www.getnelly.de/en/ai-assistance#ki-dokumentation

Why do you miss errors above all when the AI works well?

The more reliably an AI works, the less carefully its output is checked – experts call this phenomenon automation bias. In its guidance on AI scribes, the Australian Commission on Safety and Quality in Health Care explicitly warns that clinicians may accept drafts as correct and complete too quickly.

The UC Davis Health pilot project shows how real this effect is. Although 94.7% of the notes there were free of serious errors, almost 15% of the notes were approved without any changes. Among the errors found were a diagnosis the patient did not have and an incorrect medication dose.

Then there is the type of error. Biro et al. point out that omissions in particular are the hardest to detect: to notice them, you have to remember details from the conversation. A missing sentence does not stand out when reading – it simply is not there.

A tool that is almost always right invites you to wave things through. That is why the time saved by AI is real – but only if part of it goes into the review. If you spend the time you gain twice over, you don’t save anything. You just shift the risk.

What do you actually need to check when reviewing?

When reviewing an AI draft, first check what is missing, then what is wrong – and do it on the same day. This order follows directly from the research: omissions are the most common error and at the same time the hardest to spot. The following routine helps you read in a targeted way rather than skimming.

  1. Read on the same day: You can only spot omissions while you still remember the conversation. On the evening of the same day you will still notice a missing detail; a week later, hardly at all. So build approval firmly into your daily routine.
  2. Look for gaps first: Go through the treatment calmly in your mind: has anything changed in the medical history? Which findings did you record, which procedures did you carry out, what did you agree? If any of this is missing from the draft, add it.
  3. Check critical details character by character: The Hessisches Ärzteblatt lists diagnoses, medications, dosages, allergies and treatment recommendations as points that need particularly careful checking. In your practice, tooth numbers and surfaces need to be added to this list.
  4. Check attribution: Make sure statements are attributed to the right person. This applies especially if accompanying persons were present or someone interpreted.
  5. Add what you only saw: The AI cannot know findings you recorded at the chair but did not say out loud. Add these silent findings before approval.

How can you improve quality right at the treatment chair?

The most effective lever is not in the review but in the treatment room itself. Say findings out loud and precisely, for example “36 distal carious”. That way, the AI captures what would otherwise only happen in your head. At the same time, your patient knows what you are seeing – which creates transparency and trust.

How does Nelly build in control?

Nelly’s AI Documentation is deliberately designed so that you always have the final say: the AI delivers a draft, not a finished record entry. It prepares your documentation but does not make any medical decisions. Only once you have reviewed and confirmed the draft does it become part of your documentation and go into your practice management system. Nobody has to type anything up or transfer it.

This pays off twice. The AI captures what is said in the conversation, and you add and correct the rest. This reduces the risk of errors and gaps in the patient record. At the same time, your documentation becomes more detailed. And complete documentation is the basis for being able to bill for all the services you have provided to the patient.

The review step is not a detour added afterwards, but a fixed part of the process. Nelly takes the writing off your hands, not the responsibility – and that is exactly how it should be. Everything also remains transparent for your patients: anyone who objects to AI documentation can opt out. You can see this before the appointment.

How can you test reliability in your own practice?

Studies provide averages. How reliably AI documentation works in your practice, on the other hand, depends on your rooms, your team and your patients. So take the time to test the solution under real conditions:

  • Document in parallel: For the first few days, also document selected treatments the way you usually do and compare the two versions.
  • Test difficult situations: Pay particular attention to treatments with suction, conversations with accompanying persons and patients who speak little German.
  • Track corrections: Note what you have to change. Recurring types of error show you what to pay particular attention to in future.
  • Measure review time: Record how long approval takes per patient. This shows you how much time you actually save after the review.
  • Involve your team: Discuss with your dental assistants how findings are clearly stated in the treatment room.

Try Nelly’s AI Documentation free for 14 days

Whether AI documentation works reliably for you only becomes clear in practice. So try Nelly’s AI Documentation free of charge for 14 days – with your patients, your team and your processes.

Try it free now

Conclusion: AI documentation becomes reliable through your review, not in spite of it

AI documentation is very accurate, but not error-free. Its most common mistake is leaving things out – and you can only spot that if you review promptly and in a targeted way. The review step is therefore not a vote of no confidence in the AI, but the precondition for it to really help you. If you plan for control deliberately, you gain both: complete patient records and more time for what matters – your patients.

Get informed now – with no obligation

Would you like to see how Nelly’s AI Documentation works in your everyday practice? Book a free consultation now.

FAQs about the reliability of AI documentation

How high is the error rate of AI documentation?

Studies on AI documentation systems report error rates of around one to three per cent. The figures vary considerably depending on the system and how errors are measured. The most common error is omission: content from the conversation is missing from the draft. That is why review by the dentist before approval remains essential.

What are AI hallucinations in documentation?

AI hallucinations are content the AI generates even though it did not come up in the conversation – for example an examination that never took place. They are less common than omissions, but tricky because they sound plausible. So check that everything documented was actually said or done.

Do I have to read every AI draft in full?

Yes, because by approving it, you adopt the draft as your documentation. It doesn’t have to take long: read on the same day, check for gaps first and then specifically check critical details such as tooth numbers, medication and allergies. This way, you keep the time savings without giving up control.

Redaktion

Nelly Editorial Team

The Nelly editorial team engages daily with the challenges and processes of everyday dental practice. Staying close to the practice and the people who work in it every day, they present digital solutions in a practical and understandable way, so that dentists and their teams can make well-informed decisions for the future of their practice.

Want to know more?

Get all the details - completely free and with no obligation.

Let's talk

You might also be interested in: