Understanding Confidence Scores in Document Extraction: What They Mean and How to Use Them

Understanding Confidence Scores in Document Extraction: What They Mean and How to Use Them

When AI extracts data from a document, it doesn't always know with equal certainty whether it got each field right. A clearly printed invoice number on a clean digital PDF is straightforward. A handwritten date on a low-quality scan is not. Confidence scores communicate this uncertainty.

Understanding confidence scores helps you configure your extraction workflow correctly — setting thresholds that match your documents, your team's capacity to review, and your tolerance for errors reaching downstream systems.

What a Confidence Score Represents

A confidence score is a number between 0 and 1 (or 0% and 100%) that represents how certain the extraction system is about a specific field value.

A score of 0.97 means the system is highly confident the extracted value is correct. This field is almost certainly right.

A score of 0.61 means the system is uncertain. The extracted value might be correct, or it might not be. Human review is warranted.

A score of 0.30 means the system has low confidence — it found something that might be the right field, but it's not sure. This field should definitely be reviewed.

Importantly, confidence scores are per-field, not per-document. A single invoice might have 12 fields extracted, with 11 at 0.95+ confidence and one — perhaps an ambiguous due date — at 0.64. Only that one field triggers human review, not the whole document. (For how that review step works in practice, see our walkthrough of the Human Review queue.)

How ExtractBee Calculates Confidence Scores

Our confidence scores aggregate several signals:

1. OCR Confidence (for scanned documents)

When Tesseract OCR converts a scanned image to text, it assigns a confidence score to each character recognised. We aggregate these into a page-level OCR confidence score. (OCR and AI extraction play different roles here — we unpack the distinction in OCR vs AI document extraction.)

Low OCR confidence typically indicates:

• Poor scan quality (low DPI, blurry, skewed)
• Unusual or degraded fonts
• Stamps or annotations overlapping printed text
• Handwritten content

A page with OCR confidence below 70% goes to the vision fallback — the AI model reads the image directly rather than the potentially garbled OCR text. (For more on why we run OCR pre-processing at all, see how we cut extraction cost with Tesseract.)

2. The AI Model's Extraction Confidence

When our extraction model (Gemini 2.5 Flash as the primary, with Claude Sonnet as an automatic fallback) returns a field value, we infer confidence from its behaviour:

High confidence signals:

• Field value returned as a specific, well-formed value
• Value matches expected format (date parses correctly, IBAN passes checksum)
• Value appears in a clear, unambiguous position in the document

Low confidence signals:

• The model returns null for a field that appears to have a value (uncertainty expressed as abstention)
• Value format is unusual (non-standard date format, mixed currency symbols)
• Multiple candidate values exist (two numbers that could both be the total)

3. Validation Rules

After extraction, we apply validation rules that can either increase or decrease confidence:

Confidence boosters:

• IBAN checksum validation passes
• Date is a plausible business date (not in the future by more than 1 year, not more than 5 years in the past)
• Currency code is a valid ISO 4217 code
• Total = subtotal + VAT amount (within rounding tolerance of ±0.02)

Confidence reducers:

• Date format is ambiguous (01/02/2026 could be Jan 2 or Feb 1)
• Amount uses an unusual decimal separator format
• Vendor name contains OCR character errors (detected by dictionary check)
• Invoice number contains unusual characters

4. Cross-field Consistency

We check consistency across related fields:

• Does subtotal + VAT ≈ total?
• Is the due date after the invoice date?
• Is the currency consistent across all amount fields?
• Does the IBAN country code match the vendor address country?

Inconsistencies reduce confidence on the affected fields and may flag the entire document for review.

What Confidence Thresholds Mean in Practice

Your confidence threshold is the minimum score a field must reach to be auto-approved. Fields below this threshold enter the Human Review queue.

Threshold: 0.95 (very strict) Nearly everything gets reviewed. Appropriate for: high-value invoices where errors have significant financial consequence, documents from vendors with variable quality, workflows where review capacity is high.

Threshold: 0.85 (recommended for most teams) Routine, clean extractions are auto-approved. Genuinely uncertain fields go to review. Typical review rate: 10–20% of documents.

Threshold: 0.75 (relaxed) Only clearly uncertain extractions trigger review. Appropriate for: high-volume workflows where review capacity is limited, document types with very consistent formatting.

Threshold: 0.00 (no review) All extractions auto-approved regardless of confidence. Only appropriate if you've validated that your specific document mix consistently produces high-confidence extractions.

Typical Confidence Scores by Document Type and Condition

Fields That Commonly Have Lower Confidence

Some fields are structurally harder to extract reliably than others:

Due date — especially when invoices use terms like "Net 30" or "30 days from invoice date" rather than an explicit date. The AI must calculate the date from the invoice date, which introduces uncertainty.

VAT rate — when not explicitly stated as a percentage (some invoices show only the VAT amount, requiring back-calculation from subtotal).

Line item descriptions — when descriptions span multiple lines, contain special characters, or are in a non-Latin script.

IBAN / bank details — when appearing as an image (e.g. a footer graphic) rather than text, or when OCR introduces character errors in the alphanumeric string. This is also why dense, tabular documents like bank statements get their own dedicated extraction handling.

Vendor name — when the legal name differs from the trading name on the invoice, or when the invoice header contains both a vendor name and a customer name in close proximity.

Using Confidence Scores in Your Workflow

Setting the Right Threshold

Start with 0.85 and adjust based on your observation:

• If Human Review is processing too many trivially correct extractions → lower to 0.80
• If errors are reaching your accounting system → raise to 0.90

Monitoring Confidence Trends

Low average confidence across many documents is a signal worth investigating:

• Consistently low confidence from one vendor → their invoice format may have an unusual element worth examining
• Decreasing confidence over time for a previously reliable vendor → they may have changed their invoice format
• Low confidence on specific fields across many documents → that field type may benefit from AI Training refinement

Combining with Audit Logs

The audit log (Business plan and above) records the confidence score for every extraction decision. Reviewing this periodically helps identify systematic extraction issues before they accumulate into significant errors.

Frequently Asked Questions

Is a 0.85 confidence score "accurate enough"? Confidence is not the same as accuracy. A 0.85 confidence score means the system is fairly certain about the value — but certainty and correctness are not identical. In our experience, fields at 0.85 confidence tend to be correct most of the time, but this varies considerably by document type and field. The Human Review step exists precisely because confidence scores are estimates, not guarantees.

Why does the same document sometimes get different confidence scores on re-extraction? If the document is a digital PDF, re-extraction should produce identical results (we use temperature 0 for extraction calls). If scores vary, it typically indicates the document has variable-quality content — perhaps a mix of digital and scanned pages, or a PDF with inconsistent text encoding.

Can I see confidence scores per field in exports? Yes. When exporting as JSON, confidence scores are included as a _confidence property on each field. CSV exports include confidence as additional columns. You can use this data in downstream systems to route low-confidence records to a review step in your own workflow.

Does improving scan quality improve confidence scores? Yes, significantly. The single most impactful change for scanned document quality is scan DPI: in our experience 300 DPI typically improves OCR confidence noticeably over 150 DPI, which flows through to higher field-level confidence scores. Keeping the document flat (not curled) and ensuring good lighting (no shadows) also helps substantially.

What's the difference between a null field and a low-confidence field? A null field means the AI determined the field is genuinely absent from the document (e.g. an invoice with no PO reference). A low-confidence field means the AI found a candidate value but isn't certain it's correct. Both appear in the extraction result, but only low-confidence fields (not nulls) trigger Human Review.

Try ExtractBee free — 20 pages/month →

Last updated: June 2026.

ExtractBee is operated by MB Dokigo, Vilnius, Lithuania.