Back to Blog Room
Data Integrity & Verification

Double-Sided Verification: Auditing Unstructured Report Claims Against Structured Databases

By Datamagics TeamJuly 12, 20268 min read
Report Validator Banner Illustration

LLM Core Insights (TL;DR)

  • 📌 Purpose: Automate the auditing of numerical claims in financial/operational text reports.
  • 🔄 Double-Sided Ground-Truth: Reference raw files (CSV, XLSX, JSON) and relational databases simultaneously.
  • 🛠️ How it Audits: The AI extracts claims, identifies relevant columns, and writes and runs SQL/Pandas scripts to verify metrics.

Operational summaries and financial statements are critical document assets. Yet, they are frequently loaded with minor metric discrepancies—such as slightly outdated sales calculations, wrong averages, or incorrect user summaries. Checking these numbers against your primary data stores manually represents a significant operational cost.

1. The Concept of Double-Sided Auditing

Datamagics introduces the **Double-Sided Report Validator**, a verification engine that links natural language text directly to two separate ground-truth references:

  • Dataset Reference: Raw logs, static exports (CSV, XLSX, JSON) uploaded inline.
  • Database Reference: Live SQL databases (PostgreSQL, MySQL) connected via custom database connectors.

2. Real-Time SQL & Pandas Verification

When a user uploads a report (PDF, text, docx), our AI-driven audit engine performs three key processes:

  1. Claim Extraction: The LLM reads the document and isolates sentences containing numerical assertions (e.g., 'Q3 Revenue grew by 15.4%').
  2. Schema Query Synthesis: Based on the combined schema description of the dataset and the database connection, the AI translates the assertion into verification queries. It generates a Python Pandas script for dataset rows and a SQL query for database records.
  3. Isolated Query Sandbox: The generated queries are executed on the backend in a secure, isolated sandbox, and the returns are cross-references with the report claims.

3. Detailed Auditing Logs & Variance Flags

The audit returns a comprehensive visual breakdown of every claim, categorizing them as **Matched**, **Discrepancy**, or **Error**. If a discrepancy occurs, the validator prints the exact difference between the claimed value and the database's actual value, showing the exact SQL/Pandas code block utilized to discover the variance.