AuditQA — AI Financial Statement Quality Review

Financial statement QA is slow when every casting, note, period, and presentation check is manual. Growing prompt libraries with new rules made models harder to control. Teams needed arithmetic and structure checks that are deterministic, LLM judgment only where it belongs, and a learning path that improves quality without bloating prompts.

We built AuditQA as a hybrid audit agent: • Pipeline: upload → extract text (python-docx) → tools-first agent → LLM judgment → sanitize/score → store findings • Quality Review mode: ~32 checklist findings (casting, notes, periods, presentation, and more) • Bilingual Cross-Check mode: 7 findings comparing two language versions of the FS • Inference: Ollama (default llama3.2) with auditqa-auditor Modelfile persona; tools run first, LLM only on judgment checks • Minimal system prompt (identity + JSON contract + check names); deterministic tools for cast, note cross-ref, period, comparative/interim, spelling, presentation • Post-processing: JSON repair, sanitize, scoring, narrative/executive summary • Tool results override the LLM for the same check name • Learning loop: AgentEpisode logging → human labels (confirmed, false_positive, missed_issue, …) → JSONL export → golden eval → optional LoRA adapter with deploy gate (score ≥ baseline) → continuous run_learning_cycle • Supporting surfaces: auditor chat (scoped Q&A), compact knowledge retrieval, learning dashboard (FP/miss trends, tokens, eval metrics)

AuditQA delivers production FS review with a clear AI/ML principle—shrink prompts, grow tools and adapters: • Structured Quality Review and Bilingual Cross-Check outputs auditors can act on • Deterministic tools own arithmetic/structure; LLM owns judgment only • Same-check tool results win over the model when they conflict • Feedback → dataset → adapter → golden eval gate before shipping improvements • Phases 0–4 complete: episode logging, golden eval, tools-first agent, adapter path, continuous learning + dashboard (runtime MD injection removed from inference)