From Hallucination to Trust: How to Guardrail Enterprise Financial AI Before It Hits Investor Desks

The problem: small miscalculations, big consequences

An enterprise finance platform uses large language models to summarize quarterly earnings reports for analysts. The summaries read well, which is exactly the danger. A growth rate computed from the wrong base, a figure pulled from the prior-year column, or a percentage the model simply made up looks just as confident as a correct one. In financial reporting, one such error damages investor trust and can create regulatory exposure.

Better prompts reduce errors but never eliminate them. What's needed is a gate: no summary reaches an analyst until it has been scored against its sources.

The solution: online and offline evaluation

We use the Azure AI Evaluation SDK in two places:

Online, on every summary: before a draft is shown, it's scored for groundedness (is every claim supported by the retrieved filings?) and similarity (is it close to an analyst-approved reference?), and every number is re-matched against the ledger in plain code.
Offline, on every change: a golden dataset of past quarters runs through the same evaluators in CI. A new prompt or model version ships only if the scores hold.

Solution architecture and data flow

Every summary is scored before release; every prompt or model change is regression-tested first.

Ingestion: 10-K/10-Q filings and general-ledger extracts land in Azure Blob Storage and are indexed in Azure AI Search (vector plus keyword).
Grounded retrieval: the summarization service retrieves the exact passages and figures for the quarter (RAG).
Generation: Azure OpenAI drafts the earnings summary from that context only.
Evaluation gate: the draft, the retrieved context and the reference are scored:
- Groundedness: every statement must be supported by the source text.
- Similarity: the summary must stay close to the analyst-approved reference.
- Numeric check (code): every figure in the summary must appear in the ledger. LLM judges are good at meaning and weak at arithmetic, so numbers get a deterministic check.
Release or review: summaries that pass go to the analyst dashboard with their scores attached; anything that fails goes to a human review queue with the evaluator's reasons. Both are written to an audit store.

The online gate

Each evaluator returns a 1–5 score, a reason, and a pass/fail result against the threshold you set. The gate reads the pass/fail results rather than hard-coding score comparisons:

import os
from azure.ai.evaluation import AzureOpenAIModelConfiguration, GroundednessEvaluator, SimilarityEvaluator

# The judge model. Use a different deployment than the one that wrote the summary.
judge = AzureOpenAIModelConfiguration(
    azure_endpoint=os.environ["AZURE_OPENAI_ENDPOINT"],
    azure_deployment="gpt-4o-judge",
    api_version="2024-10-21",
)

# Scores are 1-5; threshold sets the pass/fail cut for *_result.
groundedness = GroundednessEvaluator(judge, threshold=4)
similarity = SimilarityEvaluator(judge, threshold=4)


def gate(query: str, context: str, summary: str, analyst_reference: str) -> dict:
    """Score a generated earnings summary and decide whether it ships."""
    g = groundedness(query=query, context=context, response=summary)
    s = similarity(query=query, response=summary, ground_truth=analyst_reference)
    approved = g["groundedness_result"] == "pass" and s["similarity_result"] == "pass"
    return {
        "decision": "APPROVED" if approved else "HUMAN_REVIEW",
        "groundedness": g["groundedness"],
        "groundedness_reason": g["groundedness_reason"],
        "similarity": s["similarity"],
    }

Use a separate deployment as the judge, not the one that wrote the summary, so the model isn't grading its own work. Store the scores and reasons with every published summary: that becomes your audit trail.

The offline regression suite

Prompt tweaks and model upgrades are the most common source of silent regressions. Run the same evaluators over a golden dataset in CI and fail the build if quality drops:

from azure.ai.evaluation import evaluate
from gate import groundedness, similarity

results = evaluate(
    data="golden_set.jsonl",            # one row per past quarter: query, context, summary, reference
    evaluators={"groundedness": groundedness, "similarity": similarity},
    evaluator_config={
        "groundedness": {"column_mapping": {
            "query": "${data.query}", "context": "${data.context}", "response": "${data.summary}"}},
        "similarity": {"column_mapping": {
            "query": "${data.query}", "response": "${data.summary}", "ground_truth": "${data.reference}"}},
    },
    output_path="eval_results.json",
)

m = results["metrics"]                    # e.g. {"groundedness.groundedness": 4.6, ...}
assert m["groundedness.groundedness"] >= 4.5, m   # fail the pipeline: don't ship this prompt/model
assert m["similarity.similarity"] >= 4.0, m

Start the golden set with a handful of past quarters and their approved summaries, and add every summary a reviewer corrects. It grows into your best defence against regressions.

Key benefits

Hallucinated figures are caught before release. A summary that isn't grounded, or whose numbers don't match the ledger, never reaches an analyst unreviewed.
A continuous audit trail. Every published summary carries its sources, scores and reasons, which is what regulators and auditors ask for.
Safe model upgrades. New models and prompts are judged on your own data before they ship, not after.

Takeaways

Treat LLM output in regulated workflows as untrusted until scored.
Combine LLM-judged metrics (groundedness, similarity) with deterministic checks for numbers.
Use a different model deployment as the judge.
Gate every prompt and model change on a golden dataset in CI.