How do you evaluate LLM output quality: reference metrics, LLM-as-judge, golden datasets and regression evals?

There is no single metric for LLM quality. A good evaluation setup layers cheap, objective checks with more expensive judgement, all running against a curated dataset.

1. Start with deterministic checks

Many requirements are objective and should be checked with ordinary code:

• output parses as JSON and matches a schema;
• required facts or identifiers appear; banned phrases don’t;
• the correct tool was called with valid arguments;
• length limits, language, no URLs outside an allow-list.

These are fast, cheap and reproducible. Teams often skip them and go straight to an LLM judge, which is a mistake.

2. Reference-based metrics

When a reference answer exists:

• Exact match / F1 for short factual answers or classification labels.
• BLEU / ROUGE: n-gram overlap, historically used for translation and summarisation. They penalise valid paraphrases and reward surface overlap, so they correlate poorly with quality for open-ended answers.
• Embedding similarity (cosine similarity of embeddings, or BERTScore): captures paraphrase better, but two answers can be semantically close while one contains a critical factual error, such as “refunds within 30 days” vs “within 90 days”.

Reference metrics are useful for tracking trends and catching large regressions, not as the sole quality signal.

3. LLM-as-judge

A separate model grades output against a rubric: correctness relative to source documents, groundedness (no claims unsupported by context), completeness, tone, policy compliance. It scales far better than human review, but has documented biases:

• Position bias: in pairwise comparisons, preferring the first (or second) answer shown.
• Verbosity bias: preferring longer answers.
• Self-preference: favouring output from the same model family.
• Inconsistency between runs, and weakness on specialised domain facts.

Mitigations:

• Use specific rubrics with criteria and examples instead of “rate 1 to 10”.
• Prefer binary or small categorical scores per criterion over fine-grained scales.
• In pairwise comparisons, swap the order and count only consistent verdicts.
• Ask for reasoning before the verdict, and give the judge the source context.
• Calibrate against human labels: measure agreement on a labelled sample before trusting the judge, and re-check periodically.

JUDGE_RUBRIC = """
You are grading a support answer. Use ONLY the provided policy text.
Criteria (answer PASS or FAIL for each, with one sentence of reasoning):
1. factual: every claim is supported by the policy text
2. complete: answers the customer's actual question
3. safe: makes no promise the policy does not allow
Return JSON: {"factual": ..., "complete": ..., "safe": ..., "reasons": ...}
"""

4. Human review

Still the reference standard for nuanced quality. Use it to label the golden set, to calibrate judges, and to review a sample of production traffic and all judge/check disagreements.

5. Golden datasets

A versioned set of inputs with expected outcomes or grading criteria:

• representative of real traffic (anonymised), weighted towards high-risk intents;
• covering edge cases, adversarial inputs and multi-turn context;
• grown from every production incident so the same failure can’t return silently;
• kept separate from any data used to tune prompts, to avoid overfitting to the test set.

6. Regression evals

Every change to the model version, prompt, retrieval index, tools or temperature runs the eval suite:

• compare with the baseline per category, not only an overall average, so a gain in one area can’t hide a loss in a critical one;
• run cases multiple times and use pass rates, because outputs vary;
• set thresholds that block release for critical categories (safety, policy) and warn for others;
• store results so trends are visible over time.

Open-source eval frameworks and vendor tooling can run this pipeline, but the dataset and rubrics are the real asset, and they are specific to your product.

What interviewers look for

• Layered methods with deterministic checks first.
• Honest limits of BLEU/ROUGE and embedding similarity.
• Named LLM-as-judge biases and concrete mitigations, including human calibration.
• Golden data and regression evals treated like a real test suite with baselines and thresholds.