How do you evaluate LLM outputs, and what is LLM-as-a-judge?

How to think about it

The direct answer

I evaluate LLM outputs at several levels: deterministic checks for format and safety, task-specific metrics for correctness, human review for nuanced qualities, and model-based judging for scalable assessment. I do not trust a single number because an answer can be fluent, well formatted, and completely wrong.

LLM-as-a-judge means using one capable language model to score or compare another model’s outputs against a written rubric. It is cheaper and faster than having people inspect every answer, but it is a measurement instrument, not ground truth.

Why there is no single score

The first question I ask is: what does “good” mean for this application?

For a code-generation assistant, good may mean that the code passes tests. For a customer-support assistant, it may mean resolving the customer’s issue using the approved policy, citing the right source, and refusing requests it cannot safely handle. A poetic answer and a correct answer are not interchangeable merely because both sound polished.

I normally separate evaluation into four layers:

Layer	What it checks	Example
Contract checks	Whether the output obeys hard rules	Valid JSON, required fields, no secret leakage
Task metrics	Whether it completes the intended task	Accuracy, F1, code tests, retrieval recall
Quality judgment	Whether the response is useful and well reasoned	Correctness, relevance, completeness, tone
Human and production evidence	Whether the measurement reflects reality	Expert ratings, task completion, escalations

A reference-based metric compares generated text with one or more human-written reference answers. BLEU measures matching word sequences with a brevity penalty; ROUGE measures overlapping sequences, with several variants that emphasize recall. These metrics are useful when wording should resemble a reference, such as translation regression tests. They are poor judges of an open-ended answer: a correct paraphrase can receive a low score, while a copied but incorrect phrase can receive a high one.

A task benchmark is a fixed collection of tests. MMLU, for example, measures multiple-choice knowledge across subjects. HumanEval checks whether generated code passes functional tests. Those benchmarks are useful regression points, but they do not tell me whether a payroll assistant follows my company’s refund policy at 3 a.m. Production evaluation must resemble the actual task.

A concrete evaluation design

Suppose I am evaluating an assistant for a payroll product. It answers questions using an internal policy library and can create a support ticket.

I freeze a test set of 1,000 cases before comparing models:

• 700 routine questions, such as changing a bank account
• 200 edge cases, such as a payment reversal after the processing deadline
• 100 out-of-scope or unsafe requests

Each case has an expected action, relevant policy passages, and acceptance criteria. An acceptance criterion is a condition the answer must satisfy, such as “do not promise a refund” or “escalate when identity is unverified.”

The evaluation might produce these results:

• 986 of 1,000 responses are valid JSON
• 650 of 700 routine cases are correct
• 122 of 200 edge cases are correct
• 88 of 100 out-of-scope cases are correctly refused or escalated

The overall task success rate is 860 out of 1,000, or 86 percent. That number hides the important fact: routine accuracy is about 93 percent, while edge-case accuracy is 61 percent. If edge cases cause expensive payroll errors, I would not approve this system just because its global average looks respectable.

I would also inspect whether the 900 in-scope answers are supported by the supplied policy. If 783 contain only claims supported by the policy passages, the support rate is 87 percent. That is a different measurement from answer correctness. An answer can reach the right conclusion for the wrong reason, or cite a real policy that does not actually support its conclusion.

For release, I would set gates based on risk. For example, valid output might need to exceed 99 percent, unsupported policy claims might need to stay below 2 percent, and edge-case accuracy might need its own threshold. Those values are product decisions, not universal laws. The key is to choose them before looking at the result, then report results by meaningful subgroups, or slices, rather than celebrating one blended average.

How LLM-as-a-judge works

A judge receives the user question, the candidate answer, and a rubric. For a retrieval-augmented system, I also provide the retrieved evidence. Without that evidence, the judge cannot reliably determine whether a claim is supported.

A judge can use pointwise evaluation, which scores one answer, or pairwise evaluation, which chooses between two answers. Pairwise comparison is often easier for a model than assigning an absolute score such as 4.2 out of 5, but it introduces ordering problems. I randomize which answer appears first and repeat the comparison on a sample with the order swapped.

A simple judging prompt might look like this:

Task: Answer the payroll question using only the supplied policy.

Question:
Can I reverse a direct deposit after payroll has been submitted?

Policy evidence:
[insert retrieved policy passages]

Candidate answer:
[insert model answer]

Score each dimension from 0 to 2:
- Correctness: Does the answer reach the policy-supported conclusion?
- Grounding: Is every important claim supported by the evidence?
- Completeness: Does it mention the required next step or escalation?
- Safety: Does it avoid promising an action the system cannot take?

Do not reward length, confidence, or polished wording.
Return the scores and one short evidence-based reason.

The judge’s output is useful only after calibration. I create a calibration set, meaning a small collection of examples independently labeled by qualified human reviewers. I compare the judge with those labels, examine disagreements, and test deliberately difficult cases. For pairwise judging, I check whether the judge agrees with human preferences. For pass-or-fail judging, I examine false passes and false failures, not just raw agreement.

This distinction matters. If 95 percent of answers are acceptable, a lazy judge that always says “pass” gets 95 percent agreement while missing every bad answer. Per-class precision and recall, plus manual review of disagreements, reveal that failure.

The senior-level nuance

An LLM judge is not automatically objective because it produces a numerical score. It may prefer longer answers, mirror the style of its own model family, or reward confident language. If I optimize directly against that score, the system may learn to write persuasive fluff instead of becoming more correct. That is evaluation gaming in a nicer shirt.

The first symptom of verbosity bias is that a longer, hedged answer consistently beats a short answer containing the correct action. I counter it with explicit rubric anchors, a penalty for irrelevant material, and pairwise tests where the concise answer is known to be correct.

The first symptom of position bias is that the winner changes when I swap answer A and answer B. Randomizing order and measuring swap consistency exposes it.

A more dangerous failure occurs when the judge sees only the answer, not the source documents. It may call an unsupported claim “faithful” because the claim sounds plausible. For grounded systems, I use claim-level checks, citation audits, retrieval tests, and a judge that can inspect the evidence. Faithfulness means that the answer is supported by the provided source; it does not simply mean that the answer sounds factual.

I also keep humans in the loop where errors are costly or the test set is small: medical advice, legal decisions, financial approvals, safety incidents, and new products with unclear requirements. Human review is expensive and variable, but replacing it entirely with a model hides uncertainty rather than removing it.

Finally, I version the test set, rubric, judge model, prompts, and scoring code. If the judge model changes, yesterday’s score may no longer be comparable with today’s. I retain sampled outputs so a regression can be inspected rather than argued about from a dashboard.

What they’ll ask next

How would you evaluate a RAG system specifically?

I would separate retrieval from generation. I would measure whether the relevant document was retrieved, then whether the answer was correct and supported by that document. A judge would receive the question, retrieved passages, answer, and citation locations. An answer-only judge cannot measure faithfulness reliably.

How do you know the judge is reliable?

I would label a representative human calibration set, compare judge decisions with those labels, inspect false passes and false failures, and run adversarial tests for verbosity and answer-order bias. I would also use a different judge model or human review for a holdout sample where practical. Agreement is evidence, not proof.

What if offline scores improve but users are still unhappy?

I would inspect production slices and measure task completion, escalation rate, repeat contacts, latency, cost, and user feedback. Offline cases may omit ambiguous or newly changed requests. I would trace real failures back into the test set, while remembering that a thumbs-down is a useful signal but not automatically a correct label.

Say this in the interview

“I use deterministic checks, task-specific metrics, calibrated human or LLM judgment, and production outcomes together; LLM-as-a-judge scales rubric-based review, but I validate its bias and agreement before treating its score as evidence.”