Evaluating LLM Outputs Systematically

Shipping LLM features without an evaluation framework is flying blind. Here is how to build one that gives you real signal.

Abstract

LLM evaluation frameworks provide the feedback loop that separates experimental AI features from production-grade systems. This article compares RAGAS, LangSmith, and custom evaluation pipelines — covering dataset construction, metric design, regression testing, and the organizational practices that make evals actionable.

May 8, 2026

3 min read

Every serious LLM deployment eventually hits the same wall: you make a prompt change that seems better on the examples you checked, ship it, and something else degrades. Without an evaluation framework, you discover this from users. With one, you discover it in CI.

What You Are Actually Measuring

LLM evaluation covers three distinct concerns:

Retrieval quality (for RAG systems): Is the retrieval step surfacing the right context?
Generation quality: Is the model producing correct, grounded, coherent outputs given that context?
End-to-end task performance: Does the full system accomplish what the user intended?

Most teams start at #3 and work backward. That is fine — measure what hurts first.

RAGAS for RAG Pipelines

RAGAS (Retrieval-Augmented Generation Assessment) provides four metrics designed specifically for RAG:

from ragas import evaluate
from ragas.metrics import faithfulness, answer_relevancy, context_precision, context_recall

result = evaluate(
    dataset,  # HuggingFace dataset with question, contexts, answer, ground_truth
    metrics=[faithfulness, answer_relevancy, context_precision, context_recall]
)
print(result)
# {'faithfulness': 0.89, 'answer_relevancy': 0.76, ...}

Faithfulness measures whether the answer is grounded in the retrieved context — the primary hallucination signal.

Context precision measures whether the retrieved documents are relevant (retrieval quality).

Context recall measures whether the relevant documents were actually retrieved.

The limitation: RAGAS uses an LLM-as-judge internally, which means its scores are only as reliable as the judge model. Use GPT-4 class models as judges; using the same model you are evaluating introduces obvious bias.

LangSmith for Continuous Evaluation

LangSmith provides tracing + evaluation in a single platform. The key workflow:

from langsmith import Client
from langsmith.evaluation import evaluate

client = Client()

def my_evaluator(run, example):
    # run.outputs contains LLM response
    # example.outputs contains ground truth
    score = compute_similarity(run.outputs['answer'], example.outputs['answer'])
    return {'key': 'answer_similarity', 'score': score}

results = evaluate(
    my_chain,
    data='my-eval-dataset',
    evaluators=[my_evaluator],
    experiment_prefix='prompt-v2'
)

The experiment comparison UI is the main value: you can see exactly which examples regressed between prompt versions, not just aggregate scores.

Building Custom Evals

For domain-specific quality, you need custom evaluators. The pattern:

interface EvalCase {
  input: string;
  expectedOutput: string;
  metadata?: Record<string, unknown>;
}

interface EvalResult {
  score: number;      // 0-1
  passed: boolean;
  reasoning?: string;
}

async function gradeWithLLM(
  actual: string,
  expected: string,
  rubric: string
): Promise<EvalResult> {
  const response = await llm.complete({
    messages: [
      { role: 'system', content: `Grade the following response against the rubric. Return JSON with score (0-1) and reasoning.\n\nRubric: ${rubric}` },
      { role: 'user', content: `Expected: ${expected}\n\nActual: ${actual}` }
    ]
  });
  return JSON.parse(response.content);
}

Dataset Construction

The hardest part is building a good dataset. Strategies that work:

Mine production traces: Take real user queries from your logs. Sample failures specifically — these are your hardest cases.
Adversarial generation: Use an LLM to generate edge cases given your system prompt. Ask it to "generate 20 inputs that would challenge this system."
Stratify by intent: Ensure your dataset covers all user intent categories proportionally.

Start with 100 examples. 100 well-curated examples beat 1000 synthetic ones.

Integrating Evals into CI

# .github/workflows/eval.yml
- name: Run eval suite
  run: python scripts/run_evals.py --min-faithfulness 0.85 --min-relevancy 0.75
  env:
    OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

Gate on thresholds that matter, not on zero-degradation — LLM outputs have natural variance. A 2% drop in aggregate score from noise is not a regression; a 15% drop on your "refusal" test cases probably is.

The infrastructure cost is real: running an eval suite against GPT-4-class judges can cost $5-50 per run depending on dataset size. That is cheap compared to a bad prompt reaching production.