Approaches to Evaluating Foundation Models

You spent the last topic fine-tuning a model. It answers your test prompts well, the demo went fine, and the team wants to ship. Then someone asks the only question that matters: is it actually better than the base model you started with, and better by how much?

"It looked good when I tried it" is not an answer. Ten prompts typed by the person who built the thing is the weakest evidence in software. This topic is about replacing that with something defensible, and it starts here, with the three families of evidence you can collect and what each one is worth.

Why a generative model is hard to grade

A classifier is easy to grade. Feed it 1,000 labeled emails, count how many it filed as spam correctly, and you have accuracy. Every input has exactly one right answer, so the scoring is arithmetic.

A foundation model breaks that. Ask it to summarize an incident report and there are hundreds of good summaries and no single correct one. Two summaries can share almost no words and both be excellent. So there is nothing to compare against by equality, and every evaluation approach below is a different way of working around that one problem.

Keep that framing, because it explains the whole design of the tooling. Automatic evaluation works around it by comparing to a reference answer and measuring similarity instead of equality. A judge model works around it by reading the output and rating it. Human evaluation works around it by asking people. Three answers to the same awkward question.

The grading analogy, and where it breaks

Think of grading student essays. A marking scheme that counts required keywords is fast, consistent, and blind to a beautifully argued essay that used different words. A teaching assistant with a rubric reads for meaning and scales to a hundred papers. The examiner is the real authority but can only read a few.

Automatic metrics are the keyword scheme, a judge model is the teaching assistant, and human reviewers are the examiner. The analogy breaks in one place worth remembering: a teaching assistant trained from the same textbook as the student shares the student's blind spots. A judge model has the same problem, which is why nobody treats judge scores as ground truth.

Approach 1: automatic evaluation against a dataset

Automatic evaluation, which Bedrock also calls programmatic evaluation, runs your model over a set of prompts and computes scores with no human in the loop. You pick three things when you create the job: a task type, the metrics you want, and the prompt dataset.

The task type tells Bedrock what kind of work the model is doing, which decides what can be measured. Bedrock offers general text generation, text summarization, question and answer, and text classification, plus a custom option.

Metrics fall into three groups that recur everywhere in this topic:

• Accuracy is how close the output is to a reference answer.
• Robustness is how much quality drops when the input is perturbed without changing its meaning, such as typos, random casing changes, and added or removed whitespace.
• Toxicity is whether the output contains harmful language, scored by a toxicity detection model rather than by comparison to a reference.

Accuracy and robustness both need to know the right answer. Toxicity does not, because it judges the output on its own.

Built-in datasets, and your own

Bedrock ships built-in prompt datasets so you can run a job before you have written any test data. Each one is a random 100-prompt sample from a well-known open-source dataset, and each is paired with the task type and metric it suits.

These are useful for a first pass and for sanity checks, and they are not your application. A support assistant answering questions about your product will not be decided by BoolQ. That is what a custom prompt dataset is for.

A custom dataset is a JSONL file in Amazon S3, up to 1,000 prompts per job, one JSON object per line:

{"prompt": "Aurillac is the capital of", "referenceResponse": "Cantal", "category": "Capitals"}
{"prompt": "Bamiyan city is the capital of", "referenceResponse": "Bamiyan Province", "category": "Capitals"}
{"prompt": "Sokhumi is the capital of", "referenceResponse": "Abkhazia", "category": "Capitals"}

prompt is the input the model receives. referenceResponse is the ground truth, and it is required for accuracy and robustness because those metrics have nothing to compare against without it. category is optional and buys you scores broken out per group, which is how you discover that a model is fine on billing questions and weak on refunds.

The effort of writing 200 real prompts with real answers is the single highest-value thing a team does before launch, and it is the step that gets skipped.

Benchmark datasets, and the trap in the leaderboard

Public benchmarks are shared datasets that let anyone compare models on the same questions. A few names show up constantly:

• GLUE and SuperGLUE collect language understanding tasks such as inference and sentence similarity.
• MMLU, Massive Multitask Language Understanding, is a multiple-choice test spanning 57 subjects from elementary mathematics to US history, computer science, and law, designed to probe broad world knowledge and problem solving.
• HELM, Holistic Evaluation of Language Models, scores models across 42 scenarios on 7 metrics: accuracy, calibration, robustness, fairness, bias, toxicity, and efficiency. Its point is that accuracy alone is a thin description of a model.

Now the misconception, and it is the expensive one. It is tempting to read a leaderboard as a ranking of which model is best for you. It is not. A benchmark measures performance on that benchmark's data and task mix, and your application has neither. A model that tops MMLU may be mediocre at summarizing your incident reports in the format your on-call engineers need. Popular benchmarks also leak into training data over time, which inflates scores without improving the model.

The working rule: benchmarks build your shortlist, your own prompt dataset picks the winner. HELM is a useful corrective here because it forces the comparison across several dimensions instead of one number.

Approach 2: an LLM as the judge

Judge-based evaluation uses a second model to score the first one. The job needs two models in two roles. The generator model answers the prompts in your dataset. The evaluator model reads each prompt and response pair, assigns a score, and writes an explanation of why it scored that way.

Bedrock provides built-in metrics you select from, and you can define custom ones for your own criteria. The built-ins cover more ground than any similarity score can:

Two things make this approach attractive. Most of these metrics need no reference answer, so you can evaluate open-ended work where no ground truth exists. And it runs at machine speed and machine cost, so you can score 500 responses in the time a human panel needs for 20.

You can also supply a reference response, and the evaluator will take it into account when scoring correctness and completeness. And if you are evaluating a model outside Bedrock, you can hand over your own inference responses and Bedrock skips the invoke step and scores the data you supplied.

The limit is the one from the analogy: the judge is a foundation model. It has preferences about phrasing and length, and it can be confidently wrong. Judge scores are strong evidence, not verdicts, and teams that depend on them spot-check a sample by hand.

Approach 3: human evaluation

Human evaluation puts people in the loop. In Bedrock you create a work team, which can be your own employees or subject-matter experts from your industry, managed through an Amazon Cognito user pool created from the console. A human job can rate one model or compare responses from two.

The rating mechanism decides what you learn:

The comparison mechanisms answer "which model do people prefer," and the individual mechanisms answer "is this model good enough." Both depend on the instructions you write. A 5-point scale with no defined anchors produces five reviewers using five different scales, and the histogram that comes back is noise.

Reach for humans when the quality bar is subjective (voice, tone, tact), when domain judgment is required (is this legal summary actually right), or when the stakes make a wrong answer expensive. Accept the cost: it is slower and more expensive than everything else here, which is why teams reserve it for the final decision and the periodic audit rather than the daily loop.

Choosing an approach

The decision rule that covers most scenario questions: if the question hands you a reference answer or asks for a repeatable score at volume, that is automatic. If it asks about qualities like helpfulness or faithfulness on open-ended output at volume, that is a judge model. If the words "subject-matter expert", "preference", "tone", or "subjective" appear, that is human evaluation.

These are not exclusive, and real teams stack them: automatic metrics in the pipeline on every change, judge metrics weekly on a larger sample, humans on the release candidate.

Exam tips

• The exam guide names two approaches directly, human evaluation and benchmark datasets. Recognize the third, judge-based evaluation, by its two-model structure: a generator and an evaluator.
• Accuracy and robustness need a reference response. Toxicity, helpfulness, and most judge metrics do not. Questions about "no ground truth available" are ruling out reference-based scoring.
• "Subject-matter experts", "preference between two models", or a subjective quality bar point at human evaluation. "Score thousands of open-ended responses cheaply" points at a judge model.
• Match the built-in dataset to its purpose: RealToxicityPrompts for toxicity, BOLD for bias in open-ended generation, Gigaword for summarization, BoolQ, NaturalQuestions, and TriviaQA for question and answer, TREX for factual knowledge.
• Any option claiming a public leaderboard rank proves fitness for a specific business use case is wrong.
• Robustness is about semantically neutral perturbations of the input, such as typos and casing. Do not confuse it with accuracy on clean input.

The point to carry: an evaluation approach is chosen by what evidence you have and what quality means for your application, not by which tool is most convenient. Next comes the layer underneath the word "accuracy", the specific metrics that turn two pieces of text into a number.