Choosing a Foundation Model

The previous domain already taught you how to choose a model: eliminate on hard constraints, evaluate the survivors on your own task, then optimize the ones that pass for cost and latency. That method is the order of operations. This lesson is the checklist it operates on.

The exam guide, in Task Statement 3.1, names a specific set of criteria for choosing a pre-trained foundation model: "cost, modality, latency, multilingual, model size, model complexity, customization, and input/output length." Each one is a different kind of question, and each carries a design consequence you feel later if you skip it. A question on this topic usually hides one of these criteria in the scenario wording and makes the tempting answer the model that would win if that sentence were not there. So the useful skill is recognizing which criterion a scenario is really testing.

We will walk them in the order they tend to bite, not the order the guide lists them.

Modality: what goes in, what comes out

Modality is the kind of data a model accepts and produces. A text model reads and writes text. A multimodal model can accept images, audio, or video alongside text. An embedding model returns vectors instead of prose. An image generation model produces pictures from a description.

These are not interchangeable, and no other criterion can rescue a modality mismatch. A text-only model cannot read a scanned invoice, transcribe a call, or describe a photo, and fine-tuning does not add an input type. This is why modality is the first thing to check: it eliminates candidates outright, before quality is even on the table.

It is also the criterion the exam disguises most often. A scenario mentions scanned documents, product photos, or audio recordings, then offers a text-only model as the option with the best benchmark scores. The scores are a distraction. Read the scenario for what the input actually is before you read the model's capabilities.

Input and output length: two limits, not one

Every model publishes two length limits, and confusing them is a reliable way to ship a broken feature.

The context window is the maximum number of tokens the model can take in on a single request. Everything counts against it: your instructions, any retrieved documents, the conversation so far, and the user's question. The maximum output length is a separate cap on how many tokens the model can generate in one response, and it is usually far smaller than the context window.

The gap between them is larger than most people expect. A model can publish a 200,000-token context window with a maximum output of only a few thousand tokens. That combination reads a very long document comfortably but cannot write a very long one. A team that reads "200,000 tokens" and plans to generate a full report in one call gets a response that stops in the middle, and nothing in the error explains why.

So the design question splits in two. How much does a single request need to take in, which sets the context window floor, and how long does the answer need to be, which sets the output-length floor. A summarization feature is heavy on input and light on output. A long-form drafting feature is the reverse. Match the model to whichever direction your workload leans.

Multilingual support

A model trained mostly on English will still produce Spanish or Arabic when asked, but often with weaker grammar, thinner vocabulary, and more mistakes than it makes in English. General capability does not transfer evenly across languages, which is why the guide lists multilingual support as its own criterion rather than folding it into accuracy.

The consequence is practical. If your users write in Portuguese and read answers in Portuguese, a model that leads English benchmarks but stumbles in Portuguese is the wrong choice, and you only see the problem by testing in the language your users actually use. When a scenario names a non-English audience or a multi-language requirement, this criterion moves to the front.

Model size and model complexity

The guide lists model size and model complexity as two criteria, and they point at the same tradeoff from two angles. Size refers to the number of parameters. Complexity refers to how much capability that buys and what it costs to run.

The instinct almost everyone brings is that a bigger model is a safer choice. It is not, and the exam rewards knowing why. A larger model costs more per token, generates more slowly because it does more work per token, and often shows no measurable improvement on a narrow, well-specified task. A classifier sorting support tickets into eight categories does not need frontier reasoning. Paying for capability the task never exercises is waste that recurs on every single request, forever.

The pattern that holds up in practice is to start with a smaller model and move up only when your evaluation shows a real gap. Teams that start at the top rarely move back down, because nobody volunteers to make the assistant slightly worse to save money, so the overpayment becomes permanent. When a scenario gives you a tight latency budget or a simple, bounded task and offers a large model as an option, the smaller model that still passes is usually the intended answer.

Customization support

If your plan depends on adapting the model, whether by fine-tuning it on your examples or grounding it in your data, you have to confirm the model supports that path before you commit to it. Not every model can be fine-tuned, and the ones that can differ in which methods they accept and what data format they expect.

This is a selection criterion precisely because discovering the limit late is expensive. A team that picks a model on quality alone, builds around it, and then learns it cannot be fine-tuned in the way their use case needs has to start the selection over. The customization tradeoffs themselves are a later lesson in this topic. Here the point is narrower: if customization is part of the plan, it is part of the selection.

Latency and cost

These are the two criteria you optimize once a model has cleared everything above, and the earlier lessons covered them in depth, so we keep it short.

Latency is how quickly the model responds. Larger models are slower, and generation is sequential, so a long answer from a large model can take several seconds. A live chat experience has a latency budget; a nightly batch job usually does not. Verify a model meets the budget before you build a real-time feature on it, rather than hoping to close the gap afterward.

Cost is driven mostly by model choice, more than by any prompt optimization. The same workload can differ by orders of magnitude between two models in the same family. Among the models that pass every other gate, the cheapest fast one that still meets your quality bar is the right pick.

The criteria at a glance

Exam tips

• Memorize the named list: cost, modality, latency, multilingual, model size, model complexity, customization, input and output length. Then, for any scenario, find which one is doing the deciding.
• Modality is the most common hidden filter. Non-text input in the scenario plus a text-only model in the options means that option is wrong, whatever its scores.
• Input length and output length are separate limits. A large context window does not promise a long response. Watch for a scenario that needs long generation and a model with a small output cap.
• Multilingual support surfaces when the audience is named as non-English. General benchmark scores do not settle it.
• Model complexity is a trade, not a target. Tight latency or a narrow task points at the smaller model that still passes.
• Customization support is checked before selection, not after. If the plan says fine-tune, confirm the model can be fine-tuned.

The rule to carry into a question: read for the constraint first, then match it to one of the eight criteria. Most model-choice scenarios turn on a single sentence. The next lesson moves from choosing the model to controlling how it responds, using the inference parameters you set on every request.