AWS Certified AI Practitioner
How Foundation Models Are Trained
Where a base model's knowledge comes from: self-supervised pre-training on massive text, then instruction tuning and RLHF that turn a text predictor into a helpful assistant.
- Explain how pre-training uses self-supervised next-token prediction on large unlabeled text
- Distinguish the three stages that produce a usable foundation model: pre-training, fine-tuning, and alignment
- Describe what RLHF adds that labeled examples alone cannot
- Explain why you customize a pre-trained model instead of training one from scratch
By now you can place fine-tuning on the customization ladder: it sits above prompt engineering and RAG, it changes the model's weights, and you reach for it when behavior has to be consistent. But to understand what fine-tuning actually changes, you first need to know what pre-training already put into the model. A base model like Amazon Titan or Anthropic Claude arrives knowing an astonishing amount before you send it a single request. Where did that knowledge come from, and why does it cost a provider so much to make one?
The answer is a pipeline of three stages. Knowing what each stage does tells you which parts of a model you can realistically influence, which parts you inherit, and exactly where your own customization plugs in. That is a named exam objective, and it is also the difference between choosing the right customization method and wasting a training budget on the wrong one.
Pre-training: learning language from raw text
Pre-training is the stage that gives a foundation model its raw knowledge, and it works by a single, repeated task: predict the next token. Give the model the sequence "The capital of France is" and it learns to predict "Paris." Give it "def add(a, b): return" and it learns to predict "a + b." The model slides across the text token by token, guessing each next token and adjusting its billions of parameters whenever it is wrong.
Here is the part that makes it scale. Nobody labels this data. The correct answer for each position is simply the token that actually comes next in the text, so the text supervises itself. That is what self-supervised means, and it is why pre-training can consume trillions of tokens of books, websites, code, and articles without an army of annotators. The model is not memorizing a lookup table; it is compressing the statistical patterns of language, facts, and reasoning into its weights.
This is a sharp break from the supervised learning you met earlier in the course, where every training example came with a human-provided label. In pre-training there are no separate labels at all. The next word is the label. That single design choice is what let foundation models grow to the size they are.
One mechanical detail matters for why this is even feasible: the transformer architecture processes a whole sequence in parallel rather than one word at a time, which lets providers train on large fleets of GPUs. Even so, a full pre-training run takes weeks and costs a fortune in compute, which is the reason almost nobody does it themselves.
Why pre-training alone is not enough
A model fresh out of pre-training is called a base model, and it has a peculiar personality: it predicts plausible text, but it does not know it is supposed to be helpful. Ask a raw base model "What is the capital of France?" and it might continue with more trivia questions, because in its training data a question is often followed by more questions. It has the knowledge but not the manners.
Two more stages fix this. They use far less data than pre-training, and they shape behavior rather than pour in knowledge.
Fine-tuning and instruction tuning: teaching it to follow instructions
The next stage trains the base model on a much smaller set of labeled examples, each pairing an instruction with a good response. Shown thousands of examples like "Summarize this paragraph" followed by a clean summary, the model learns the general habit of doing what it is asked. Because the examples are instructions and their answers, this is often called instruction tuning, and it is a form of supervised fine-tuning.
This is the same mechanism you will use to customize a model later in this topic. The provider applies it first, broadly, to make a general assistant; you can apply it again, narrowly, to specialize that assistant for your task. The tool is identical, only the data and the goal differ.
Alignment with human feedback (RLHF)
There is one thing labeled examples handle poorly: subjective quality. What counts as a helpful, honest, and appropriately cautious answer is hard to write down as a single correct response. You know it when you see it, but you cannot easily label it.
Reinforcement learning from human feedback (RLHF) solves this by learning from comparisons instead of fixed answers. The model produces several responses to the same prompt, humans rank which they prefer, and those rankings train a separate reward model that predicts how a human would rate any response. The main model is then tuned to score well against that reward model. The result is the polished, aligned behavior you expect from a production assistant. Notice what RLHF does and does not do: it shapes tone and helpfulness, it does not teach new facts.
The three stages together
| Stage | Data | Who runs it | What it produces |
|---|---|---|---|
| Pre-training | Trillions of tokens, unlabeled | Model provider | A base model that predicts text |
| Fine-tuning / instruction tuning | Labeled instruction-and-response pairs | Provider (and you, later) | A model that follows instructions |
| Alignment (RLHF) | Human preference rankings | Model provider | A helpful, aligned assistant |
Read the table top to bottom and watch the data change. It starts enormous and unlabeled, shrinks to labeled pairs, then shrinks again to human judgments. Effort moves from raw scale toward careful curation.
What this means for you
You almost never run stage 1. Pre-training a foundation model demands data and compute on a scale that costs providers enormous sums, and paying for that from scratch would throw away the entire advantage of foundation models, which is that someone already did it. This is the same reason the course earlier contrasted traditional ML, where you train your own model, with foundation models, where you start from a pre-trained one.
Where you enter is customization, and it maps cleanly onto the stages you just saw. Continued pre-training extends stage 1 with your own domain text. Fine-tuning repeats stage 2 with your own labeled examples. The next lesson takes these methods apart and shows when each one fits.
Exam tips
- Pre-training is self-supervised, uses unlabeled text, works by next-token prediction, runs at massive scale, and is done by the provider. Those keywords travel together.
- "Unlabeled data" plus "learns general language and patterns" points at pre-training. "Labeled examples" plus "a specific task or behavior" points at fine-tuning.
- RLHF aligns the model to human preferences through a reward model. If a question says a technique adds new factual knowledge, RLHF is the wrong answer.
- Foundation models exist so you do not pre-train from scratch. An option that suggests training a large model from zero for a normal business use case is almost always the distractor.
- The base model you call in Bedrock is already the output of all three stages, whether or not you customize it further.
The base model in your Bedrock console is the finished product of three stages you never paid for: massive unlabeled pre-training, labeled instruction tuning, and preference-based alignment. Keep that map in your head, because your own customization is not a mysterious new thing. It is one of those same two tuning stages, run again on your data, which is exactly what the next lesson unpacks.