AWS Certified AI Practitioner
Inference Parameters
Same prompt, same model, different answers: the inference parameters that control how a model chooses each token, when to reach for each one, and the settings that quietly break a feature.
Intermediate 18 minutes 5 Learning Objectives
- Explain how a model chooses each token from a probability distribution
- Describe what temperature, top-k, and top-p each do to that distribution
- Distinguish reshaping the distribution from truncating it
- Choose settings for a factual task versus a creative task
- Identify what response length, penalties, and stop sequences control