Syllaro Academy

AWS Certified AI Practitioner

Evaluation Metrics: ROUGE, BLEU, and BERTScore

The three metrics the exam names, worked with real sentences and real numbers, plus the metrics AWS actually computes for each task type and the blind spot all of them share.

Intermediate 20 minutes 5 Learning Objectives
  1. Define ROUGE and compute ROUGE-1 and ROUGE-L on a real sentence pair
  2. Explain how BLEU combines n-gram precision with a brevity penalty and why it is a corpus-level metric
  3. Explain how BERTScore differs from word-overlap metrics and what it catches that they miss
  4. Match each metric to the task it suits and name the blind spot they all share
  5. Identify which computed metric AWS uses for each evaluation task type