[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"cheat-sheet---en":3,"domain-info---en":3,"topic-info----en":3,"prev-aws-certified-ai-practitioner-ai-ml-fundamentals-ai-ml-development-lifecycle-anatomy-of-an-ml-pipeline-en":4,"lesson-aws-certified-ai-practitioner-ai-ml-fundamentals-ai-ml-development-lifecycle-anatomy-of-an-ml-pipeline-en":18,"next-aws-certified-ai-practitioner-ai-ml-fundamentals-ai-ml-development-lifecycle-anatomy-of-an-ml-pipeline-en":257},null,{"locked":5,"reason":6,"meta":7,"item":3},true,"paywall",{"title":8,"description":9,"isFree":10,"estimatedMinutes":11,"difficulty":12,"learningObjectives":13},"Traditional ML vs Foundation Models","How to choose between a task-specific traditional ML model and a large general-purpose foundation model, driven by explainability, regulation, cost, and operational constraints, not just capability.",false,17,"intermediate",[14,15,16,17],"Contrast traditional ML models and foundation models on scope, data needs, and how they are built","Choose between them based on explainability and regulatory requirements","Weigh operational constraints such as cost, latency, and control in the decision","Reject the assumption that the newer, more powerful foundation model is always the better choice",{"locked":10,"reason":3,"meta":19,"item":29},{"title":20,"description":21,"isFree":5,"estimatedMinutes":22,"difficulty":23,"learningObjectives":24},"Anatomy of an ML Pipeline","The stages that carry a model from raw data to monitored production: data collection, EDA, preprocessing, feature engineering, training, tuning, evaluation, deployment, and monitoring, and why the pipeline is a loop, not a line.",18,"beginner",[25,26,27,28],"List the stages of an ML pipeline in order, from data collection to monitoring","Distinguish EDA, data preprocessing, and feature engineering, which learners routinely confuse","Explain why deployment is not the finish line and monitoring closes the loop back to data","Match a described activity to the pipeline stage it belongs to",{"id":30,"title":20,"body":31,"description":21,"difficulty":23,"estimatedMinutes":22,"extension":195,"infographics":196,"isFree":5,"learningObjectives":215,"meta":216,"navigation":5,"path":217,"quiz":218,"seo":254,"stem":255,"__hash__":256},"courses/courses/aws-certified-ai-practitioner/en/domains/01-ai-ml-fundamentals/03-ai-ml-development-lifecycle/01-anatomy-of-an-ml-pipeline.md",{"type":32,"value":33,"toc":185},"minimark",[34,43,46,51,54,59,74,78,84,90,96,102,117,121,127,133,139,143,149,155,162,166,182],[35,36,37,38,42],"p",{},"A model that scores 95% accuracy in a notebook on your laptop is not a product. Between that promising result and a system your users can trust sits a series of steps that most beginners underestimate: getting the right data, cleaning it, shaping it, training, testing, shipping, and then watching the model in the wild so it does not quietly rot. That series of steps is the ",[39,40,41],"strong",{},"ML pipeline",", and the exam expects you to name its stages, tell the confusable ones apart, and understand why it never really ends. This lesson walks the whole path once, end to end.",[35,44,45],{},"Picture one concrete goal to anchor every stage: a bank wants a model that predicts which customers are about to close their accounts, so it can reach out before they leave. We will follow that churn model from raw data to production.",[47,48,50],"h2",{"id":49},"the-pipeline-at-a-glance","The pipeline at a glance",[35,52,53],{},"The pipeline has a natural order, and the order matters: each stage depends on the one before it, and you cannot skip ahead.",[55,56],"infographic",{"alt":57,"slug":58},"A left-to-right flow of ML pipeline stages from data collection through monitoring, grouped into preparing the data, building the model, and running in production, with a return arrow from monitoring back to data collection.","ml-pipeline-stages",[35,60,61,62,65,66,69,70,73],{},"Read it in three chunks. First you ",[39,63,64],{},"prepare the data"," (collection, EDA, preprocessing, feature engineering). Then you ",[39,67,68],{},"build the model"," (training, tuning, evaluation). Then you ",[39,71,72],{},"run it in production"," (deployment, monitoring). And monitoring loops back to the start. Now let us walk each stage with the churn model in hand.",[47,75,77],{"id":76},"preparing-the-data","Preparing the data",[35,79,80,83],{},[39,81,82],{},"Data collection"," comes first because a model can only learn from what you feed it. For the churn model, that means gathering account history, transaction records, support tickets, and login activity, then pulling them into one place. The rule that governs this whole phase: garbage in, garbage out. No later stage can rescue a model trained on the wrong or missing data.",[35,85,86,89],{},[39,87,88],{},"Exploratory data analysis (EDA)"," is the stage where you look before you leap. Before changing a single value, you explore what the data actually contains: how many accounts you have, which fields are missing, whether \"balance\" is sometimes negative, whether churned customers are rare (they usually are). EDA does not transform the data. It builds your understanding of it, and that understanding shapes every decision that follows.",[35,91,92,95],{},[39,93,94],{},"Data preprocessing"," is the cleaning stage. Real data is messy: missing values, duplicate rows, inconsistent formats (\"USA\" vs \"U.S.\" vs \"United States\"), obvious errors like an age of 200. Preprocessing fixes these so the data is consistent and usable. You fill or drop missing values, standardize formats, and remove bad records.",[35,97,98,101],{},[39,99,100],{},"Feature engineering"," is where you build the actual inputs the model will learn from, called features. This is not cleaning; it is creation. From a raw \"last login\" timestamp you might derive \"days since last login.\" From a year of transactions you might compute \"average monthly spend\" and \"spend trend over the last 3 months.\" Good features are often what separates a mediocre model from a strong one, because they hand the algorithm the signal in a form it can use.",[35,103,104,105,108,109,112,113,116],{},"These three data stages get confused constantly, so pin the distinction now. EDA is ",[39,106,107],{},"understand",", preprocessing is ",[39,110,111],{},"clean",", feature engineering is ",[39,114,115],{},"create",". A missing value: EDA notices it, preprocessing fills it, feature engineering has nothing to do with it. That minimal example separates all three.",[47,118,120],{"id":119},"building-the-model","Building the model",[35,122,123,126],{},[39,124,125],{},"Model training"," is the stage most people picture when they think of machine learning, though by now you can see it is only one step of many. You feed the prepared data to an algorithm, and it learns the patterns that link the features to the outcome (here, the features to \"churned or stayed\"). The output is a trained model.",[35,128,129,132],{},[39,130,131],{},"Hyperparameter tuning"," adjusts the settings that control how training happens. These are not learned from the data; you set them. Think of the number of layers in a network or how fast the model adjusts during training. Tuning tries different combinations to find the ones that produce the best model, often automatically. It is dialing in the knobs on the training process, not on the data.",[35,134,135,138],{},[39,136,137],{},"Evaluation"," is the honesty check. You measure the model on data it has never seen during training, a held-out test set, because a model that memorized its training data can look perfect and still fail on real customers. If the churn model catches only 40% of the customers who actually leave, evaluation is where you find out before your users do. The next lesson on metrics goes deep here; for now, know that evaluation is the gate between \"trained\" and \"trusted.\"",[47,140,142],{"id":141},"running-in-production","Running in production",[35,144,145,148],{},[39,146,147],{},"Deployment"," puts the model to work. You make it available so the rest of the bank's systems can send it a customer and get back a churn score, whether through an always-on endpoint answering in real time or a nightly batch job scoring every account at once. The following lesson covers those deployment choices in detail.",[35,150,151,154],{},[39,152,153],{},"Monitoring"," is the stage beginners forget, and it is where models quietly fail. A churn model trained on last year's behavior can slowly lose accuracy as customers change, the economy shifts, or the bank launches new products. This is called drift. Monitoring watches the live model, so you catch the decay and act, usually by collecting fresh data and retraining. That is the arrow that loops back to the start.",[35,156,157,158,161],{},"Here is the misconception to kill: that shipping the model is the finish line. It is not. A deployed model is a perishable asset. The pipeline is a ",[39,159,160],{},"loop",", not a line, because the world the model describes keeps moving, and a model that is never retrained is a model slowly going out of date.",[47,163,165],{"id":164},"exam-tips","Exam tips",[167,168,169,173,176,179],"ul",{},[170,171,172],"li",{},"Know the stage order and be ready to place an activity in it. \"Plotting distributions to understand the data\" is EDA; \"filling missing values\" is preprocessing; \"creating a days-since-last-purchase field\" is feature engineering.",[170,174,175],{},"The most common trap is blurring EDA, preprocessing, and feature engineering. Fix the verbs: understand, clean, create. A stem that describes cleaning is preprocessing, even if it uses the word \"prepare.\"",[170,177,178],{},"\"The model's accuracy dropped three months after launch\" is a monitoring-and-drift scenario, and the fix is retraining, not re-deploying the same model.",[170,180,181],{},"Watch for answers that treat deployment as the last stage. Monitoring comes after, and the pipeline loops back to data.",[35,183,184],{},"Carry one picture out of this lesson: data in, model out, production, and back to data again. Every AWS service in the next lessons plugs into one of these stages, and MLOps (later in this topic) is the discipline of running this whole loop reliably and repeatedly. Next you will see where those models come from and how you actually put them into production.",{"title":186,"searchDepth":187,"depth":187,"links":188},"",3,[189,191,192,193,194],{"id":49,"depth":190,"text":50},2,{"id":76,"depth":190,"text":77},{"id":119,"depth":190,"text":120},{"id":141,"depth":190,"text":142},{"id":164,"depth":190,"text":165},"md",[197],{"slug":58,"concept":198,"style":199,"aspectRatio":200,"labels":201},"A left-to-right flow of the ML pipeline as connected stages: Data collection, Exploratory data analysis, Data preprocessing, Feature engineering, Model training, Hyperparameter tuning, Evaluation, Deployment, Monitoring. Group the first four under a bracket labeled 'Prepare the data' and the middle three under 'Build the model' and the last two under 'Run in production'. A curved return arrow runs from Monitoring back to Data collection, labeled to show that drift and new data restart the cycle. The emphasis is that the flow is a loop, not a one-way line, and that the three early data stages are distinct.","diagram","16:9",[202,203,204,205,206,207,208,209,210,211,212,213,214],"Data collection: gather raw data from sources","EDA: look at the data, find gaps and patterns","Data preprocessing: clean, fix missing values, format","Feature engineering: build the inputs the model learns from","Model training: fit the algorithm to the data","Hyperparameter tuning: adjust the training settings","Evaluation: test on data the model never saw","Deployment: serve predictions to real users","Monitoring: watch quality, catch drift","Prepare the data","Build the model","Run in production","Monitoring feeds new data back to the start: the pipeline is a loop",[25,26,27,28],{},"/courses/aws-certified-ai-practitioner/en/domains/01-ai-ml-fundamentals/03-ai-ml-development-lifecycle/01-anatomy-of-an-ml-pipeline",{"passingScore":219,"questions":220},70,[221,226,234,239,248],{"question":222,"type":223,"options":224,"correctAnswer":88,"explanation":225},"A data scientist has just gathered raw data and, before changing anything, plots distributions and counts missing values to understand what the dataset actually contains. Which pipeline stage is this?","single",[100,88,125,147],"EDA is the investigate-and-understand stage: you look at the data, spot gaps, and find patterns before transforming anything. Feature engineering builds model inputs, training fits the algorithm, and deployment serves predictions, all of which come later and all of which depend on what EDA reveals.",{"question":227,"type":223,"options":228,"correctAnswer":230,"explanation":233},"Why is deployment not the final stage of the ML pipeline?",[229,230,231,232],"Because models never actually reach production","Because a live model can degrade as real-world data drifts, so monitoring must watch it and can trigger retraining","Because deployment always fails the first time","Because evaluation happens after deployment","A model is trained on a snapshot of the past, but the world keeps changing, so its accuracy can decay after launch. Monitoring is the ongoing stage that catches this drift and feeds new data back to the start, which is why the pipeline is a loop rather than a line.",{"question":235,"type":223,"options":236,"correctAnswer":94,"explanation":238},"Match the activity to the correct stage: filling in missing values and converting text categories into a consistent format.",[94,131,153,237],"Exploratory data analysis","Preprocessing is the cleaning stage: handling missing values, fixing errors, and formatting data so the model can use it. Feature engineering (a separate stage) then builds new predictive inputs from that clean data, and tuning adjusts training settings, not the data.",{"question":240,"type":241,"options":242,"correctAnswers":246,"explanation":247},"Which of the following are stages you would expect in a typical ML pipeline? (Select TWO.)","multiple",[131,243,244,245],"Model monitoring","Choosing a company logo","Writing the marketing email",[131,243],"Hyperparameter tuning (adjusting training settings to improve performance) and monitoring (watching the live model for drift) are both core pipeline stages. Choosing a logo and writing marketing copy are business activities outside the ML pipeline.",{"question":249,"type":223,"options":250,"correctAnswer":252,"explanation":253},"Feature engineering and data preprocessing are two names for exactly the same step.",[251,252],"True","False","They are distinct. Preprocessing cleans and formats the raw data (fixing missing values, standardizing formats), while feature engineering creates the actual predictive inputs the model learns from (for example, deriving 'days since last purchase' from a raw timestamp). Preprocessing makes data usable; feature engineering makes it informative.",{"title":20,"description":21},"courses/aws-certified-ai-practitioner/en/domains/01-ai-ml-fundamentals/03-ai-ml-development-lifecycle/01-anatomy-of-an-ml-pipeline","K_VbYgRJtfHGbGt2nDNQwqPg0ntIfr-eSGpmf82fNSc",{"locked":5,"reason":6,"meta":258,"item":3},{"title":259,"description":260,"isFree":10,"estimatedMinutes":261,"difficulty":12,"learningObjectives":262},"Model Sources and Deployment Options","Where the model in your application comes from (pre-trained open source, a managed model, or one you train yourself) and how you serve it in production: managed API versus self-hosted, and real-time, serverless, asynchronous, or batch inference.",20,[263,264,265,266],"Compare the three sources of a model: use a pre-trained model, customize one, or train your own","Distinguish a managed API service from a self-hosted API for serving a model","Choose among real-time, serverless, asynchronous, and batch inference by traffic and payload","Match a production scenario to the deployment option its constraints demand"]