Predict Customer Churn with SageMaker

Build a no-code ML model that predicts telecom customer churn.

Introduction

30 Second Summary

Every subscription business loses customers. The ones that thrive are the ones that can predict who is about to leave and reach out before it happens.

In this project, you will build a machine learning model that predicts customer churn using Amazon SageMaker Canvas. No coding required.

What You'll Build

A no-code machine learning model that predicts customer churn for a telecom company, built entirely inside AWS's visual ML platform.

By the end of this project, you'll have:

A SageMaker AI domain configured with Canvas, ready to build models.
A trained binary classification model that predicts churn with approximately 95% accuracy.
Feature importance analysis showing which customer attributes drive churn, plus batch and single predictions on real data.
Secret Mission: Deploy your model to a real-time SageMaker endpoint and test it with live API requests.

Are there any prerequisites?

You need an AWS account (free tier or paid plan). New accounts receive up to $200 in credits, which cover this project's costs.

Set Up Your SageMaker Domain and Launch Canvas

Before you can build a machine learning model, you need a workspace. Amazon SageMaker Canvas is a no-code ML platform that handles the entire model lifecycle for you.

In this step, you will set up the infrastructure Canvas needs to run. This means creating a SageMaker AI domain (which provisions storage and compute) and then launching the Canvas application itself.

In this step, get ready to:

Create a SageMaker AI domain using Quick Setup.
Launch SageMaker Canvas and wait for provisioning to complete.
Confirm you have your churn dataset ready to import.

Create a SageMaker AI domain

A SageMaker AI domain is the foundation for everything you do in SageMaker. It provisions the storage, compute, and permissions Canvas needs to function.

Sign in to the AWS Management Console at https://console.aws.amazon.com.
In the search bar at the top of the console, type SageMaker and select Amazon SageMaker AI from the results.
In the left navigation pane, choose Canvas.
Choose Create a SageMaker AI domain.
Choose Set up. This triggers the Quick Setup process which creates your domain and user profile automatically.

What is a SageMaker AI domain?

A domain is a virtual boundary for your ML work. It creates an IAM role with the right permissions, provisions an S3 bucket for storing your data and model artifacts, and sets up a VPC for network access.

Quick Setup handles all of this automatically. You do not need to configure anything manually.

The domain creation takes a few minutes. Wait until you see the Canvas page with your user profile listed before continuing.

Domain creation stuck or failing?

Make sure you are using an account with administrative permissions. The Quick Setup requires permissions to create IAM roles and S3 buckets.

If you already have an existing SageMaker domain in this region, the console may skip the creation step and show your existing domain instead. That is fine. Proceed to launching Canvas.

help me troubleshoot my SageMaker domain creation

Launch SageMaker Canvas

With your domain ready, you can now launch the Canvas workspace. The first launch provisions dedicated compute resources for your session.

On the Canvas page in the SageMaker AI console, select your user from the dropdown.
Choose Open Canvas.
Wait 1-2 minutes for the Canvas workspace to finish provisioning. You will see a loading screen during this time.

Once provisioning completes, you land on the Canvas home screen. This is where you will import data, build models, and generate predictions.

What about Canvas workspace costs?

Canvas charges $1.90/hour while your workspace is running. The free tier provides 160 hours/month for 2 months. New AWS accounts also receive up to $200 in credits which cover this usage.

Always log out of Canvas when you are done working to stop workspace charges. You will learn how to do this in the Before You Go section.

Canvas not loading?

The first launch can take up to 2 minutes. If it takes longer than 5 minutes, try refreshing the page.

If you see a permissions error, return to the SageMaker AI console and verify that your domain was created successfully under Admin configurations > Domains.

help me get Canvas to load

Confirm your churn dataset is ready

You downloaded churn.csv earlier. This synthetic telecom dataset contains 5,000 customer records with 21 attributes.

Verify that churn.csv is on your local machine (check your Downloads folder).
Confirm the file is not empty. It should be approximately 350-400 KB.

What is in the churn dataset?

Each row represents a telecom customer. The columns include usage metrics like day minutes, evening minutes, night minutes, and international minutes. It also tracks how many times the customer called customer service.

The target column is Churn? which has two possible values: True (the customer left) or False (the customer stayed). This is what your model will learn to predict.

Your workspace is ready. Next up, you will import the churn dataset into Canvas and start exploring the data.

Import Data and Explore It

Your SageMaker Canvas workspace is up and running. Now it's time to feed it the data your model will learn from.

Before any machine learning model can make predictions, it needs to study historical examples. In this step, you will import the telecom churn dataset into Canvas, explore its columns, and create a new model ready for training.

In this step, get ready to:

Import the churn dataset into Canvas.
Explore the dataset columns and features.
Create a new model for churn prediction.

Import the dataset into Canvas

Canvas needs data to learn from. You will upload the churn.csv file you downloaded in the previous step directly from your computer into the Canvas workspace.

In the left navigation pane of Canvas, choose Datasets.
Choose Import data.
From the dropdown menu, choose Tabular.
In the popup dialog box, enter a name for the dataset (for example, churn) and choose Create.
On the data source page, open the Data Source dropdown menu and choose Local upload.
Choose Select files from your computer and select the churn.csv file from your local machine.
Choose Preview dataset to confirm the data looks correct.
Choose Create dataset to finish the import.

The import takes about 10 seconds. Once complete, your dataset appears on the Datasets page.

What just happened?

Canvas uploaded your CSV file and stored it in the background in an Amazon S3 bucket that was automatically created when you set up your domain. You do not need to manage this bucket directly.

Canvas can import data from over 40 sources including S3, Snowflake, and Salesforce. For this project, a local file upload is the simplest path.

Don't see your file after import?

Make sure you selected Tabular in the data type dropdown. Canvas also supports image and document datasets, so selecting the wrong type hides CSV files.

If the import seems stuck, refresh the browser tab and return to the Datasets page to check whether the file appears in the list.

help me troubleshoot my Canvas data import

Visually inspect the data

Before building a model, you should understand what your data looks like. Canvas shows a preview of all columns so you can identify which ones are numeric, which are categorical, and which one you want to predict.

On the Datasets page, select your churn dataset to open the preview.
Scroll through the columns to review the data types and sample values.

You should see 21 columns across the dataset. Here is what to look for:

Categorical columns like State, International Plan, and Voice Mail Plan (text values like "yes" or "no").
Numeric columns like Account Length, Day Mins, Eve Mins, Night Mins, Intl Mins, and CustServ Calls (numbers representing usage and behavior).
The target column called Churn? with two possible values: True (customer left) and False (customer stayed).

Why does the mix of column types matter?

Machine learning models handle numeric and categorical data differently. Numeric columns (like call minutes) represent continuous measurements. Categorical columns (like State or International Plan) represent distinct groups.

Canvas handles these differences automatically during training. You do not need to manually encode or transform anything.

Create a new model

Now that your data is imported, you can create a model that Canvas will train to predict customer churn.

In the left navigation pane, choose My Models.
Choose New model.
For the model name, enter CustomerChurnPredictor.
Select Predictive analysis as the problem type.
Choose Create.

Canvas takes you to the model configuration page where you will later select your dataset and target column.

What is Predictive analysis?

Predictive analysis is the model type Canvas uses when you want to predict an outcome based on historical data. Canvas supports several problem types including numeric prediction, 2 category prediction, 3+ category prediction, and time series forecasting.

Since the Churn? column has only two possible values (True or False), Canvas will automatically detect this as a binary classification problem in the next step when you select it as the target.

Don't see My Models in the navigation?

Make sure you are on the main Canvas home screen. If you are still inside the dataset preview, use the back arrow or the Canvas logo to return to the home page.

help me find My Models in Canvas

Your dataset is imported and your model shell is ready. In the next step, you will connect the dataset to this model, choose your target column, and kick off the training process.

Build and Train Your Model

Your data is imported and your model shell is ready. Now it is time to tell SageMaker Canvas what to predict and let it train a model automatically.

In this step, you will point your model at the churn dataset, select the column you want to predict, and kick off a Quick Build. Canvas handles all the algorithm selection and hyperparameter tuning behind the scenes.

In this step, get ready to:

Configure the dataset and target column for your model.
Run Quick Build to train the model.
Review the training accuracy results.

Configure the model

Before Canvas can train, it needs two things: the dataset to learn from and the specific column it should predict. The target column is the answer your model learns to output for each row.

In Canvas, make sure you are inside your CustomerChurnPredictor model.
On the Select tab, choose the radio button for the churn.csv dataset.
Choose Select dataset.

Canvas moves you to the Build tab automatically.

In the Target column dropdown, select Churn?.

Canvas detects that this column has only two possible values (True and False) and automatically sets the model type to a 2 category prediction.

What is a target column?

The target column contains the answer your model learns to predict. Every other column in the dataset becomes an input feature that Canvas uses to find patterns.

Think of it as the question on a test. The features are the study material, and the target is the correct answer.

What is binary classification?

Binary classification is a type of machine learning problem where the model predicts one of exactly two outcomes. In your case, the model predicts whether a customer will churn (True) or stay (False).

Canvas calls this a "2 category prediction" because the target column contains two categories. It handles all the algorithm configuration for you.

Run Quick Build

Canvas offers two training modes. Quick Build prioritizes speed by testing fewer model combinations. Standard Build tests hundreds of combinations for higher accuracy but takes 2-4 hours.

On the Build tab, choose Quick build.

Canvas begins training your model. This takes approximately 2-15 minutes depending on system load.

What happens during Quick Build?

Canvas uses AutoML to automatically try different algorithms and settings on your data. It splits the dataset into training and validation sets, trains multiple candidate models, and picks the best performer.

Quick Build trains fewer combinations to prioritize speed. Standard Build tests hundreds of candidate pipelines for higher accuracy but takes 2-4 hours.

Grab a coffee while it trains

Quick Build typically finishes in 2-15 minutes. You do not need to keep the browser tab active. Canvas will show your results when you return.

Review the training results

Once training completes, Canvas automatically moves to the Analyze tab. This is where you see how well your model learned to predict churn.

Confirm you are on the Analyze tab.
Check the accuracy score displayed at the top of the page. You should see approximately 95-96% accuracy.

What does 95% accuracy mean?

An accuracy of ~95% means that for every 100 customers in the validation set, the model correctly predicted whether they churned or stayed for about 95 of them. This is a strong result for a first model built in minutes without any manual tuning.

Seeing a lower accuracy than expected?

If your accuracy is significantly lower than 95%, double check that you selected Churn? as the target column (not a different column like Phone or State). If the wrong target was selected, go back to the Build tab, change the target column, and re-run Quick Build.

Why is my model accuracy low?

Your model is trained and showing strong results. Next up, you will dig into the feature importance chart to understand which customer behaviors actually drive churn.

Analyze Model Performance and Feature Importance

Your model is trained and reporting approximately 95-96% accuracy. That number sounds great, but a percentage alone doesn't tell you why the model makes its predictions.

In this step, you will dig into the model's analysis to understand which customer attributes drive churn. These insights are what turn a prediction model into an actionable business tool.

In this step, get ready to:

Review the Column impact chart to identify the top churn drivers.
Explore the Scoring tab and Advanced metrics to understand prediction accuracy.
Interpret business insights from feature importance patterns.

Understand column impact

Feature importance tells you how much each column in your dataset contributes to the model's predictions. Canvas calls this Column impact.

On the Analyze tab, locate the Overview section at the top of the page.
Review the Column impact chart, which ranks each feature by how much it influences the model's churn prediction.
Notice which features appear at the top of the chart. For this dataset, key drivers typically include CustServ Calls, Day Mins, Eve Mins, and Int'l Plan.

What is Column impact?

Column impact is Canvas's way of showing feature importance. It estimates how much each column contributes to the model's ability to predict the target variable (Churn?).

A high-impact column like CustServ Calls means that changes in customer service call volume strongly influence whether the model predicts churn. A low-impact column means it barely affects the prediction.

Explore the Scoring tab and Advanced metrics

Accuracy gives you a single number, but the Scoring tab breaks down exactly where the model gets it right and where it makes mistakes.

Within the Analyze tab, choose the Scoring tab.
Review the Sankey diagram that maps predictions to actual outcomes. This visual shows how many customers the model predicted as True (will churn) or False (will not churn) compared to what actually happened.
Choose Advanced metrics to open the confusion matrix view.
Review the four quadrants of the confusion matrix.

What do the confusion matrix quadrants mean?

True Positive: The model correctly predicted a customer would churn, and they did.
True Negative: The model correctly predicted a customer would stay, and they did.
False Positive: The model predicted churn, but the customer actually stayed.
False Negative: The model predicted the customer would stay, but they actually churned.

For a churn model, false negatives are the most costly. They represent at-risk customers the model missed entirely.

Don't see the Scoring tab?

Make sure you are on the Analyze tab for your model. If you navigated away, choose My models in the left navigation, select CustomerChurnPredictor, and then choose the Analyze tab at the top.

help me find the Scoring tab and Advanced metrics in SageMaker Canvas

Interpret business insights from feature importance

The column impact chart is not just a technical readout. It tells a business story about why customers leave.

Return to the Overview section of the Analyze tab.
Look at how CustServ Calls impacts churn. Both very high and very low customer service call volumes increase the likelihood of churn.
Consider what actions a business could take based on these patterns. For example, improving self-service FAQs could reduce call volume, while proactive outreach could catch disengaged customers who never call at all.

Why does feature importance matter for business?

A prediction model that says "this customer will churn" is useful. A model that also says "they will churn because their customer service calls spiked" is actionable.

Feature importance turns predictions into targeted interventions. A marketing team can run retention campaigns focused on the exact behaviors that signal risk.

You now understand not just how accurate your model is, but why it makes the predictions it does. Next up, you will put this model to work by generating actual predictions on customer data.

Generate Predictions

Your model is trained and you've explored what drives customer churn. Now it's time to put that model to work by generating actual predictions on customer data.

SageMaker Canvas offers two prediction modes. Batch predictions score an entire dataset at once, giving you a churn prediction for every customer. Single predictions let you tweak individual values and watch how the churn probability changes in real time.

In this step, get ready to:

Generate batch predictions on your full customer dataset.
Run single predictions by modifying feature values in real time.
Download your prediction results as a CSV file.

Generate batch predictions

Batch predictions let you score an entire dataset in one go. You'll get a churn prediction and confidence score for every single customer record.

Choose the Predict tab at the top of the model page.
Select Manual for a one-time batch prediction.
Select the churn.csv dataset from the list.
Choose Generate predictions.

Canvas processes the entire dataset and outputs a prediction for each customer. This takes a few moments depending on dataset size.

What does batch prediction do?

Batch prediction runs your trained model against every row in the selected dataset. For each of the 5,000 customer records, Canvas produces a predicted Churn? label (True or False) along with a confidence score indicating how certain the model is about that prediction.

In a real business scenario, you'd use these results to identify at-risk customers and prioritize retention outreach based on the confidence score.

Run a single prediction

Single prediction mode lets you play "what-if" scenarios. You can change individual feature values and immediately see how the churn probability updates.

Switch to Single prediction mode at the top of the Predict tab.
Find the CustServ Calls field and change its value to 5.
Find the Int'l Plan field and set it to yes.
Choose Update to see how these changes affect the churn probability.

Watch how the predicted churn probability shifts as you modify these values. A customer with 5 service calls and an international plan is typically flagged as high-risk for churn.

Why does this matter for business decisions?

Single predictions simulate real scenarios your business might face. If a customer calls support 5 times, you can instantly see their churn risk and decide whether to proactively offer a retention deal.

Try experimenting with other values too. Reduce CustServ Calls back to 1 and watch the probability drop. This directly shows the relationship your model learned between service calls and churn.

Single prediction not updating?

Make sure you choose Update after modifying your values. The prediction does not refresh automatically when you type new numbers.

If the prediction takes a while on first use, that's normal. Canvas provisions an inference endpoint behind the scenes which can take up to a minute on the first request.

Help me troubleshoot my Canvas single prediction

Download your results

Your batch predictions are ready to download. The output file contains the original customer data plus the model's predictions, so you can share actionable results with your team.

Switch back to the batch prediction results view (if you navigated away, choose Predict and then Manual).
Preview the results to see the predicted churn labels and confidence scores added to each row.
Choose Download to save the batch prediction results as a CSV file to your local machine.

What's in the downloaded CSV?

Each row in the output contains the original 21 columns of customer data, plus a predicted Churn? label (True or False) and a confidence score. The confidence score tells you how certain the model is about each prediction.

A customer predicted as True with 0.95 confidence is almost certainly going to churn. A customer predicted as True with 0.55 confidence is borderline and might need a different retention strategy.

Open the downloaded CSV file on your computer to confirm it contains the original data plus the prediction columns.

Can't find the downloaded file?

Check your browser's default downloads folder. The file is typically named with a timestamp or your model name.

If the download button is greyed out, your batch prediction may still be processing. Wait for the results to fully load before downloading.

Help me download my Canvas batch prediction results

Secret mission

Deploy Your Model to a Real-Time Endpoint

Your model works great inside Canvas. But what if you want other applications to use it? In this secret mission, you'll deploy your trained model to a live SageMaker endpoint that can serve predictions via API. This is how production ML models work in the real world.

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. Canvas only charges for workspace time while you are logged in ($1.90/hour after the free tier). Your trained model and dataset are stored in S3 at minimal cost.

Cost warning

SageMaker Canvas charges $1.90/hour while you are logged in. The free tier covers 160 hours/month for 2 months. If you completed the Secret Mission, real-time endpoints charge per hour while running. Make sure your endpoint is deleted if you are done testing.

Resources you used:

SageMaker AI Domain (created via Quick Setup).
SageMaker Canvas workspace (running while you are logged in).
CustomerChurnPredictor model (stored in S3).
Imported dataset (churn.csv stored in S3 via Canvas).
Batch predictions CSV (downloaded locally).
Real-time endpoint (Secret Mission only, should already be deleted).

Keep everything running

No action needed. Choose this if you are still actively exploring your model or want to experiment with different prediction inputs.

Your domain and trained model persist at minimal S3 storage cost.
Canvas only charges while you are actively logged in.
If you step away, Canvas automatically shuts down after 2 hours of inactivity (if configured by your admin). Otherwise, remember to log out manually.

Pause - I'll come back to this later

Log out of Canvas to stop workspace charges. Your models and datasets remain intact and ready for when you return.

In the SageMaker Canvas application, choose the Log out button in the left panel.
Your models and imported datasets are not affected by logging out.
When you are ready to return, relaunch Canvas from the SageMaker AI console. Logging back in takes about 1 minute.

Delete - I don't want to use this again

Remove all project resources to stop any potential charges. Follow these steps in order.

Confirm your real-time endpoint is deleted (Secret Mission only). In the SageMaker AI console, choose Inference in the left navigation, then Endpoints. Verify no active endpoints remain.
Close the browser tab where Canvas is open.
In the SageMaker AI console, choose Admin configurations in the left navigation pane.
Under Admin configurations, choose Domains.
Select your domain, then select your user profile under User profiles.
Under Apps, find the row where App type says Canvas. In the Action column, choose Delete app.
In the Delete app dialog, select Yes, delete app, type delete in the text field, and choose Delete.
Wait until the Status column shows Deleted. This confirms the Canvas workspace instance has been terminated.
To also delete the SageMaker AI domain, return to Admin configurations > Domains, select your domain, choose the User profiles tab, and delete each user profile. Then choose the Domain settings tab, find the Delete domain section, and choose Delete domain.
Delete your locally downloaded batch predictions CSV file if you no longer need it.

Nice Work!

You just built a machine learning model that predicts customer churn without writing a single line of code.

Here's what you accomplished:

Trained a binary classification model using Amazon SageMaker Canvas Quick Build on a 5,000-record telecom dataset.
Analyzed feature importance to understand which customer attributes (like customer service calls and day minutes) drive churn predictions.
Generated batch and single predictions with confidence scores, turning raw customer data into actionable churn risk assessments.
Deployed your model to a live SageMaker real-time endpoint for API-accessible predictions.

Ready to quiz yourself?