Smart Image Analyzer with Rekognition

Use Amazon Rekognition ML to detect objects, analyze faces, and read text.

Introduction

30 Second Summary

Every time your phone unlocks with your face or tags a friend in a group photo, machine learning is doing the heavy lifting. You are about to use that same technology yourself.

In this project, you will upload your own photos to Amazon Rekognition and watch ML models analyze them in real time. Each feature you try solves a limitation of the previous one, building your understanding of how ML works from the ground up.

What You'll Build

You will upload a photo, click analyze, and instantly see a detailed breakdown of everything the ML model detected, each result tagged with a confidence score showing how certain the model is.

By the end of this project, you'll have:

Uploaded a photo and watched label detection identify every object, scene, and concept in it, each with a confidence score showing how sure the model is.
Analyzed a face photo and watched the model predict emotions, estimate an age range, and map facial landmarks like eyes, nose, and mouth.
Compared two different photos to determine if they contain the same person, complete with a similarity score measuring the match.
Secret Mission: Detected and extracted printed text from images using ML-powered text recognition.

Do I need an AWS account?

Yes, you will need an AWS account to access Amazon Rekognition. If you do not have one yet, Step 1 walks you through creating a free account.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up Your AWS Console

Before you can use machine learning to analyze images, you need access to the Amazon Rekognition console. This console is where you will upload photos and see ML results in real time.

In this step, you will sign into AWS and navigate to the Rekognition console. You will also prepare a set of images to use throughout the project.

In this step, get ready to:

Sign into the AWS Management Console.
Set up an IAM user for this project.
Open the Amazon Rekognition console.
Prepare your images for analysis.

Sign into the AWS Management Console

The AWS Management Console is the web portal where you access all AWS services, including Rekognition. You need to be signed in before you can analyze any images.

✔️ I already have an AWS account

Open https://console.aws.amazon.com/ in your browser.
Select Root user.
Enter the email address you used to create your AWS account.
Click Next.
Enter your password and click Sign in.

You should see the AWS Management Console dashboard with a search bar at the top.

ⓧ I need to create an AWS account

Open https://aws.amazon.com/free/ in your browser.
Click Create a free account.
Enter your email address and choose a password.
Enter your contact information.
Enter a valid payment method when prompted.
Choose the Free account plan during sign-up.

Will I be charged?

AWS will not charge you unless you upgrade to a paid plan. You will receive $100 in credits immediately when you choose the free account plan.

Account activation usually takes a few minutes but can take up to 24 hours. Once your account is active, sign in at https://console.aws.amazon.com/.

Having trouble signing in?

Double check that your email address and password are correct. If you recently created your account, it may still be activating. Try again in a few minutes.

Help me troubleshoot my AWS sign-in issue.

Set up an IAM user for this project

When you create an AWS account, you get a root user that has full access to everything. Using the root user for everyday tasks is risky because a single mistake could affect your entire account. IAM users let you create a separate identity with only the permissions you need.

✔️ I already have an IAM user

Sign out of the root account if you are currently signed in.
Open https://console.aws.amazon.com/ and sign in with your IAM user credentials.

You should see your IAM username in the top-right corner of the console instead of your root email address.

ⓧ I need to create an IAM user

Click the search bar at the top of the AWS Management Console.
Type IAM and choose IAM from the search results.
In the left navigation, click Users.
Click Create user.
Enter a username of your choice (for example, nextwork-admin).
Check the box for Provide user access to the AWS Management Console.
Select I want to create an IAM user.
Choose Custom password and enter a strong password you will remember.

Keep your password safe

Save your IAM username and password somewhere secure. You will use these credentials to sign in from now on.

Uncheck Users must create a new password at next sign-in (since you are creating this user for yourself).
Click Next.
Select Attach policies directly.
In the search box, type AdministratorAccess.
Check the box next to AdministratorAccess.
Click Next.
Review the details and click Create user.

You should see a success message confirming your new user was created.

Copy the sign-in URL shown on the success page. It looks like https://123456789012.signin.aws.amazon.com/console.
Sign out of the root account by clicking your account name in the top-right corner, then clicking Sign out.
Paste the sign-in URL into your browser.
Enter your IAM username and password to sign in.

You should see the AWS Management Console dashboard with your IAM username in the top-right corner.

What is an IAM user?

Your root user is like the master key to your house. An IAM user is like a copy you carry every day. If you lose the copy, you can always use the master key to make a new one.

AWS recommends using an IAM user for day-to-day work and keeping the root account for emergencies. The AdministratorAccess policy gives your IAM user full permissions to all AWS services, which is everything you need for this project.

Open the Amazon Rekognition console

Now that you are signed in, you need to find the Rekognition service inside the console. AWS has hundreds of services, so the fastest way to find one is the search bar.

Click the search bar at the top of the AWS Management Console.
Type Rekognition.
Choose Amazon Rekognition from the search results.

You should see the Amazon Rekognition console landing page. The left navigation includes options like Label detection, Face detection, and Face comparison. You will use each of these in the upcoming steps.

Don't see Amazon Rekognition in search?

Make sure you are typing Rekognition (with a "k", not a "c"). Also confirm you are signed into the AWS Management Console and not on a different AWS page.

Help me find Amazon Rekognition in the AWS console.

Prepare your images for analysis

Throughout this project, you will upload your own photos to Rekognition and watch ML models analyze them. Each step uses a different type of image to demonstrate a different ML capability.

Find or take the following photos and save them to your computer:

One clear photo of an outdoor scene or everyday objects (a park, a street, a desk with items).
One clear front-facing portrait or selfie.
One photo of a person wearing sunglasses or showing a side profile.
One group photo with multiple people.
Two different photos of the same person (taken at different times or in different settings).
One photo of a different person (someone who is not in the two photos above).
One photo containing clearly visible text (a book cover, street sign, product label, or poster).

Verify that every image is in JPEG or PNG format. Check the file extension at the end of the filename. It should end in .jpg, .jpeg, or .png.
Verify that every image is under 5MB in size. Right-click the file and select Get Info (macOS) or Properties (Windows) to check. Photos taken on modern smartphones are usually under this limit.

Why these specific images?

Each image maps to a different Rekognition feature you will use. The outdoor scene is for detecting objects and scenes. The portrait is for analyzing facial attributes like emotions and age. The sunglasses or side profile photo tests how face analysis handles partially hidden faces.

The group photo shows how face analysis handles multiple faces at once. The two photos of the same person are for face comparison, where Rekognition determines if two different images contain the same individual. The different-person photo lets you test what happens when faces do not match.

The text-containing photo is for the Secret Mission, where you will use Rekognition to read printed words inside images.

Your AWS console is ready and your images are prepared. In the next step, you will upload your first photo and watch Rekognition identify every object and scene it finds.

Detect Objects and Scenes in Your First Image

In the last step, you signed into AWS and got the Amazon Rekognition console ready. Everything is in place, but you have not seen machine learning do anything yet.

That changes now. Label detection is the fastest way to experience ML in action. You will upload a photo and watch Rekognition identify every object, scene, and concept it finds. Each detection comes with a confidence score that tells you how certain the model is about what it found.

In this step, get ready to:

Explore label detection results on a pre-loaded sample image.
Upload your own photo and interpret ML confidence scores.
Analyze multiple image types to discover how ML categorizes objects, scenes, and concepts.

Explore the sample image in label detection

Rekognition's label detection page comes pre-loaded with a sample image so you can see ML results immediately, before uploading anything of your own.

In the Amazon Rekognition console, choose Label detection from the left navigation panel.

A sample image loads automatically. It shows a person doing a stunt on a skateboard in a city street. The analysis results are already displayed on the right.

Examine the Labels | Confidence pane on the right side of the page.
Notice how each detected label (like Skateboard, Person, Sport) has a confidence percentage next to it.
Click Show more in the Labels | Confidence pane to reveal all detected labels, including ones with lower confidence scores.

You should see a long list of labels. Some near the top have confidence scores above 99%, while others further down drop into the 50-60% range.

What is label detection?

Label detection is an ML feature that scans an image and identifies everything it recognizes. It returns a list of labels, where each label is a word or phrase describing something the model found.

The confidence score next to each label tells you how sure the model is. A label at 99% means the model is very certain. A label at 55% means the model sees something that might be there, but it is not confident. This uncertainty is how every ML prediction works.

Upload your own photo and analyze confidence scores

The sample image shows you what label detection can do. Now it is your turn to test it with your own photo.

Before you upload, look at the outdoor or everyday scene photo you prepared. What objects and scenes do you think Rekognition will detect? Will it catch everything in the image, or miss some things?

Click Upload above the image area.
Navigate to the outdoor or everyday scene photo you prepared and select it.
Wait a moment for Rekognition to process your image. The results appear in the Labels | Confidence pane.

Compare your results to what you predicted. Did Rekognition catch everything you expected? Did it find things you did not notice in the photo yourself?

Look for labels with high confidence (above 90%). These are detections the model is very certain about.
Look for labels with lower confidence (55-70%). These are detections where the model is less sure about what it found.

Why do some labels have higher confidence than others?

ML models do not give simple yes/no answers. They assign a probability to every prediction. A label at 99.5% means the model is almost certain that object exists in the image. A label at 58% means the model sees something that might be there, but it is not confident.

This is a fundamental property of machine learning. Every prediction is a probability, not a guarantee. The clearer and more prominent something is in a photo, the higher its confidence score tends to be.

🙋‍♀️ Image not uploading?

Make sure your image is in JPEG or PNG format and under 5MB in size. Other formats or larger files are not accepted by the console demo.

If you see no labels appearing after upload, try refreshing the page, choosing Label detection from the left navigation again, and re-uploading.

Help me troubleshoot my Rekognition image upload.

Compare labels across different image types

One image gives you a snapshot of what label detection can do. Uploading different types of images reveals how the ML model categorizes the world.

Click Upload again and select one of the other photos you prepared (your portrait, group photo, or text-containing image).
Review the new set of labels in the Labels | Confidence pane. Notice how the labels are completely different from your first image.
Click Upload one more time and try a third image. Compare the labels across all three uploads.

Across your images, notice how the labels fall into different categories. Some labels name specific items. Others describe the environment. Others capture abstract ideas.

Three types of labels

Rekognition's labels fall into three categories. Objects are specific items like Table, Dog, or Car. Scenes are environments like Outdoor, Restaurant, or Beach.

Concepts are abstract ideas like Nature, Adventure, or Architecture. Each image tells a different story through the unique mix of labels it generates.

You have now analyzed multiple images using label detection. Each image produced a unique set of labels with confidence scores, showing you how ML interprets different types of visual content.

Take one more look at any image that contained a person. You will see a label like Person in the results. But that is all label detection can tell you. It cannot identify emotions, estimate an age, or describe facial features. That gap is exactly what the next step addresses.

Analyze Faces and Emotions

You've used Label Detection to identify objects, scenes, and concepts in your photos. You even saw it label a person as simply "Person" with a high confidence score.

But that is all Label Detection could say about a person in the photo. It could not tell you their emotions, their approximate age, or whether they were wearing glasses. That is a real limitation. Face Analysis is a completely different machine learning model, trained specifically on human faces, and it picks up exactly where Label Detection stops.

In this step, get ready to:

Upload a portrait and review detected facial attributes like emotions and age range.
Compare face detection accuracy between clear and partially obscured faces in a group photo.
Test how sunglasses and side profiles affect ML accuracy.

Upload a portrait and review face attributes

Time to see what a face-specific ML model can do with the same kind of photo that Label Detection could only call "Person".

In the Amazon Rekognition console, choose Face detection from the left navigation.
Choose Upload and select the clear, front-facing portrait or selfie you prepared earlier.

A bounding box appears around the detected face. The console displays a set of facial attributes in the Faces | Confidence pane on the right.

Review the emotion percentages listed for the face (such as Happy, Calm, or Surprised, each with its own confidence score).
Check the estimated age range shown for the detected face.
Look at the additional attributes like whether the person is wearing glasses and whether their eyes are open.

What does Face Analysis detect?

Unlike Label Detection, which identifies broad categories like "Person" or "Outdoor", Face Analysis zooms in on the human face itself. It returns emotions with individual confidence percentages, an estimated age range (for example, 25-33), and boolean attributes like glasses, eyes open, and smile.

Each attribute gets its own confidence score, just like labels did. A "Happy: 98.5%" result means the model is very confident the person appears happy based on their facial expression.

Compare faces in a group photo

A single clear portrait is the easiest input for the model. A group photo with multiple people at different distances and angles is a much harder challenge.

Choose Upload again and select the group photo with multiple people you prepared.

Rekognition detects each face separately. Each one gets its own bounding box and its own set of attributes.

Click on a face that is large and clearly visible in the foreground to view its attributes in the Faces | Confidence pane.
Click on a face that is small or partially obscured in the background and compare its attributes to the clear face.

The small or turned faces will have lower confidence scores or missing attributes compared to the large, clear faces.

Why do some faces have lower confidence?

Input quality directly affects output quality. A small face in the background gives the model fewer pixels to analyze, which means less data to work with.

This is a fundamental ML concept that applies everywhere. The better the input data, the more accurate the predictions.

Test how face orientation affects accuracy

You have seen how distance and size affect results. Now test what happens when the face itself is partially hidden.

Before you upload the next photo, predict: do you think the model will detect emotions just as accurately when the face is covered by sunglasses or turned to the side?

Choose Upload and select a photo of a person wearing sunglasses or showing a side profile.
Compare the results to the clear front-facing portrait you uploaded in the first substep.

Some attributes become less accurate or unavailable entirely. Emotion detection in particular loses accuracy when key facial features like eyes are hidden behind sunglasses.

Why does accuracy drop with sunglasses or side profiles?

ML models are trained on specific types of data. Face Analysis models are trained primarily on clear, front-facing photos where all facial features are visible.

When the input does not match what the model was trained on, accuracy drops. Sunglasses hide the eyes, and a side profile hides half the face. Less visible data means less confident predictions.

🙋‍♀️ Not seeing any face detected?

If the model did not detect a face at all, your photo may be too far from a standard face orientation. Try a photo where the face is at least partially visible from the front.

Also confirm your image is in JPEG or PNG format and under 5MB in size.

Why isn't Rekognition detecting any face in my uploaded photo?

You have now seen a core ML principle in action: the quality and orientation of the input directly determines the quality of the output. A clear, front-facing portrait produces rich, high-confidence attributes. A side profile or sunglasses-covered face produces weaker results.

Face Analysis tells you everything about individual faces, but it has its own limitation. It cannot tell you whether two different photos show the same person. In the next step, you will use Face Comparison to match identities across images.

Match Faces Across Photos

In the last step, you analyzed individual faces and discovered how Face Analysis detects emotions, age ranges, and facial landmarks.

But here is a real limitation you may have noticed. Face Analysis can tell you everything about a single face, but it cannot tell you whether two different photos show the same person. That is a completely different ML task. In this step, you will use Face Comparison to match identities across images using a similarity score.

In this step, get ready to:

Compare two photos of the same person and review a high similarity score.
Test with photos of different people and observe a dramatically lower score.
Understand how ML uses feature extraction to match faces across images.

Compare two photos of the same person

Face Comparison takes two images and calculates how similar the faces in them are. It returns a similarity score from 0 to 100, where a high score means the model is confident both photos show the same person.

In the Amazon Rekognition console, choose Face comparison from the left navigation.
The console loads with a sample image already analyzed. Look at the sample results in the Results pane. You will see pairs of faces with a similarity score displayed for each pair.

What is the Results pane showing?

The Results pane shows each face detected in the target image alongside the similarity score comparing it to the largest face in the source image. A high percentage means a strong match. A low percentage means the faces belong to different people.

Now upload your own images. For the source image (labeled Reference face on the left), choose Upload and select the first of your two prepared photos of the same person.
For the target image (labeled Comparison faces on the right), choose Upload and select the second photo of that same person. This should be taken at a different time or in a different setting.

Before you look at the results, take a guess. Do you think the similarity score will be above or below 90%?

Check the Results pane. You should see a similarity score above 90%, meaning the model is highly confident both photos show the same person.

What does the similarity score mean?

A similarity score of 90% or above means the model is very confident the two faces belong to the same person. The score is not a percentage of matching pixels. It reflects how closely the mathematical features of the two faces align.

Rekognition extracts features like the distance between the eyes, the shape of the jawline, and other facial geometry. It then compares those feature sets to produce the score.

🙋‍♀️ Seeing a low score for the same person?

Make sure both photos clearly show the person's face from the front. Photos taken from a steep angle, with heavy shadows, or with sunglasses can reduce the similarity score.

Also confirm your images are JPEG or PNG format and under 5MB in size.

Why is my similarity score low for two photos of the same person?

Test with a different person

A high similarity score for the same person is useful, but how does the model handle two completely different people? Testing the opposite case reveals exactly how Face Comparison distinguishes between faces.

Keep one of your current photos in place (either the left or right side).
On the other side, click Upload and select the photo of a different person you prepared in Step 1.
Wait for Rekognition to process both images.

Before you check the results, predict what you think will happen. Will the similarity score stay high, drop to around 50%, or fall close to 0%?

Check the Results pane. The similarity score should drop dramatically, often below 10%. That contrast is the whole point.

How does the model tell faces apart?

The model does not compare raw pixels between the two images. Instead, it converts each face into a set of numerical features. This process is called feature extraction.

Features include measurements like the distance between the eyes, the shape of the jawline, and the proportions of the nose and mouth. When two faces produce similar feature sets, the similarity score is high. When the features diverge, the score drops.

Connect face comparison to real-world applications

The feature extraction technique you just saw powers some of the most common technology in everyday life.

Phone unlock via Face ID uses the same principle. Your phone extracts facial features and compares them against a stored reference every time you pick it up.
Airport security systems match passengers to their passport photos by comparing facial features across two images, just like you did.
Photo library apps (like Apple Photos or Google Photos) automatically group photos of the same person by running feature extraction across every face they detect.

What is feature extraction?

Feature extraction is the core machine learning concept behind face comparison. The model converts visual information (a photo of a face) into structured numerical data (a set of measurements).

Comparing numbers is fast and reliable. Comparing raw images is not. This same principle applies to every ML service you have used in this project. Label detection extracts visual features to identify objects. Face analysis extracts facial features to predict attributes. Face comparison extracts features to match identities.

You have now used three different Rekognition features, and each one solved a limitation of the previous one. Label detection found objects but could not analyze faces. Face analysis read facial attributes but could not compare across photos. Face comparison matched identities using feature extraction.

Next up, you will take on a Secret Mission that pushes ML even further. Instead of analyzing what a photo looks like, you will teach ML to read text inside images.

Secret mission

Detect Text in Images

You have analyzed objects, faces, and identities. But ML can also read. In this secret mission, you will use Amazon Rekognition Text Detection to extract printed and handwritten text from images. You will discover a completely different data structure and explore real-world use cases like reading signs, scanning documents, and recognizing license plates.

Clean Up Your Resources

This project runs entirely in the Amazon Rekognition console using built-in demos. No AWS resources were created, so there are no ongoing costs or services to shut down.

Resources you used:

Amazon Rekognition console demos (Label Detection, Face Detection, Face Comparison, Text Detection).
Your AWS account (if you created one during Step 1).
4-5 personal images stored on your local computer.

Why is there nothing to clean up?

The Amazon Rekognition console demos process images on the fly. They do not create S3 buckets, databases, or any other persistent AWS resources. Your uploaded images are analyzed in real time and are not stored by the service after the demo session.

Keep everything running

No action needed. Choose this if you want to keep exploring Rekognition.

You can return to the Rekognition console anytime to analyze more images using the built-in demos.
All image analysis in this project is covered by the Amazon Rekognition free tier, which allows 1,000 images per month for 12 months from account creation.
If you created a new AWS account after July 15, 2025, your account also includes up to $200 in AWS credits that cover Rekognition usage.
Your personal images remain on your local computer. The Rekognition demos did not copy or store them.

Pause - I'll come back to this later

No action needed. Nothing is running in the background.

The Rekognition console demos process images on the fly and stop when you close the browser tab. There are no running services to pause.
You can sign out of the AWS Management Console by clicking your account name in the top-right corner and choosing Sign Out.
When you are ready to come back, sign into the AWS Management Console and search for Rekognition to pick up where you left off.

Delete - I don't want to use this again

No AWS resources were created during this project, so there is nothing to delete.

The Rekognition console demos do not store your uploaded images or analysis results. No S3 buckets, databases, or other AWS resources exist from this project.
If you prepared images specifically for this project and no longer need them, you can delete them from your computer.
Your AWS account remains active. If you do not plan to use AWS again, you can close your account by going to Account settings in the AWS Management Console, scrolling to Close Account, and following the prompts. Keep in mind that closing your account is permanent after a 90-day grace period.

Nice Work!

Nice work! You have just analyzed your own photos with machine learning using Amazon Rekognition, and you understand how ML models return structured predictions with confidence scores.

You've learned how to:

Detected objects, scenes, and concepts in images using Label Detection, and learned how ML models assign confidence scores to every prediction.
Analyzed facial attributes including emotions, estimated age range, and facial landmarks using Face Analysis, and discovered how input quality directly impacts ML accuracy.
Compared faces across different photos using Face Comparison with similarity scores, and built a mental model of task-specific ML where different models are trained for different jobs and return different structured data.
Secret Mission: Extracted printed text from images using Text Detection, expanding your ML toolkit beyond visual recognition to text recognition.

Ready to quiz yourself?