Image Label Generator with Rekognition

Analyze images with AWS Rekognition and visualize detected objects.

Introduction

30 Second Summary

Every photo you take contains information. Your phone already uses image recognition to search photos by keyword, identify plants, and tag faces. Behind the scenes, cloud AI services make this possible at massive scale.

In this project, you will build a Python script that sends any photo to AWS Rekognition and gets back a detailed breakdown of everything the AI detects. You will also draw colored bounding boxes on the image showing exactly where each object was found.

What You'll Build

You will run your script against a photo, watch a table of detected objects and confidence scores appear in your terminal, then see the annotated image pop open with green rectangles around every recognized object.

By the end of this project, you'll have:

  • A reusable Python script that uploads any JPEG or PNG to Amazon S3 and returns a formatted table of detected objects, scenes, and concepts with confidence percentages.
  • A visual output where bounding boxes are drawn directly on your image, showing exactly where each detected object sits in the photo.
  • Control over detection sensitivity through confidence filtering, removing low-quality guesses so only meaningful detections remain.
  • Secret Mission: Add dominant color detection to build a color palette alongside your labels, revealing both what is in the image and what colors define it.

Are there any prerequisites?

You need an existing AWS account and the AWS CLI already installed and configured on your Mac. If you have not set these up yet, complete those steps before starting this project.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters to you.

Set Up Python and AWS Permissions

Your image analysis tool needs three pieces working together: Python to run your script, the AWS SDK to communicate with Rekognition, and IAM permissions that grant your machine access to cloud AI services.

In this step, you'll confirm each piece is connected and ready before you start writing code.

In this step, get ready to:
  • Confirm Python 3.14.6 is running correctly.
  • Verify the boto3 and Pillow libraries are importable.
  • Confirm your AWS CLI credentials have Rekognition and S3 permissions.
Confirm your Python version

Python is the language you'll use to write your image analysis script. The libraries this project uses require Python 3.10 or higher, and you have 3.14.6 installed.

  • Confirm your Python version by running this command:
python3 --version

You should see:

Python 3.14.6

Seeing a different version or command not found?

If you see a version lower than 3.10, download and install Python 3.14.6 from python.org/downloads using the macOS installer. After installation, run the Install Certificates command found in /Applications/Python 3.14/.

If you see command not found, Python is not installed. Download the macOS installer from the link above and run it.

help me install Python 3.14.6 on macOS

Verify your Python libraries

Your script uses two libraries: boto3 (the AWS SDK for Python) to call Rekognition, and Pillow (an image processing library) to draw bounding boxes on your photos.

  • Verify both libraries are importable by running this command:
python3 -c "import boto3; import PIL; print('Ready')"

You should see Ready printed in your terminal.

What are boto3 and Pillow?

boto3 is the official AWS SDK for Python. It lets your script communicate with any AWS service, including Rekognition and S3.

Pillow is a Python imaging library that can open, manipulate, and save image files. You'll use it to draw rectangles around detected objects directly on your photo.

Seeing ModuleNotFoundError?

If the import fails, install both libraries by running:

pip3 install boto3 pillow

Then re-run the import command above to confirm they're importable.

help me fix a ModuleNotFoundError for boto3 or Pillow on macOS

Confirm your AWS credentials and permissions

Your IAM user needs two managed policies attached: AmazonRekognitionFullAccess (to call the DetectLabels API) and AmazonS3FullAccess (to create a bucket and upload images). The quickest way to confirm everything is wired up is to ask AWS who you are.

  • Verify your CLI credentials by running this command:
aws sts get-caller-identity

You should see a JSON response containing your Account ID (a 12-digit number) and your IAM user's Arn.

What does this command prove?

aws sts get-caller-identity asks AWS to identify the credentials your CLI is using. If it returns your account ID and user ARN, your CLI is properly configured to make authenticated calls to AWS services.

The actual permissions (Rekognition and S3 access) are granted by the managed policies attached to your IAM user. You'll confirm those work in the next step when you create a bucket and upload an image.

Seeing 'Unable to locate credentials'?

This means your AWS CLI is not configured with access keys. Run aws configure and enter your Access Key ID, Secret Access Key, and set the default region to us-east-1.

If you see a response but are unsure about your policies, open the IAM console, find your user, and confirm AmazonRekognitionFullAccess and AmazonS3FullAccess appear under Permissions policies.

help me configure my AWS CLI credentials and attach IAM policies

Before you move on, predict what you'll see when you run the final check below. Will it print Ready, or will something fail?

  • Run the full verification one more time to confirm everything is connected:
python3 -c "import boto3; import PIL; print('Ready')"

You see Ready printed. Python, boto3, and Pillow are all working together.

Your environment is ready. Next up, you'll create an S3 bucket and upload an image for Rekognition to analyze.

Create an S3 Bucket and Upload an Image

Your development environment is ready with Python, boto3, and the right IAM permissions. The next step is giving Rekognition an image to analyze.

Rekognition reads images directly from S3 buckets. It does not accept file uploads from your local machine. You need to create a bucket and store your image there before analysis can begin.

In this step, get ready to:
  • Create an S3 bucket in the us-east-1 region.
  • Upload a photo with multiple objects to the bucket.
Create an S3 bucket in us-east-1

S3 bucket names must be globally unique across all AWS accounts. Choose a name that includes your own name or initials so it does not collide with anyone else's bucket.

  • Create your S3 bucket by running this command:
aws s3 mb s3://[[BUCKET_NAME="your-rekognition-bucket-name"]] --region us-east-1

What does this command do?

  • aws s3 mb stands for "make bucket" and creates a new S3 bucket.
  • --region us-east-1 places the bucket in the US East (N. Virginia) region, which is where your Rekognition calls will run.
  • Confirm the bucket was created by listing your S3 buckets:
aws s3 ls

You should see your new bucket name in the list with today's date.

Seeing a BucketAlreadyExists error?

This means someone else on the internet already owns that bucket name. Try a more unique name by adding your initials, a date, or extra words.

If you see an access denied error, confirm that your IAM user has the AmazonS3FullAccess policy attached from the previous step.

help me fix my S3 bucket creation error

Why upload to S3 instead of sending the file directly?

Rekognition supports two ways to pass an image: as raw bytes (limited to 5 MB) or as an S3 reference (up to 15 MB). Using S3 is the standard workflow because it handles larger images and keeps your analysis separate from file transfer.

Upload an image to your bucket

Rekognition works best with photos that contain multiple recognizable objects. A street scene, a park, or your desk are all great choices.

  • Find or take a photo (JPEG or PNG) with several visible objects.
  • Move the image file to your Desktop for easy access.
  • Navigate to your Desktop in the terminal by running this command:
cd ~/Desktop
  • Upload your image to S3 by running this command:
aws s3 cp [[IMAGE_FILE="your-image.jpg"]] s3://[[BUCKET_NAME="your-rekognition-bucket-name"]]/

What does this command do?

aws s3 cp copies a file from your local machine to S3. Your local copy of the image stays in place on your Desktop. You will need this local copy later for drawing bounding boxes on the image.

Before you run the next command, what do you think the output will show?

  • Verify your image is in the bucket by running:
aws s3 ls s3://[[BUCKET_NAME="your-rekognition-bucket-name"]]/

You should see a line with today's date, the file size in bytes, and your image filename listed.

Don't see your image listed?

Double check that the bucket name in your ls command matches the name you used when creating the bucket.

If you see "No such file or directory" for the image, confirm you are in the same directory as your image file by running ls to list local files.

help me troubleshoot my S3 upload

Your image is stored in S3 and ready for analysis. Next up, you will write a Python script that sends this image to Rekognition and gets back a list of everything the AI detects in it.

Detect Labels with a Python Script

Your image is sitting in S3, ready for analysis. Now you can write a Python script that sends that image to AWS Rekognition and gets back a list of everything the AI detects in it.

Rekognition's DetectLabels API reads your image directly from S3, analyzes it, and returns a set of labels describing what it found. Each label comes with a confidence score showing how certain the AI is about that detection.

In this step, get ready to:
  • Write a Python function that calls the Rekognition DetectLabels API.
  • Run the script and see detected objects with confidence scores printed in your terminal.
Create the detect_labels function

The core of your script is a function that connects to the Rekognition service and asks it to analyze a specific image in your S3 bucket.

  • Create a new file called detect_labels.py in your working directory by running:
touch detect_labels.py
  • Confirm the file exists by running:
ls

You should see detect_labels.py listed alongside your your-image.jpg file.

  • Open detect_labels.py in your text editor and add the imports and detection function by pasting this code:
import boto3
from PIL import Image, ImageDraw


def detect_labels(photo, bucket, region):
    # Create a Rekognition client for the specified region
    client = boto3.client('rekognition', region_name=region)

    # Call DetectLabels with the S3 image location
    response = client.detect_labels(
        Image={'S3Object': {'Bucket': bucket, 'Name': photo}},
        MaxLabels=15,
        MinConfidence=70,
        Features=["GENERAL_LABELS"]
    )

    return response['Labels']

What does this code do?

This function does three things:

  • boto3.client('rekognition') creates a connection to the Rekognition service in your chosen region.
  • Image={'S3Object': {...}} tells Rekognition to read the image directly from your S3 bucket instead of uploading raw bytes.
  • MaxLabels=15 caps results at 15 labels, and MinConfidence=70 filters out anything below 70% confidence so you only get meaningful detections.
  • Features=["GENERAL_LABELS"] requests standard object and scene detection from the API.
Add the main function and entry point

The detection function is ready, but it needs a caller that passes in your specific bucket name and image filename.

  • Add the following code below your detect_labels function in detect_labels.py:
def main():
    # Set your S3 bucket name and image filename
    bucket = '[[BUCKET_NAME="your-rekognition-bucket-name"]]'
    photo = '[[IMAGE_FILE="your-image.jpg"]]'
    region = 'us-east-1'

    print(f"Detecting labels in: {photo}")
    print("-" * 50)

    # Call the detection function
    labels = detect_labels(photo, bucket, region)

    # Print each detected label with its confidence score
    for label in labels:
        print(f"{label['Name']}: {label['Confidence']:.1f}%")


if __name__ == "__main__":
    main()

What does this code do?

The main() function sets up the configuration (your bucket name, image filename, and region) then calls your detect_labels function.

The for loop prints each label's Name and Confidence score, formatted to one decimal place. The if __name__ == "__main__" block ensures the script runs main() when you execute it from the terminal.

  • Save detect_labels.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved detect_labels.py with both the detect_labels function and the main function.

ⓧ I'd like to double check the full code

Here is the complete detect_labels.py file. Make sure yours matches:

import boto3
from PIL import Image, ImageDraw


def detect_labels(photo, bucket, region):
    client = boto3.client('rekognition', region_name=region)

    response = client.detect_labels(
        Image={'S3Object': {'Bucket': bucket, 'Name': photo}},
        MaxLabels=15,
        MinConfidence=70,
        Features=["GENERAL_LABELS"]
    )

    return response['Labels']


def main():
    bucket = '[[BUCKET_NAME="your-rekognition-bucket-name"]]'
    photo = '[[IMAGE_FILE="your-image.jpg"]]'
    region = 'us-east-1'

    print(f"Detecting labels in: {photo}")
    print("-" * 50)

    labels = detect_labels(photo, bucket, region)

    for label in labels:
        print(f"{label['Name']}: {label['Confidence']:.1f}%")


if __name__ == "__main__":
    main()
Run the script and observe the output

Your script is complete. Time to see what Rekognition can detect in your image.

  • Before you run this, predict what kinds of labels you think Rekognition will return for your image. Will it detect specific objects, general scenes, or both?
  • Run the script by executing:
python3 detect_labels.py

You should see a list of detected labels with confidence scores printed in your terminal, something like:

Detecting labels in: your-image.jpg
--------------------------------------------------
Car: 99.2%
Person: 97.8%
Traffic Light: 94.1%
Road: 91.5%
Building: 88.3%

The exact labels depend on your image. You now know WHAT is in the image, but not WHERE each object is located. The output is also a flat list with no structure or spatial information.

Seeing an error instead of labels?

  • If you see NoCredentialsError, your AWS CLI credentials are not configured. Run aws sts get-caller-identity to check.
  • If you see NoSuchBucket, double check the bucket name in your script matches the one you created.
  • If you see InvalidS3ObjectException, the image filename in your script does not match the key you uploaded to S3. Run aws s3 ls s3://your-bucket-name/ to see what is actually in the bucket.

Help me debug my Rekognition DetectLabels script

Your script successfully identifies what is in your image. Next up, you will format the output into a proper table and add bounding box visualization so you can see exactly WHERE each object was detected.

Filter Results and Visualize Bounding Boxes

Your script can now call Rekognition and print detected labels with confidence scores.

The output tells you WHAT is in the image, but not WHERE each object sits in the frame. In this step, you will format the results into a clean table and draw bounding boxes directly on the image so you can see the exact location of every detection.

In this step, get ready to:
  • Format the label output as an aligned table with columns for name, confidence, and instance count.
  • Add a visualization function that draws bounding boxes on the original image.
  • Run the script and confirm the annotated image opens in Preview.
Format the label output as a table

The current script prints each label on its own line, but there is no structure showing how many spatial instances each label has. A formatted table makes it easy to scan results at a glance.

  • In detect_labels.py, find the main() function.
  • Locate these lines inside main():
    for label in labels:
        print(f"{label['Name']}: {label['Confidence']:.1f}%")
  • Replace those two lines with the following table-formatted output:
    # Print a formatted table header
    print(f"{'Label':<25} {'Confidence':<12} {'Instances'}")
    print("-" * 50)

    # Print each label with its confidence and instance count
    for label in labels:
        instance_count = len(label.get('Instances', []))
        print(f"{label['Name']:<25} {label['Confidence']:<12.1f} {instance_count}")

    print("-" * 50)
    print(f"Total labels detected: {len(labels)}")

What does this code do?

The :<25 and :<12 syntax pads each column to a fixed width so values line up vertically.

The instance_count tells you how many spatial bounding boxes exist for each label. Labels like "Person" may have multiple instances (one box per person), while scene-level labels like "Nature" have zero instances because they describe the whole image rather than a specific object.

  • Save detect_labels.py.
  • Run the script to confirm the table prints correctly:
python3 detect_labels.py

You should see columns of labels neatly aligned with their confidence percentages and instance counts.

Seeing a syntax error?

Make sure the f-string curly braces match. Each {label['Name']:<25} needs single quotes inside the dictionary key and double quotes wrapping the whole f-string.

Check that you removed the old for label in labels: loop completely before pasting the new one.

Help me debug a syntax error in my Python f-string formatting.

Add the bounding box visualization function

Rekognition returns normalized bounding box coordinates (values between 0 and 1) for object-level labels. You need to convert these to pixel positions and draw rectangles on the image using Pillow.

  • In detect_labels.py, add the following function between the detect_labels function and the main function:
def draw_bounding_boxes(photo_path, labels, output_path):
    image = Image.open(photo_path)
    draw = ImageDraw.Draw(image)
    img_width, img_height = image.size

    for label in labels:
        for instance in label.get('Instances', []):
            box = instance['BoundingBox']
            # Convert normalized coordinates to pixel positions
            left = int(box['Left'] * img_width)
            top = int(box['Top'] * img_height)
            width = int(box['Width'] * img_width)
            height = int(box['Height'] * img_height)

            # Draw a lime-green rectangle around the object
            draw.rectangle(
                [left, top, left + width, top + height],
                outline='lime',
                width=3
            )
            # Label the box with the detection name and confidence
            draw.text(
                (left, top - 15),
                f"{label['Name']} ({instance['Confidence']:.1f}%)",
                fill='lime'
            )

    image.save(output_path)
    print(f"Annotated image saved to: {output_path}")
    image.show()

What does this code do?

Rekognition returns bounding box values as ratios (e.g., Left: 0.35 means 35% from the left edge). Multiplying by the image's pixel dimensions converts these to actual pixel coordinates.

The label.get('Instances', []) safely handles labels that have no bounding boxes. Only object-level labels (like "Person" or "Car") include spatial instances. Scene-level labels (like "Nature" or "Outdoors") describe the whole image and have no boxes to draw.

Finally, image.show() opens the annotated image in Preview on macOS.

  • Save detect_labels.py.
Wire up the visualization and run the full script

The function exists but nothing calls it yet. You need to add variables for the local file path and output path, then call draw_bounding_boxes at the end of main().

  • In the main() function, find the line that says region = 'us-east-1'.
  • Add these two variables directly below it:
    local_photo = 'your-image.jpg'
    output_path = 'output-labeled.jpg'
  • Now scroll to the very end of main(), after the print(f"Total labels detected: {len(labels)}") line.
  • Add these lines to call the visualization function:
    print()

    draw_bounding_boxes(local_photo, labels, output_path)

What does this code do?

The local_photo variable points to the image file on your machine (the same one you uploaded to S3 earlier). The function reads this local copy, draws boxes on it, and saves the result as output-labeled.jpg.

The print() adds a blank line between the table and the file-saved message for readability.

  • Save detect_labels.py.

Before you run this, what do you expect to see in the terminal AND on your screen? Think about which labels had instance counts greater than zero in your last run.

  • Run the full script:
python3 detect_labels.py

You should see the formatted label table in your terminal, followed by a message saying the annotated image was saved. Preview opens automatically showing your original photo with lime-green bounding boxes drawn around detected objects.

Why don't all labels have boxes?

If some labels in your table show 0 instances, that is expected. Scene-level labels like "Nature" or "Urban" describe the overall image content. They have no spatial location, so no box is drawn for them. Only object-level labels (people, vehicles, animals) include bounding box data.

Image not opening or no boxes visible?

If Preview does not open, make sure local_photo = 'your-image.jpg' matches the actual filename of the image in your working directory. Check with ls to confirm the file is there.

If the image opens but has no green boxes, check your label table. If every label shows 0 instances, your image may only contain scene-level detections. Try a photo with distinct objects like people, cars, or animals.

Help me figure out why my bounding boxes are not appearing on the annotated image.

✔️ Awesome, I've got everything!

Great. Double check you have saved detect_labels.py and that output-labeled.jpg exists in your working directory.

ⓧ I'd like to double check the full code

Your detect_labels.py should contain these sections in order. Scroll up and confirm each one is present:

  • Imports: boto3, Image, and ImageDraw from Pillow.
  • The detect_labels function that calls Rekognition with MaxLabels=15 and MinConfidence=70.
  • The draw_bounding_boxes function that opens the image, draws lime rectangles and label text, saves, and shows.
  • The main() function with bucket, photo, region, local_photo, and output_path variables, the formatted table loop, and the call to draw_bounding_boxes.
  • The if __name__ == "__main__": block at the bottom.

You now have a complete image analysis pipeline. Your script sends a photo to AWS Rekognition, receives structured detections, and renders those detections visually on the original image. Next up, you will take on the Secret Mission and extend this tool to detect dominant colors.

Secret mission

Detect Dominant Colors with IMAGE_PROPERTIES

Your label detector knows WHAT is in the image, but what about the visual feel? In this secret mission, you'll tap into Rekognition's IMAGE_PROPERTIES feature to extract a dominant color palette from your photo. The result is a complete analysis showing both detected objects and the colors that define the image.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses an S3 bucket for image storage, which costs fractions of a cent per GB per month under the free tier.

Resources you used:

  • S3 bucket (stores your uploaded images in us-east-1).
  • IAM policies (AmazonRekognitionFullAccess and AmazonS3FullAccess attached to your user).
  • Local project files (detect_labels.py, your-image.jpg, output-labeled.jpg).

Keep everything running

No action needed. Choose this if you're still actively building or want to experiment with more images.

  • Your S3 bucket stays available for future image analysis experiments. Storage cost is negligible for a few images.
  • Your IAM policies remain attached, so you can run the script anytime.
  • Your local files stay in your working directory, ready to use.

Pause - I'll come back to this later

There are no running services to pause in this project. S3 storage is passive, so there is nothing to shut down.

  • Your bucket and files remain available whenever you want to return.
  • No ongoing compute costs are being incurred.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow these steps to delete everything.

  • Delete your S3 bucket and all its contents by running:
aws s3 rb s3://your-rekognition-bucket-name --force
  • Detach the AmazonRekognitionFullAccess policy from your IAM user by running:
aws iam detach-user-policy --user-name [[IAM_USERNAME="your IAM username"]] --policy-arn arn:aws:iam::aws:policy/AmazonRekognitionFullAccess
  • Detach the AmazonS3FullAccess policy from your IAM user by running:
aws iam detach-user-policy --user-name [[IAM_USERNAME="your IAM username"]] --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess
  • Delete your local project files by running:
rm detect_labels.py your-image.jpg output-labeled.jpg

That's a Wrap!

That's a Wrap!

Nice work! You just built a complete image analysis pipeline that sends any photo to AWS Rekognition and gets back both a text report and a visually annotated image with bounding boxes around detected objects.

You've learned how to:

  • Configured AWS IAM permissions and created an S3 bucket to store images for cloud AI analysis.
  • Built a Python script that calls Rekognition's DetectLabels API and prints detected objects with confidence scores.
  • Added confidence filtering and bounding box visualization using Pillow to annotate the original image with labeled rectangles around detected objects.
  • Secret Mission: Extended the script to detect dominant colors using IMAGE_PROPERTIES for a complete visual analysis.

Ready to quiz yourself?