Build an AI DevOps Copilot

Connect Amazon Bedrock AI to your infrastructure for natural language incident analysis.

Introduction

30 Second Summary

When something breaks in production at 2 AM, the last thing you want to do is dig through raw log files and alarm histories to figure out what happened. What if you could just ask your infrastructure "What went wrong?" and get a clear answer in plain English?

In this project, you will build an AI-powered DevOps Copilot that connects to your self-healing infrastructure from Part 1. You'll create a serverless API that collects real operational data from CloudWatch, analyzes it with Amazon Bedrock's Nova Micro model, and returns human-readable summaries in response to natural language questions.

What You'll Build

A serverless REST API backed by Lambda and Amazon Bedrock that collects live infrastructure data and uses AI to answer natural language questions about your system's health and incidents.

By the end of this project, you'll have:

A Lambda function that collects CloudWatch alarm history, remediation logs, and EC2 instance status using Boto3.
An Amazon Bedrock integration using the Converse API with Nova Micro to analyze and summarize infrastructure data.
A REST API built with API Gateway that accepts natural language questions and returns AI-generated answers.
A chat dashboard (Secret Mission) with a dark ops-console theme for a visual DevOps Copilot experience.

Are there any prerequisites?

This project is Part 2 of a series. You need the infrastructure from Part 1 (self-healing EC2/nginx setup with CloudWatch alarms, EventBridge rules, Lambda remediation function, SNS notifications, and Systems Manager) still deployed and running in your AWS account.

Before We Start

Before We Start...

Enable Amazon Bedrock Model Access

Your self-healing infrastructure from Part 1 can detect problems and fix them automatically. But what if you could ask your infrastructure what happened in plain English?

That is exactly what you will build in this project. You need access to a foundation model to power the AI brain behind your copilot.

In this step, you will enable Amazon Nova Micro in Amazon Bedrock and verify it works by running a test prompt.

In this step, get ready to:

Confirm Amazon Nova Micro model access is enabled in Amazon Bedrock.
Test the model in the Bedrock playground to verify it responds to prompts.

Navigate to Amazon Bedrock and confirm model access

Amazon Bedrock is a fully managed service that gives you access to foundation models from Amazon and other providers through a single API. Nova Micro is Amazon's fastest and cheapest text-only model, which makes it perfect for analyzing log data without racking up costs.

Open the AWS Management Console at https://console.aws.amazon.com.
Check that your region is set to US East (N. Virginia) (us-east-1) in the top-right corner of the console.
Click the region name to change it if it shows a different region.
Type Bedrock in the search bar at the top of the console.
Click Amazon Bedrock from the search results.

What is Amazon Bedrock?

Bedrock is AWS's managed service for accessing foundation models. Instead of hosting and managing your own AI model, you send API requests to Bedrock and pay only for the tokens you use.

Nova Micro costs $0.035 per 1,000 input tokens and $0.14 per 1,000 output tokens. For this project, that works out to fractions of a penny per query.

Amazon Nova models are first-party Amazon models, so access is typically enabled automatically in your AWS account. Let's confirm this is the case.

In the left navigation pane, expand Bedrock configurations.
Click Model access.
Look for Nova Micro in the model list. It should show an Access granted status.

I see Access granted

You are all set. Nova Micro is ready to use in your account.

I don't see Access granted

If Nova Micro does not show Access granted, you need to enable it manually.

Click Enable specific models (or Modify model access depending on your console version).
Select Nova Micro from the list of base models.
Click Next.
On the Review and submit page, click Submit.
Refresh the page after a moment. You should see Nova Micro with an Access granted status.

How long does activation take?

For Amazon-native models like Nova Micro, access is usually granted instantly. In rare cases it can take up to a few minutes. Refresh the page if the status has not updated.

Why Nova Micro?

Nova Micro is Amazon's fastest text-only model, optimized for speed and low cost. It has a 128K token context window, which is more than enough to hold the alarm history and log data your copilot will collect.

Since you only need text summarization (not image or video analysis), Nova Micro is the ideal choice over larger, more expensive models.

Test Nova Micro in the Bedrock playground

Before writing any code, let's verify that Nova Micro actually responds to prompts. The Bedrock playground is a built-in tool in the console where you can send prompts to any enabled model and see the response.

In the left navigation pane, expand Test and click Playground.
Select the Single prompt mode at the top of the playground.
Click Select model.
Under Provider, select Amazon.
Under Model, select Nova Micro.
Click Apply.
In the prompt text box, type: What are three things a DevOps engineer monitors in a production environment?.
Click Run.

After a few seconds, Nova Micro returns a response below your prompt. You should see a clear, structured answer about monitoring topics like uptime, logs, and resource usage.

What should I see?

Nova Micro generates a text response listing common DevOps monitoring practices. The exact wording varies each time, but you should see a coherent answer with three distinct points.

You can also check the Model metrics section below the response to see token counts and cost for the request.

Not seeing a response?

Double-check that your region is set to us-east-1 in the top-right corner of the console. Nova Micro must be accessed from the same region where you enabled it.
Make sure you selected Amazon as the provider and Nova Micro as the model, then clicked Apply.
If you see an access denied error, go back to Model access under Bedrock configurations and confirm Nova Micro shows Access granted.
Help me troubleshoot my Bedrock playground.

Nova Micro is confirmed and ready to go. Next up, you will build the Lambda function that collects real infrastructure data from your Part 1 setup and sends it to this model for AI-powered analysis.

Build the Log Collector Lambda Function

In Step 1, you enabled Amazon Bedrock and confirmed that Nova Micro is ready to analyze your data. Now you need the raw data itself.

In this step, you'll create a Lambda function that acts as your log collector. It reaches into three AWS services from your Part 1 infrastructure and pulls out the operational data your AI copilot needs.

In this step, get ready to:

Create the DevOpsCopilot Lambda function with the right permissions.
Write three log collector functions that scrape CloudWatch alarms, remediation logs, and EC2 instance status.
Test the log collector by invoking the Lambda and verifying it returns real data from your infrastructure.

Create the DevOpsCopilot Lambda function

Your copilot needs a home to run in. Lambda lets you run Python code without managing servers, and it only costs you when it actually executes.

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
Type your browser name and press Enter to open it.
Navigate to the AWS Management Console and search for Lambda in the top search bar.
Click Lambda to open the Lambda console.
Click Create function.
Select Author from scratch.
Enter DevOpsCopilot in the Function name field.
Select Python 3.12 from the Runtime dropdown.
Under Change default execution role, select Create a new role with basic Lambda permissions.
Click Create function.

You should see a green Successfully created the function DevOpsCopilot banner at the top of the page.

Why Python 3.12?

The Lambda Python 3.12 runtime includes Boto3 out of the box. Boto3 is the AWS SDK for Python, and the bundled version supports the Bedrock Converse API. No extra dependencies needed.

The default Lambda timeout is 3 seconds. Your function will call multiple AWS APIs, and later it will call Bedrock, which can take several seconds to respond. You need to increase the timeout now so your function does not get cut off mid-execution.

Click the Configuration tab in the function page.
Click General configuration in the left sidebar.
Click Edit.
Change the Timeout to 0 minutes and 30 seconds.
Click Save.

Next, your Lambda function needs permission to read from CloudWatch, CloudWatch Logs, EC2, and Bedrock. The default role only has basic logging permissions, so you need to add an inline policy to the function's IAM role.

Stay on the Configuration tab and click Permissions in the left sidebar.
Under Execution role, click the role name link. It opens the IAM console in a new tab.
Click Add permissions, then click Create inline policy.
Click the JSON tab in the policy editor.
Replace the existing content with this policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "cloudwatch:DescribeAlarmHistory",
                "logs:FilterLogEvents",
                "ec2:DescribeInstanceStatus",
                "bedrock:InvokeModel"
            ],
            "Resource": "*"
        }
    ]
}

What does this policy do?

cloudwatch:DescribeAlarmHistory lets the function read alarm state changes from your Part 1 alarm.
logs:FilterLogEvents lets the function search through your remediation Lambda's log entries.
ec2:DescribeInstanceStatus lets the function check the health of your EC2 instances.
bedrock:InvokeModel lets the function call the Nova Micro model. You'll use this in the next step.

Click Next.
Enter DevOpsCopilotPermissions as the Policy name.
Click Create policy.

You should see DevOpsCopilotPermissions listed alongside the AWSLambdaBasicExecutionRole policy in the role's permissions table.

Don't see the policy listed?

Make sure you clicked Create policy at the bottom of the page. If you navigated away before clicking it, the policy was not saved.
Refresh the IAM role page. Sometimes the new policy takes a moment to appear.

Still stuck? Help me troubleshoot my Lambda IAM role permissions.

Close the IAM tab and go back to your Lambda function tab.

Write the log collector functions

Your Lambda function needs to collect data from three sources: CloudWatch alarm history, remediation Lambda logs, and EC2 instance health. Each source tells the AI a different part of the story about what happened in your infrastructure.

In the Lambda function page, click the Code tab.
In the code editor, select all the existing code in lambda_function.py and delete it.
Add the following imports and client setup code:

import json
import boto3
import os
from datetime import datetime, timedelta, timezone

# Initialize AWS clients
cloudwatch = boto3.client('cloudwatch', region_name='us-east-1')
logs = boto3.client('logs', region_name='us-east-1')
ec2 = boto3.client('ec2', region_name='us-east-1')

# Configuration - update these to match your Part 1 resource names
LOG_GROUP_NAME = '/aws/lambda/NginxRemediationFunction'
ALARM_NAME = 'NginxHealthCheckAlarm'

What does this code do?

The boto3.client() calls create connections to three AWS services: CloudWatch (for alarms), CloudWatch Logs (for Lambda logs), and EC2 (for instance health).
LOG_GROUP_NAME points to the log group where your Part 1 remediation Lambda writes its output.
ALARM_NAME is the CloudWatch alarm from Part 1 that monitors your nginx health check.

Update these values to match your Part 1 setup

The LOG_GROUP_NAME and ALARM_NAME values must match the exact names from your Part 1 project. If your remediation Lambda was called something different, update LOG_GROUP_NAME to /aws/lambda/YourFunctionName. Check your Part 1 Lambda console if you are unsure of the name.

You will not see a change yet. The next code block adds the functions that use these clients.

Now you need the three collector functions. Each one queries a different AWS service and returns the results in a structured format that the AI can read later.

Below the configuration section you just wrote, add these three collector functions:

def collect_alarm_history():
    """Scrape CloudWatch alarm history for recent state changes."""
    try:
        now = datetime.now(timezone.utc)
        start_time = now - timedelta(hours=24)

        response = cloudwatch.describe_alarm_history(
            AlarmName=ALARM_NAME,
            HistoryItemType='StateUpdate',
            StartDate=start_time,
            EndDate=now,
            MaxRecords=10,
            ScanBy='TimestampDescending'
        )

        history_items = []
        for item in response.get('AlarmHistoryItems', []):
            history_items.append({
                'alarm': item.get('AlarmName', 'Unknown'),
                'timestamp': item.get('Timestamp', '').isoformat() if hasattr(item.get('Timestamp', ''), 'isoformat') else str(item.get('Timestamp', '')),
                'summary': item.get('HistorySummary', ''),
                'type': item.get('HistoryItemType', '')
            })

        return history_items
    except Exception as e:
        return [{'error': f'Failed to collect alarm history: {str(e)}'}]


def collect_remediation_logs():
    """Scrape CloudWatch Logs from the remediation Lambda function."""
    try:
        now = int(datetime.now(timezone.utc).timestamp() * 1000)
        one_day_ago = now - (24 * 60 * 60 * 1000)

        response = logs.filter_log_events(
            logGroupName=LOG_GROUP_NAME,
            startTime=one_day_ago,
            endTime=now,
            limit=20,
            filterPattern='ERROR OR WARN OR restart OR remediat OR nginx OR ssm'
        )

        log_entries = []
        for event in response.get('events', []):
            log_entries.append({
                'timestamp': datetime.fromtimestamp(
                    event.get('timestamp', 0) / 1000, tz=timezone.utc
                ).isoformat(),
                'message': event.get('message', '').strip()
            })

        return log_entries
    except Exception as e:
        return [{'error': f'Failed to collect remediation logs: {str(e)}'}]


def collect_instance_status():
    """Check EC2 instance health status."""
    try:
        response = ec2.describe_instance_status(
            IncludeAllInstances=True
        )

        instances = []
        for status in response.get('InstanceStatuses', []):
            instances.append({
                'instance_id': status.get('InstanceId', 'Unknown'),
                'state': status.get('InstanceState', {}).get('Name', 'Unknown'),
                'system_status': status.get('SystemStatus', {}).get('Status', 'Unknown'),
                'instance_status': status.get('InstanceStatus', {}).get('Status', 'Unknown'),
                'availability_zone': status.get('AvailabilityZone', 'Unknown')
            })

        return instances
    except Exception as e:
        return [{'error': f'Failed to collect instance status: {str(e)}'}]

What do these three functions do?

collect_alarm_history() calls describe_alarm_history() to pull the last 24 hours of alarm state changes. Each entry tells you when your nginx health check alarm went into ALARM or returned to OK.
collect_remediation_logs() calls filter_log_events() with a filter pattern that narrows results to lines containing keywords like ERROR, restart, or nginx. This keeps the data focused on what matters.
collect_instance_status() calls describe_instance_status() to get the current health of every EC2 instance in your account. The IncludeAllInstances=True flag makes sure stopped instances show up too.
Each function wraps its logic in a try/except block. If one data source fails, the others still return data instead of crashing the entire function.

You will not see a change yet. The next code block adds a handler so you can test these functions.

Your collector functions need a handler to tie them together. Lambda calls the handler function every time it runs. For now, you will write a temporary handler that just calls the three collectors and returns their raw data so you can verify everything works.

Below the three collector functions, add this temporary handler:

def lambda_handler(event, context):
    """Temporary handler to test the log collector functions."""
    # Collect infrastructure data from all three sources
    alarm_history = collect_alarm_history()
    remediation_logs = collect_remediation_logs()
    instance_status = collect_instance_status()

    # Return the raw collected data for testing
    return {
        'statusCode': 200,
        'headers': {
            'Content-Type': 'application/json'
        },
        'body': json.dumps({
            'data_collected': {
                'alarm_events': len(alarm_history),
                'log_entries': len(remediation_logs),
                'instances_checked': len(instance_status)
            },
            'alarm_history': alarm_history,
            'remediation_logs': remediation_logs,
            'instance_status': instance_status
        })
    }

What does this handler do?

This handler calls all three collector functions, counts how many events each one found, and returns everything as JSON. You will replace this handler in the next step when you add the Bedrock AI analysis layer.

Click Deploy above the code editor to save and deploy your function.

You should see a green Changes deployed banner at the top of the code editor.

Seeing a syntax error?

Check that your function names are spelled exactly as shown: collect_alarm_history, collect_remediation_logs, collect_instance_status.
Make sure each function definition starts at the leftmost column with no extra indentation.
Verify all parentheses, brackets, and braces are properly closed. Missing a single ) or } causes a syntax error.

Help me debug a syntax error in my Lambda function code.

Awesome, I've got everything!

Make sure you have clicked Deploy and see the green banner before moving on.

I'd like to double check the full code

Here is the complete code you should have in lambda_function.py at this point.

import json
import boto3
import os
from datetime import datetime, timedelta, timezone

# Initialize AWS clients
cloudwatch = boto3.client('cloudwatch', region_name='us-east-1')
logs = boto3.client('logs', region_name='us-east-1')
ec2 = boto3.client('ec2', region_name='us-east-1')

# Configuration - update these to match your Part 1 resource names
LOG_GROUP_NAME = '/aws/lambda/NginxRemediationFunction'
ALARM_NAME = 'NginxHealthCheckAlarm'


def collect_alarm_history():
    """Scrape CloudWatch alarm history for recent state changes."""
    try:
        now = datetime.now(timezone.utc)
        start_time = now - timedelta(hours=24)

        response = cloudwatch.describe_alarm_history(
            AlarmName=ALARM_NAME,
            HistoryItemType='StateUpdate',
            StartDate=start_time,
            EndDate=now,
            MaxRecords=10,
            ScanBy='TimestampDescending'
        )

        history_items = []
        for item in response.get('AlarmHistoryItems', []):
            history_items.append({
                'alarm': item.get('AlarmName', 'Unknown'),
                'timestamp': item.get('Timestamp', '').isoformat() if hasattr(item.get('Timestamp', ''), 'isoformat') else str(item.get('Timestamp', '')),
                'summary': item.get('HistorySummary', ''),
                'type': item.get('HistoryItemType', '')
            })

        return history_items
    except Exception as e:
        return [{'error': f'Failed to collect alarm history: {str(e)}'}]


def collect_remediation_logs():
    """Scrape CloudWatch Logs from the remediation Lambda function."""
    try:
        now = int(datetime.now(timezone.utc).timestamp() * 1000)
        one_day_ago = now - (24 * 60 * 60 * 1000)

        response = logs.filter_log_events(
            logGroupName=LOG_GROUP_NAME,
            startTime=one_day_ago,
            endTime=now,
            limit=20,
            filterPattern='ERROR OR WARN OR restart OR remediat OR nginx OR ssm'
        )

        log_entries = []
        for event in response.get('events', []):
            log_entries.append({
                'timestamp': datetime.fromtimestamp(
                    event.get('timestamp', 0) / 1000, tz=timezone.utc
                ).isoformat(),
                'message': event.get('message', '').strip()
            })

        return log_entries
    except Exception as e:
        return [{'error': f'Failed to collect remediation logs: {str(e)}'}]


def collect_instance_status():
    """Check EC2 instance health status."""
    try:
        response = ec2.describe_instance_status(
            IncludeAllInstances=True
        )

        instances = []
        for status in response.get('InstanceStatuses', []):
            instances.append({
                'instance_id': status.get('InstanceId', 'Unknown'),
                'state': status.get('InstanceState', {}).get('Name', 'Unknown'),
                'system_status': status.get('SystemStatus', {}).get('Status', 'Unknown'),
                'instance_status': status.get('InstanceStatus', {}).get('Status', 'Unknown'),
                'availability_zone': status.get('AvailabilityZone', 'Unknown')
            })

        return instances
    except Exception as e:
        return [{'error': f'Failed to collect instance status: {str(e)}'}]


def lambda_handler(event, context):
    """Temporary handler to test the log collector functions."""
    # Collect infrastructure data from all three sources
    alarm_history = collect_alarm_history()
    remediation_logs = collect_remediation_logs()
    instance_status = collect_instance_status()

    # Return the raw collected data for testing
    return {
        'statusCode': 200,
        'headers': {
            'Content-Type': 'application/json'
        },
        'body': json.dumps({
            'data_collected': {
                'alarm_events': len(alarm_history),
                'log_entries': len(remediation_logs),
                'instances_checked': len(instance_status)
            },
            'alarm_history': alarm_history,
            'remediation_logs': remediation_logs,
            'instance_status': instance_status
        })
    }

Test the log collector

Your code is deployed. Time to invoke the function and see if it can actually pull data from your Part 1 infrastructure.

Click the Test tab in the Lambda function page.
Enter TestCollector in the Event name field.
Replace the event JSON with this empty object:

{
  "body": "{}"
}

Why this test event?

The test event mimics what API Gateway sends to Lambda. The body field contains a JSON string. Your handler does not parse the body yet (that comes in Step 3), so an empty object is fine for now.

Click Save to save the test event.
Click Test to invoke the function.

After a few seconds, you should see a green Execution result: succeeded panel. Expand the Details section to see the response.

What should I see?

The response body contains a data_collected section showing how many events each collector found. You should see numbers for alarm_events, log_entries, and instances_checked.

If your Part 1 infrastructure has been running and had incidents, you will see actual alarm history entries, remediation log messages, and instance status data. If your system has been quiet, some counts may be 0. That is normal.

Seeing an error instead of a success?

If you see AccessDeniedException, your IAM inline policy was not attached correctly. Go back to the IAM role and verify that DevOpsCopilotPermissions is listed.
If you see ResourceNotFoundException mentioning the log group, your LOG_GROUP_NAME does not match the actual name of your Part 1 remediation Lambda's log group. Check CloudWatch Logs in the console to find the correct log group name.
If the function times out, make sure you increased the timeout to 30 seconds in the General configuration.

Help me troubleshoot my DevOpsCopilot Lambda test failure.

Your log collector is pulling real data from your Part 1 infrastructure. Next up, you will connect it to Amazon Bedrock so it can analyze that data and answer your questions with AI.

Add the AI Analysis Layer with Bedrock

Your log collector is pulling real data from CloudWatch alarms, remediation logs, and EC2 instance status. That is a solid foundation, but raw logs are hard to read and even harder to act on quickly.

Now you will connect your Lambda function to Amazon Bedrock so that a foundation model can analyze that raw data and return a plain-English summary. You will use the Converse API to send your collected infrastructure data to Nova Micro along with a natural language question.

In this step, get ready to:

Add a Bedrock analysis function that sends collected logs and a user question to Nova Micro.
Update the Lambda handler to orchestrate log collection and AI analysis end-to-end.
Test the copilot with sample questions about your infrastructure.

Add the Bedrock analysis function

Your three collector functions return raw data, but that data needs to go somewhere useful. The analyze_with_bedrock function bundles everything into a structured prompt, sends it to Nova Micro via the Converse API, and returns the AI's answer.

In your lambda_function.py in the Lambda console, scroll down to below the collect_instance_status function.
Add the following function above your existing lambda_handler function:

def analyze_with_bedrock(question, alarm_history, remediation_logs, instance_status):
    """Send collected infrastructure data to Bedrock Nova Micro for AI analysis."""

    # Define the AI's role and response guidelines
    system_prompt = """You are an AI DevOps Copilot. You analyze AWS infrastructure data and provide clear, 
actionable summaries. You have access to CloudWatch alarm history, Lambda remediation logs, 
and EC2 instance status data. 

When answering:
- Be concise and direct
- Highlight any failures or issues first
- Mention timestamps when relevant
- Suggest next steps if appropriate
- If no issues are found, confirm the system is healthy"""

    # Structure all collected data and the user's question into one prompt
    context = f"""## Infrastructure Data Collected

### CloudWatch Alarm History (Last 24 Hours)
{json.dumps(alarm_history, indent=2)}

### Remediation Lambda Logs (Last 24 Hours)
{json.dumps(remediation_logs, indent=2)}

### EC2 Instance Status (Current)
{json.dumps(instance_status, indent=2)}

## User Question
{question}"""

    # Send the prompt to Nova Micro using the Converse API
    response = bedrock.converse(
        modelId=MODEL_ID,
        messages=[
            {
                'role': 'user',
                'content': [{'text': context}]
            }
        ],
        system=[{'text': system_prompt}],
        inferenceConfig={
            'maxTokens': 1024,
            'temperature': 0.3
        }
    )

    # Extract the AI's text answer and token usage stats
    result_text = response['output']['message']['content'][0]['text']
    usage = response.get('usage', {})

    return {
        'answer': result_text,
        'tokens_used': {
            'input': usage.get('inputTokens', 0),
            'output': usage.get('outputTokens', 0)
        }
    }

What does this code do?

system_prompt tells Nova Micro who it is and how to respond. It prioritizes failures, timestamps, and actionable next steps.
context bundles the three data sources (alarm history, remediation logs, instance status) into a Markdown-formatted string along with the user's question. The model reads the data and answers the question in one pass.
bedrock.converse() is AWS's recommended way to call Bedrock models. It provides a consistent interface across all models. If you later swap Nova Micro for a different model, you only change the modelId.
temperature: 0.3 keeps responses factual and grounded. A higher temperature makes the model more creative, but for log analysis you want precision.

Why the Converse API instead of InvokeModel?

AWS offers two ways to call Bedrock models: InvokeModel and Converse. The older InvokeModel requires a different JSON payload format for every model provider. The Converse API uses a single, consistent format for all models. AWS recommends Converse for all new integrations.

Do not click Deploy yet. You still need to update the handler function in the next substep.

Update the Lambda handler to use Bedrock

Right now your lambda_handler collects data and returns it raw. You need to update it so it parses a question from the incoming request, runs the three collectors, passes everything to analyze_with_bedrock, and returns the AI's answer.

Replace your existing lambda_handler function with the following code:

def lambda_handler(event, context):
    """Main handler for API Gateway Lambda proxy integration."""

    # Handle CORS preflight requests from browsers
    if event.get('httpMethod') == 'OPTIONS':
        return {
            'statusCode': 200,
            'headers': {
                'Access-Control-Allow-Origin': '*',
                'Access-Control-Allow-Methods': 'POST, OPTIONS',
                'Access-Control-Allow-Headers': 'Content-Type'
            },
            'body': ''
        }

    try:
        # Parse the question from the request body
        body = json.loads(event.get('body', '{}'))
        question = body.get('question', 'Give me a general system health summary.')

        # Step 1: Collect infrastructure data
        alarm_history = collect_alarm_history()
        remediation_logs = collect_remediation_logs()
        instance_status = collect_instance_status()

        # Step 2: Send to Bedrock for AI analysis
        analysis = analyze_with_bedrock(
            question, alarm_history, remediation_logs, instance_status
        )

        return {
            'statusCode': 200,
            'headers': {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            'body': json.dumps({
                'question': question,
                'answer': analysis['answer'],
                'tokens_used': analysis['tokens_used'],
                'data_collected': {
                    'alarm_events': len(alarm_history),
                    'log_entries': len(remediation_logs),
                    'instances_checked': len(instance_status)
                }
            })
        }

    except Exception as e:
        return {
            'statusCode': 500,
            'headers': {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            'body': json.dumps({
                'error': str(e)
            })
        }

What does this handler do?

The CORS preflight block handles OPTIONS requests that browsers send before making cross-origin API calls. You will need this when connecting a web dashboard later.
event.get('body', '{}') parses the incoming request body. If no question is provided, the handler defaults to asking for a general health summary.
The response includes the AI's answer, token usage stats, and a summary of how much data was collected. This metadata helps you monitor cost and coverage.
The except block catches any error (Bedrock timeout, permissions issue, data parsing failure) and returns a clean 500 response instead of crashing.

Click Deploy to save and deploy your updated function.

You should see a green "Successfully updated the function" banner at the top of the page.

Seeing a syntax error after deploying?

Make sure the analyze_with_bedrock function is placed above the lambda_handler function. Python reads top to bottom, so the handler cannot call a function defined after it.
Check that you replaced the entire old lambda_handler function. Leftover lines from the previous version can cause indentation or name errors.
Verify there are no stray characters between the triple-quoted strings (""") in the system_prompt and context variables.

Help me debug a syntax error in my Lambda function

Awesome, I've got everything!

Double check that you have saved and deployed your function before moving on.

I'd like to double check the full code

import json
import boto3
import os
from datetime import datetime, timedelta, timezone

# Initialize AWS clients
bedrock = boto3.client('bedrock-runtime', region_name='us-east-1')
cloudwatch = boto3.client('cloudwatch', region_name='us-east-1')
logs = boto3.client('logs', region_name='us-east-1')
ec2 = boto3.client('ec2', region_name='us-east-1')

# Configuration
MODEL_ID = 'amazon.nova-micro-v1:0'
LOG_GROUP_NAME = '/aws/lambda/NginxRemediationFunction'
ALARM_NAME = 'NginxHealthCheckAlarm'


def collect_alarm_history():
    """Scrape CloudWatch alarm history for recent state changes."""
    try:
        now = datetime.now(timezone.utc)
        start_time = now - timedelta(hours=24)

        response = cloudwatch.describe_alarm_history(
            AlarmName=ALARM_NAME,
            HistoryItemType='StateUpdate',
            StartDate=start_time,
            EndDate=now,
            MaxRecords=10,
            ScanBy='TimestampDescending'
        )

        history_items = []
        for item in response.get('AlarmHistoryItems', []):
            history_items.append({
                'alarm': item.get('AlarmName', 'Unknown'),
                'timestamp': item.get('Timestamp', '').isoformat() if hasattr(item.get('Timestamp', ''), 'isoformat') else str(item.get('Timestamp', '')),
                'summary': item.get('HistorySummary', ''),
                'type': item.get('HistoryItemType', '')
            })

        return history_items
    except Exception as e:
        return [{'error': f'Failed to collect alarm history: {str(e)}'}]


def collect_remediation_logs():
    """Scrape CloudWatch Logs from the remediation Lambda function."""
    try:
        now = int(datetime.now(timezone.utc).timestamp() * 1000)
        one_day_ago = now - (24 * 60 * 60 * 1000)

        response = logs.filter_log_events(
            logGroupName=LOG_GROUP_NAME,
            startTime=one_day_ago,
            endTime=now,
            limit=20,
            filterPattern='ERROR OR WARN OR restart OR remediat OR nginx OR ssm'
        )

        log_entries = []
        for event in response.get('events', []):
            log_entries.append({
                'timestamp': datetime.fromtimestamp(
                    event.get('timestamp', 0) / 1000, tz=timezone.utc
                ).isoformat(),
                'message': event.get('message', '').strip()
            })

        return log_entries
    except Exception as e:
        return [{'error': f'Failed to collect remediation logs: {str(e)}'}]


def collect_instance_status():
    """Check EC2 instance health status."""
    try:
        response = ec2.describe_instance_status(
            IncludeAllInstances=True
        )

        instances = []
        for status in response.get('InstanceStatuses', []):
            instances.append({
                'instance_id': status.get('InstanceId', 'Unknown'),
                'state': status.get('InstanceState', {}).get('Name', 'Unknown'),
                'system_status': status.get('SystemStatus', {}).get('Status', 'Unknown'),
                'instance_status': status.get('InstanceStatus', {}).get('Status', 'Unknown'),
                'availability_zone': status.get('AvailabilityZone', 'Unknown')
            })

        return instances
    except Exception as e:
        return [{'error': f'Failed to collect instance status: {str(e)}'}]


def analyze_with_bedrock(question, alarm_history, remediation_logs, instance_status):
    """Send collected infrastructure data to Bedrock Nova Micro for AI analysis."""

    system_prompt = """You are an AI DevOps Copilot. You analyze AWS infrastructure data and provide clear, 
actionable summaries. You have access to CloudWatch alarm history, Lambda remediation logs, 
and EC2 instance status data. 

When answering:
- Be concise and direct
- Highlight any failures or issues first
- Mention timestamps when relevant
- Suggest next steps if appropriate
- If no issues are found, confirm the system is healthy"""

    context = f"""## Infrastructure Data Collected

### CloudWatch Alarm History (Last 24 Hours)
{json.dumps(alarm_history, indent=2)}

### Remediation Lambda Logs (Last 24 Hours)
{json.dumps(remediation_logs, indent=2)}

### EC2 Instance Status (Current)
{json.dumps(instance_status, indent=2)}

## User Question
{question}"""

    response = bedrock.converse(
        modelId=MODEL_ID,
        messages=[
            {
                'role': 'user',
                'content': [{'text': context}]
            }
        ],
        system=[{'text': system_prompt}],
        inferenceConfig={
            'maxTokens': 1024,
            'temperature': 0.3
        }
    )

    result_text = response['output']['message']['content'][0]['text']
    usage = response.get('usage', {})

    return {
        'answer': result_text,
        'tokens_used': {
            'input': usage.get('inputTokens', 0),
            'output': usage.get('outputTokens', 0)
        }
    }


def lambda_handler(event, context):
    """Main handler for API Gateway Lambda proxy integration."""

    # Handle CORS preflight
    if event.get('httpMethod') == 'OPTIONS':
        return {
            'statusCode': 200,
            'headers': {
                'Access-Control-Allow-Origin': '*',
                'Access-Control-Allow-Methods': 'POST, OPTIONS',
                'Access-Control-Allow-Headers': 'Content-Type'
            },
            'body': ''
        }

    try:
        # Parse the question from the request body
        body = json.loads(event.get('body', '{}'))
        question = body.get('question', 'Give me a general system health summary.')

        # Step 1: Collect infrastructure data
        alarm_history = collect_alarm_history()
        remediation_logs = collect_remediation_logs()
        instance_status = collect_instance_status()

        # Step 2: Send to Bedrock for AI analysis
        analysis = analyze_with_bedrock(
            question, alarm_history, remediation_logs, instance_status
        )

        return {
            'statusCode': 200,
            'headers': {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            'body': json.dumps({
                'question': question,
                'answer': analysis['answer'],
                'tokens_used': analysis['tokens_used'],
                'data_collected': {
                    'alarm_events': len(alarm_history),
                    'log_entries': len(remediation_logs),
                    'instances_checked': len(instance_status)
                }
            })
        }

    except Exception as e:
        return {
            'statusCode': 500,
            'headers': {
                'Content-Type': 'application/json',
                'Access-Control-Allow-Origin': '*'
            },
            'body': json.dumps({
                'error': str(e)
            })
        }

Test with sample questions

The updated handler expects a request body with a question field. You need to update the Lambda test event to match this format so you can test the Bedrock integration directly.

Click the Test tab near the top of the Lambda console.
Replace the existing test event JSON with the following:

{
  "httpMethod": "POST",
  "body": "{\"question\": \"Summarize the last incident\"}"
}

Why is the body a string inside the JSON?

When API Gateway sends a request to Lambda, the body field arrives as a JSON-encoded string, not a nested object. The test event mimics this format so your handler's json.loads(event.get('body')) call works the same way it will in production.

Click Save to save your test event.
Click the Test button to invoke the function.

The function takes a few seconds to run because it calls Bedrock. You should see a green "Execution result: succeeded" banner with a JSON response in the output.

The response body contains your question, the AI's answer summarizing your infrastructure's recent activity, token usage counts, and the number of data points collected.

Pro tip: try different questions

Update the question value in your test event and run it again. Try questions like What services are unhealthy? or Were there any alarm state changes today? to see how Nova Micro responds to different prompts using your real data.

Getting a timeout or AccessDeniedException?

If you see a Task timed out error, your Lambda timeout is too short. Go to Configuration > General configuration > Edit and increase the timeout to at least 30 seconds. Bedrock calls typically take 3-8 seconds.
If you see AccessDeniedException, your Lambda execution role is missing the bedrock:InvokeModel permission. Go to the Configuration > Permissions tab, click on the execution role, and add Bedrock permissions in IAM.
If you see ResourceNotFoundException for the log group, double check that the LOG_GROUP_NAME value matches your Part 1 remediation function's actual log group name in CloudWatch.

Help me troubleshoot my Lambda Bedrock integration

Your copilot can now collect live infrastructure data and analyze it with AI. Next up, you will expose this Lambda function to the outside world by creating a REST API with API Gateway.

Create the REST API with API Gateway

Your Lambda function can collect logs and analyze them with Bedrock, but there is no way for the outside world to reach it. Right now the only way to trigger the function is through the Lambda console's test button.

In this step, you will create a REST API using Amazon API Gateway that exposes a public /ask endpoint. When someone sends a POST request to that endpoint, API Gateway forwards it straight to your DevOpsCopilot function.

In this step, get ready to:

Create a REST API called DevOpsCopilotAPI with a /ask resource.
Set up a POST method with Lambda proxy integration pointing to your DevOpsCopilot function.
Deploy the API to a prod stage and copy the invoke URL.

Create the REST API and /ask resource

API Gateway is the front door that accepts HTTP requests from clients and routes them to backend services like Lambda. You will create a REST API and then add a resource that represents the /ask path.

Open the AWS Management Console and search for API Gateway in the search bar.
Click API Gateway to open the service.
Click Create API.
Find the REST API card (not the one labeled "REST API Private") and click Build.
For API name, enter DevOpsCopilotAPI.
Keep API endpoint type set to Regional.
Click Create API.

You should now see the Resources pane showing only the root resource (/).

What is a REST API resource?

In API Gateway, a resource is a path segment in your API's URL. The root resource / is created automatically. You will add /ask as a child resource so your final endpoint looks like https://{api-id}.execute-api.{region}.amazonaws.com/prod/ask.

Select the root resource / in the Resources pane.
Click Create resource.
For Resource name, enter ask.
Keep Proxy resource turned off.
Keep CORS (Cross Origin Resource Sharing) turned off for now.
Click Create resource.

You should now see /ask listed as a child under the / root in the resource tree.

Don't see the Create resource button?

Make sure you selected the / root resource first. The Create resource button only appears when a parent resource is selected in the tree.

Help me find the Create resource button

Set up the POST method with Lambda proxy integration

A resource on its own does not do anything. You need to attach an HTTP method to it and connect that method to your Lambda function using a Lambda proxy integration.

What is a Lambda proxy integration?

With a proxy integration, API Gateway forwards the entire incoming HTTP request (headers, query parameters, body) directly to your Lambda function. Your function is responsible for returning a properly formatted response with statusCode, headers, and body.

This is the simplest integration type because you do not need to configure request or response mapping in API Gateway.

Select the /ask resource in the Resources pane.
Click Create method.
For Method type, select POST.
For Integration type, select Lambda function.
Turn on Lambda proxy integration.
For Lambda function, enter DevOpsCopilot and select your function from the dropdown.
Click Create method.
Click OK if a dialog appears asking to give API Gateway permission to invoke your Lambda function.

Don't see your Lambda function in the dropdown?

Make sure you are in the same AWS region as your DevOpsCopilot Lambda function. Check the region selector in the top-right corner of the console. Your API Gateway and Lambda function must be in the same region (us-east-1 if you followed Part 1).

Help me find my Lambda function in the dropdown

You should now see the POST method listed under the /ask resource with its integration type set to Lambda.

What happens behind the scenes?

When you clicked OK on the permission dialog, API Gateway added a resource-based policy to your Lambda function. This policy grants the apigateway.amazonaws.com service permission to invoke DevOpsCopilot. Without this permission, API Gateway would receive an Access Denied error every time it tried to call your function.

Deploy the API to a prod stage

Your API exists, but it is not callable yet. In API Gateway, you must deploy the API to a stage before clients can send requests to it.

What is a stage?

A stage is a named snapshot of your API's configuration. Common stage names include dev, staging, and prod. The stage name becomes part of the URL that clients use to call your API.

In the Resources pane, click Deploy API.
For Stage, select New stage.
For Stage name, enter prod.
Click Deploy.

You should now see the Stages pane with your prod stage listed.

Under Stage details, find the Invoke URL.
Click the copy icon next to the URL to copy it to your clipboard.
Paste it somewhere safe (a text file or notes app) because you will need it in the next step.

Your invoke URL looks like https://{api-id}.execute-api.us-east-1.amazonaws.com/prod. The full endpoint for your copilot is this URL with /ask appended to the end.

How the full URL works

https://{api-id}.execute-api.us-east-1.amazonaws.com is the base URL that API Gateway generates for your API.
/prod is the stage name you just created.
/ask is the resource you created earlier.
When a client sends a POST request to this full URL, API Gateway routes it to your DevOpsCopilot Lambda function.

Seeing a Missing Authentication Token error?

If you paste just the base invoke URL into a browser, you will see a {"message":"Missing Authentication Token"} error. This is normal. Your browser sends a GET request to the root / resource, which has no method defined.

Your POST method lives on /ask, and you need a tool like curl or PowerShell to send POST requests. You will do this in the next step.

Help me understand this error

Your REST API is live. Next up, you will test the full pipeline by sending real questions to your copilot and verifying it returns AI-generated answers.

Test the Copilot End-to-End

Your REST API is live. You have a /ask endpoint backed by your DevOpsCopilot Lambda function, ready to accept natural language questions about your infrastructure.

In this step, you'll put the copilot through its paces by sending it real questions, triggering an actual incident on your EC2 instance, and verifying the copilot can detect and explain what happened.

In this step, get ready to:

Send POST requests with natural language questions to your API endpoint and receive AI-generated analysis.
Trigger a real incident by stopping nginx on your EC2 instance and verify the copilot detects it.
Confirm the full data flow from API Gateway through Lambda to Bedrock and back.

Send your first API request

Your full API endpoint is the invoke URL you copied in Step 4 with /ask appended to the end. This is the endpoint your copilot listens on for incoming questions.

Send a health summary question to your copilot by running this command:

macOS

curl -X POST [[API_ENDPOINT="https://abc123.execute-api.us-east-1.amazonaws.com/prod/ask"]] -H "Content-Type: application/json" -d '{"question": "Give me a full system health summary"}'

Windows

Invoke-RestMethod -Uri "[[API_ENDPOINT="https://abc123.execute-api.us-east-1.amazonaws.com/prod/ask"]]" -Method POST -ContentType "application/json" -Body '{"question": "Give me a full system health summary"}'

What should I see?

You should get back a JSON response containing these fields:

question echoes back what you asked.
answer contains the AI-generated analysis from Nova Micro.
tokens_used shows how many input and output tokens the AI consumed.
data_collected shows how many alarm events, log entries, and instances were scraped.

The actual answer content depends on what your Part 1 infrastructure has been doing. If things have been quiet, the copilot reports a healthy system.

Getting a 500 error or no response?

Double check your API endpoint URL. It should end with /prod/ask. A missing /ask returns a Missing Authentication Token error.
If you see a timeout error, check that your Lambda function's timeout is set to at least 30 seconds. Bedrock calls take several seconds to complete.
If you see an AccessDeniedException in the error, your Lambda's execution role is missing the bedrock:InvokeModel permission. Go back to IAM and add it to the role.

Still stuck? Help me debug my DevOps Copilot API response.

Trigger a real incident

A health summary is useful, but the real power of the copilot is incident analysis. To prove it works, you need fresh incident data.

You'll stop nginx on your EC2 instance from Part 1, let the self-healing infrastructure kick in, then ask the copilot what happened.

Connect to your EC2 instance from Part 1 by running this SSH command:

ssh -i [[PEMPATH="path to your .pem key file"]] ec2-user@[[EC2_PUBLIC_IP="your EC2 public IPv4 address"]]

Stop nginx to simulate a service failure by running:

sudo systemctl stop nginx

Exit the SSH session by running:

exit

What happens next?

Your Part 1 infrastructure detects the failure automatically. The CloudWatch alarm notices nginx is down and transitions to ALARM state.

The remediation Lambda function from Part 1 kicks in and restarts nginx via Systems Manager. All of this generates log entries and alarm history that your copilot can now analyze.

Wait 3 to 5 minutes for the self-healing pipeline to detect the failure, trigger the alarm, and run the remediation.
Ask the copilot about the incident by running this command:

macOS

curl -X POST [[API_ENDPOINT="https://abc123.execute-api.us-east-1.amazonaws.com/prod/ask"]] -H "Content-Type: application/json" -d '{"question": "What happened in the last hour?"}'

Windows

Invoke-RestMethod -Uri "[[API_ENDPOINT="https://abc123.execute-api.us-east-1.amazonaws.com/prod/ask"]]" -Method POST -ContentType "application/json" -Body '{"question": "What happened in the last hour?"}'

The AI response should mention the nginx failure, the alarm state change, and the remediation that brought the service back online.

Copilot says no incidents found?

The self-healing pipeline needs time to complete. Wait at least 3 to 5 minutes after stopping nginx, then try the query again.
Check that your ALARM_NAME and LOG_GROUP_NAME values in your Lambda code match the actual alarm and log group names from your Part 1 setup.
Open the CloudWatch console and verify the alarm transitioned to ALARM state. If it didn't, your Part 1 health check may not be running.

Need help? Help me figure out why my copilot is not detecting the nginx incident.

Verify the full data flow

You just watched the entire pipeline work end-to-end. Every time you send a question, four things happen behind the scenes.

What is happening in the pipeline?

API Gateway receives your POST request and forwards the question to the Lambda function.
Lambda collects fresh data from three sources: CloudWatch alarm history, remediation Lambda logs, and EC2 instance status.
Lambda bundles all the collected data into a prompt and sends it to Bedrock's Converse API along with your question.
Nova Micro analyzes the raw logs, identifies patterns and incidents, and returns a human-readable summary as the API response.

The data is always collected fresh on every request. The copilot never uses stale or cached information.

Confirm the pipeline is working by checking the data_collected field in your latest API response.

You should see non-zero counts for alarm_events and log_entries now that you have triggered an incident. The instances_checked count reflects how many EC2 instances are in your account.

Pro tip: try different questions

Your copilot can answer a wide range of questions. Try asking "Summarize the last incident and how it was resolved" or "What services are currently unhealthy?" to see how Nova Micro interprets different prompts against the same data.

Secret mission

Build the AI DevOps Chat Dashboard

Your API works, but typing curl commands isn't exactly a polished ops experience. Build a dark-themed chat dashboard that lets you talk to your AI copilot through a real conversational interface, complete with quick-action buttons and live response streaming.

Before you go

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. The Lambda function and API Gateway are covered under the AWS free tier, so they cost nothing when idle. However, the EC2 instance from Part 1 continues to incur costs while running, and Bedrock charges a small per-token fee each time you query the copilot.

Cost warning

Your EC2 instance from Part 1 is the main ongoing cost. If you are not actively testing, stop the instance to avoid charges. Bedrock Nova Micro costs approximately $0.035 per 1,000 input tokens and $0.14 per 1,000 output tokens, which amounts to pennies per query. The Lambda function and API Gateway are free-tier eligible and cost nothing when idle.

Resources you used:

Lambda function (DevOpsCopilot) with IAM execution role.
API Gateway REST API (DevOpsCopilotAPI) with /ask resource and prod stage.
Amazon Bedrock model access for Amazon Nova Micro.
EC2 instance from Part 1 (with CloudWatch alarms, EventBridge rules, SNS, and remediation Lambda).
Chat dashboard HTML file (local, if you completed the Secret Mission).

Keep everything running

No action needed. Choose this if you are still actively building or testing your copilot. Your EC2 instance from Part 1 will continue to incur costs while running.

Your Lambda function and API Gateway remain available and cost nothing when idle under the free tier.
Bedrock model access stays enabled at no cost. You only pay when you send queries.
Monitor your EC2 instance costs in the AWS Billing dashboard.

Pause - I'll come back to this later

Stop the EC2 instance from Part 1 to save costs. The API Gateway and Lambda function remain deployed but cost nothing when idle.

Navigate to the EC2 console and select your Part 1 instance.
Choose Instance state, then Stop instance.
Your Lambda function, API Gateway, and Bedrock model access will remain in place at no cost.
When you come back, start the EC2 instance again and the copilot will resume collecting data from your infrastructure.

Delete - I don't want to use this again

Remove all project resources and start fresh. Bedrock model access can remain enabled at no cost.

Navigate to the API Gateway console, select DevOpsCopilotAPI, and delete it.
Navigate to the Lambda console, select the DevOpsCopilot function, and delete it.
Navigate to the IAM console, find the execution role you created for the DevOpsCopilot Lambda, and delete it.
Bedrock model access does not need to be disabled. It costs nothing when you are not sending queries.
Delete the local index.html file if you completed the Secret Mission.
To fully clean up Part 1 resources (EC2, CloudWatch alarms, EventBridge, SNS, remediation Lambda), refer to Part 1's cleanup instructions.

Mission accomplished

That's a Wrap!

You just built an AI-powered DevOps Copilot from scratch. Your infrastructure can now answer questions about its own health, incidents, and remediation history in plain English.

Here's what you accomplished:

Built a log collector Lambda that scrapes CloudWatch alarm history, remediation logs, and EC2 instance status using Boto3.
Integrated the Bedrock Converse API with Amazon Nova Micro to turn raw infrastructure data into AI-powered, human-readable summaries.
Created a REST API with API Gateway and Lambda proxy integration to expose your copilot as a queryable endpoint for natural language questions.
Built an HTML chat dashboard with a dark ops-console theme for a visual DevOps Copilot experience.

Ready to quiz yourself?