Deploy Multi-Agent IT Triage to AWS

Introduction

30 Second Summary

Every IT team drowns in support tickets. Categorizing, prioritizing, and routing each one takes human time that could go toward actually fixing things. AI agents can handle that triage instantly, but only if they're running somewhere your team can actually reach them.

In this project, you will take a working multi-agent IT triage system and deploy it as a live API on AWS. You'll containerize it with Docker, push it to a registry, and run it on ECS Fargate behind a load balancer so anyone on your team can send it a ticket and get back a structured triage report.

What You'll Build

You push a raw IT support ticket to a live endpoint on AWS and seconds later get back a structured JSON triage report with category, priority, assigned team, and a drafted response, all powered by your multi-agent CrewAI system running inside a Docker container on AWS Fargate.

By the end of this project, you'll have:

• A live API endpoint where you POST any IT ticket and get a structured triage report back as JSON.
• A lean Docker image built with a multi-stage Dockerfile and uv that caches dependencies and rebuilds in seconds when you change code.
• A running container on ECS Fargate with an Application Load Balancer routing traffic to it, health-checking it, and auto-restarting if it crashes.
• Secret Mission: Deploy a separate production environment from the same image with different model configuration.

Are there any prerequisites?

This project picks up where the "Multi-Agent IT Ticket Triage with CrewAI" project left off. You should have the working local project on your machine before starting.

You also need macOS with Docker Desktop installed and an AWS account. The first step walks you through configuring the AWS CLI and creating an OpenRouter account.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. You're taking your local multi-agent IT triage system and deploying it to AWS as a production API.

Set Up AWS CLI, Docker, and OpenRouter

Your local tools need to talk to AWS and OpenRouter before you can deploy anything. Without the CLI configured and verified, you cannot push images or create infrastructure.

In this step, you'll confirm that your AWS CLI, Docker Desktop, and OpenRouter API key are all working and ready for the deployment steps ahead.

In this step, get ready to:

• Verify your AWS CLI can authenticate with your AWS account.
• Confirm Docker Desktop is running and healthy.
• Open your project directory in the terminal.

Verify your AWS CLI configuration

The AWS CLI is the command-line tool that lets you create and manage AWS resources from your terminal. Every deployment action in this project goes through it.

Before you run the verification, what do you expect to see? If your CLI is configured correctly, you should get back JSON with your account details.

• Verify your AWS CLI can reach your account by running this command:

aws sts get-caller-identity

What does this command do?

This command asks AWS "who am I?" and returns JSON containing your Account ID, UserId, and Arn.

If you see these three fields, your CLI is authenticated and ready to push images to Amazon ECR and create ECS infrastructure.

You should see output like this:

{
    "UserId": "AIDACKCEVSQ6C2EXAMPLE",
    "Account": "123456789012",
    "Arn": "arn:aws:iam::123456789012:user/YourUserName"
}

Seeing "Unable to locate credentials"?

This means your AWS CLI is not configured yet. Run aws configure and enter your Access Key ID, Secret Access Key, us-east-1 for region, and json for output format.

If you see an "InvalidClientTokenId" error, your Access Key ID may be incorrect. Regenerate your credentials in the AWS IAM console. Help me fix my AWS CLI credentials

Confirm your OpenRouter API key is ready

Your container will call OpenRouter instead of running Ollama locally. This means your app can run anywhere on ECS Fargate without needing a GPU.

Why OpenRouter instead of Ollama?

Ollama requires a local LLM running on the same machine. Fargate containers do not have GPUs and cannot run large language models locally.

OpenRouter gives your container access to LLMs via a simple API call. Your app sends a request to OpenRouter's endpoint and gets a response back, regardless of where the container is running.

• Confirm you have your OpenRouter API key saved somewhere accessible. You will need to paste it as an environment variable in the next step.
• Confirm you have added at least $5 in credit to your OpenRouter account. This is more than enough for hundreds of triage runs.

Keep your key handy but safe

Store your API key in a password manager or a secure note. You will paste it into terminal commands and configuration files during this project. Never commit it to version control.

Verify Docker Desktop and open your project

Docker packages your application into a container image that runs identically on your laptop and on AWS. You need the Docker daemon running to build images.

• Confirm Docker Desktop is responding by running this command:

docker info

You should see a Server section in the output with details like Operating System, Storage Driver, and container counts.

Seeing "Cannot connect to the Docker daemon"?

Docker Desktop is not running. Press Cmd+Space to open Spotlight, type Docker, and press Enter to launch it. Wait for the whale icon in your menu bar to stop animating, then try docker info again.

Help me get Docker Desktop running on my Mac

• Navigate to your it-triage-crew project directory by running:

cd ~/Desktop/it-triage-crew

• Confirm you are in the right directory by running:

ls

You should see your pyproject.toml, src/ directory, and knowledge/ directory in the output.

Don't see pyproject.toml?

You may be in the wrong directory. Make sure you are inside the it-triage-crew folder from your previous project, not a parent or sibling directory.

Help me find my it-triage-crew project directory

All three tools are verified and ready. Your AWS CLI can reach your account, Docker can build images, and your OpenRouter key is standing by for API calls.

Next up, you will add a FastAPI layer to your project and swap Ollama for OpenRouter so your app can run inside a container on AWS.

Add a FastAPI Layer and Switch to OpenRouter

Your triage system works locally as a CLI tool, but a CLI cannot sit behind a load balancer on AWS. To deploy it as a production API, you need an HTTP endpoint that accepts POST requests and returns JSON.

You also need to swap out Ollama for OpenRouter. Ollama requires a local GPU or large model running on the same machine. OpenRouter is an API-based LLM router that works anywhere with an internet connection, perfect for a container on AWS Fargate.

In this step, get ready to:

• Add FastAPI and uvicorn dependencies and create an HTTP API layer.
• Rewrite the crew to use OpenRouter instead of Ollama with prompt-injected knowledge.
• Test the full system locally with a curl POST request.

Add dependencies and create the FastAPI app

Your project currently has CrewAI installed, but it does not have an HTTP framework. You need FastAPI for the web layer, uvicorn as the ASGI server, and the crewai[litellm] extra for OpenRouter support.

• Install the three new dependencies by running this command:

uv add fastapi uvicorn 'crewai[litellm]'

What does this command do?

This adds fastapi (the web framework), uvicorn (the server that runs FastAPI), and crewai[litellm] (the LiteLLM integration that lets CrewAI talk to OpenRouter) to your pyproject.toml and installs them.

• Confirm the install succeeded by checking the output ends with a resolved dependencies message and no errors.

Seeing a resolution error?

If uv reports a version conflict, make sure you are inside your it-triage-crew project directory where pyproject.toml lives. Run ls pyproject.toml to verify.

Help me troubleshoot this dependency error.

Now you can create the API file that gives your triage system an HTTP interface.

• Create the file src/it_triage_crew/api.py and paste in the following code:

import os
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel

from it_triage_crew.crew import run_triage

app = FastAPI(title="IT Ticket Triage API")

class TicketRequest(BaseModel):
    ticket: str

class HealthResponse(BaseModel):
    status: str

@app.get("/health")
def health_check() -> HealthResponse:
    return HealthResponse(status="healthy")

@app.post("/triage")
def triage_ticket(request: TicketRequest):
    if not os.getenv("OPENROUTER_API_KEY"):
        raise HTTPException(
            status_code=500,
            detail="OPENROUTER_API_KEY environment variable not set",
        )
    result = run_triage(request.ticket)
    return {"triage_report": result}

What does this code do?

This file exposes two HTTP endpoints. GET /health returns a simple status check that load balancers use to verify the container is alive.

POST /triage accepts a JSON body with a ticket field, checks that the API key is available, runs your multi-agent crew, and returns the structured triage report as JSON.

• Save api.py.
• Verify the file was created by running:

ls src/it_triage_crew/api.py

You should see the file path printed back to you, confirming it exists.

File not found?

Make sure you created the file inside src/it_triage_crew/, not in the project root. Run ls src/it_triage_crew/ to see what files are in that directory.

Help me figure out where to put my api.py file.

Rewrite crew.py for OpenRouter and prompt-injected knowledge

Your existing crew.py uses Ollama for local inference and ChromaDB for RAG. Neither works inside a container on Fargate.

Why prompt injection instead of RAG?

Your three knowledge base files total roughly 2000 tokens. For a small, static reference set like this, injecting the full text directly into the agent prompts is simpler and faster than maintaining a vector store in a container.

You already built the RAG version in the original project. This teaches a legitimate production simplification: skip the embedding pipeline when the data fits comfortably in a prompt.

• Open src/it_triage_crew/crew.py and delete all of its contents.
• Paste in the following imports, knowledge loader, and LLM configuration:

import os
from pathlib import Path
from crewai import Agent, Task, Crew, Process, LLM
from pydantic import BaseModel, Field
from typing import List

# Load knowledge base files at module level
KNOWLEDGE_DIR = Path(__file__).parent.parent.parent / "knowledge"

def _load_knowledge() -> str:
    """Load all knowledge base markdown files into a single string."""
    knowledge_text = ""
    for filename in ["escalation_policy.md", "sla_definitions.md", "team_routing_rules.md"]:
        filepath = KNOWLEDGE_DIR / filename
        if filepath.exists():
            knowledge_text += filepath.read_text() + "\n\n"
    return knowledge_text

KNOWLEDGE_BASE = _load_knowledge()

# Configure OpenRouter LLM
MODEL = os.getenv("OPENROUTER_MODEL", "openrouter/google/gemini-2.5-flash")

openrouter_llm = LLM(
    model=MODEL,
    base_url="https://openrouter.ai/api/v1",
    api_key=os.getenv("OPENROUTER_API_KEY"),
    temperature=0.2,
)

What does this section do?

_load_knowledge() reads your three markdown files from the knowledge/ directory and concatenates them into one string. This string gets injected directly into agent prompts later.

The LLM object points at OpenRouter's API using the base_url. The MODEL variable defaults to openrouter/google/gemini-2.5-flash but can be overridden per environment via the OPENROUTER_MODEL environment variable.

• Below the LLM configuration, add the structured output model:

class TriageReport(BaseModel):
    """Structured output for IT ticket triage"""
    category: str = Field(description="Ticket category: network, hardware, software, or access")
    priority: str = Field(description="Priority level: P1, P2, P3, or P4")
    assigned_team: str = Field(description="Team to handle the ticket")
    sla_response_time: str = Field(description="Required response time per SLA")
    sla_resolution_time: str = Field(description="Required resolution time per SLA")
    drafted_response: str = Field(description="Initial response to send to the ticket submitter")
    policy_references: List[str] = Field(description="List of policies referenced in the decision")

What is TriageReport?

This Pydantic model defines the structured JSON shape that the final agent must produce. CrewAI uses it to enforce the output format via OpenRouter's structured output support.

• Below the TriageReport class, add the three agent definitions:

# Agent definitions
classifier_agent = Agent(
    role="IT Ticket Classifier",
    goal="Categorize incoming IT support tickets into exactly one type: network, hardware, software, or access",
    backstory="You are a veteran IT support analyst who has processed thousands of tickets. You instantly recognize patterns that indicate the ticket category.",
    llm=openrouter_llm,
    verbose=True,
)

priority_assessor_agent = Agent(
    role="IT Priority Assessor",
    goal="Determine the urgency and business impact of an IT ticket and assign a priority level (P1-P4) based on organizational SLA policies",
    backstory="You are a senior IT operations manager who determines urgency and impact by referencing documented SLA definitions and escalation policies.",
    llm=openrouter_llm,
    verbose=True,
)

router_agent = Agent(
    role="IT Ticket Router",
    goal="Route the ticket to the correct support team and draft an initial response to the requester",
    backstory="You are an IT service desk lead who knows every team in the department. You route tickets efficiently based on documented team routing rules.",
    llm=openrouter_llm,
    verbose=True,
)

What do these agents do?

The three agents mirror your original project: one classifies the ticket category, one assesses priority against your SLA policies, and one routes it to the correct team. Each agent now uses openrouter_llm instead of a local Ollama model.

• Below the agents, add the classify task:

# Task definitions
classify_task = Task(
    description="Classify the following IT support ticket into exactly one category: network, hardware, software, or access.\n\nTicket: {ticket}",
    expected_output="A single ticket category with a brief explanation.",
    agent=classifier_agent,
)

• Below classify_task, add the priority and routing tasks that inject the knowledge base into their prompts:

prioritize_task = Task(
    description=(
        "Assess the priority level of this IT support ticket: {ticket}\n\n"
        "Use the following organizational policies to determine priority. "
        "You MUST cite specific SLA thresholds and escalation rules.\n\n"
        f"--- ORGANIZATIONAL POLICIES ---\n{KNOWLEDGE_BASE}\n--- END POLICIES ---\n\n"
        "Provide: priority level (P1-P4), response time, resolution time, "
        "and justification with policy citations."
    ),
    expected_output="Priority level with SLA times and policy citations.",
    agent=priority_assessor_agent,
)

route_task = Task(
    description=(
        "Based on the classification and priority, route this ticket to the correct team "
        "and draft an initial response.\n\n"
        f"--- TEAM ROUTING RULES ---\n{KNOWLEDGE_BASE}\n--- END RULES ---\n\n"
        "Ticket: {ticket}"
    ),
    expected_output="JSON triage report with all fields populated.",
    agent=router_agent,
    output_json=TriageReport,
)

How do tasks use the knowledge base?

Each task injects the {ticket} placeholder that CrewAI fills at runtime. The prioritize_task and route_task embed the full knowledge base text directly into their prompts using the KNOWLEDGE_BASE string.

The route_task uses output_json=TriageReport so CrewAI enforces the structured Pydantic output format.

• Finally, add the crew assembly and the run_triage function at the bottom of the file:

# Crew assembly
triage_crew = Crew(
    agents=[classifier_agent, priority_assessor_agent, router_agent],
    tasks=[classify_task, prioritize_task, route_task],
    process=Process.hierarchical,
    manager_llm=openrouter_llm,
    verbose=True,
)

def run_triage(ticket_text: str) -> dict:
    """Run the triage crew on a ticket and return the result."""
    result = triage_crew.kickoff(inputs={"ticket": ticket_text})
    if hasattr(result, "json_dict") and result.json_dict:
        return result.json_dict
    return {"raw_output": result.raw}

What does this section do?

The Crew uses Process.hierarchical with the same OpenRouter model as the manager. This means the manager LLM coordinates which agent works on each task.

run_triage() is the single entry point that api.py calls. It kicks off the crew and returns either the structured JSON dict or the raw output as a fallback.

• Save crew.py.
• Verify the syntax by running:

python -c "import ast; ast.parse(open('src/it_triage_crew/crew.py').read()); print('Syntax OK')"

You should see Syntax OK printed.

Seeing a SyntaxError?

Check the line number reported in the error. The most common issues are a missing comma between function arguments, an unmatched parenthesis, or extra whitespace at the start of a line.

Help me find the syntax issue in my crew.py file.

One last file needs updating before you can test. The existing main.py runs the CLI version of your crew. You need it to start the FastAPI server instead.

• Open src/it_triage_crew/main.py and replace all of its contents with:

import uvicorn

def main():
    uvicorn.run(
        "it_triage_crew.api:app",
        host="0.0.0.0",
        port=8000,
        reload=False,
    )

if __name__ == "__main__":
    main()

What does this change?

Instead of running the crew directly from the command line, main.py now starts a uvicorn server that hosts your FastAPI app on port 8000. This is the entry point Docker will use later.

• Save main.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved all three files: api.py, crew.py, and main.py.

ⓧ I'd like to double check the full code

Scroll up and confirm you have each section in place, in order:

• api.py: imports, TicketRequest and HealthResponse models, GET /health endpoint, POST /triage endpoint.
• crew.py: imports and knowledge loader, openrouter_llm config, TriageReport model, 3 agents, 3 tasks, triage_crew assembly, run_triage() function.
• main.py: imports uvicorn and runs the app on port 8000.

Test locally with curl

All three files are in place. Time to prove the system works end-to-end by starting the server and sending it a ticket.

• Export your OpenRouter API key as an environment variable by running:

export OPENROUTER_API_KEY=your-key-here

What goes in place of your-key-here?

Paste the actual API key you created on OpenRouter in the previous step. This makes the key available to any process you run in this terminal session.

• Start the FastAPI server by running:

uv run uvicorn it_triage_crew.api:app --host 0.0.0.0 --port 8000

You should see uvicorn output including Uvicorn running on http://0.0.0.0:8000. The server is now listening for requests.

Seeing an ImportError or ModuleNotFoundError?

If you see a message about litellm not being found, the crewai[litellm] extra did not install correctly. Re-run uv add 'crewai[litellm]' and try starting the server again.

If you see a SyntaxError pointing at crew.py, check the line number and look for a missing comma or unmatched parenthesis.

Help me debug my server startup error.

Before you send a triage request, what do you think will happen when you POST a ticket to this endpoint? The crew has three agents that each need to call OpenRouter.

• Open a second terminal tab and send a test ticket by running:

curl -X POST http://localhost:8000/triage \
  -H "Content-Type: application/json" \
  -d '{"ticket": "My laptop screen is flickering and I cannot attend my video calls. This is urgent as I have a client presentation in 2 hours."}'

This takes 15-30 seconds as the three agents process the ticket through OpenRouter. You should see a JSON response with a triage_report object containing category, priority, assigned_team, and other fields filled in by your agents.

Getting an empty or error response?

If you see a 500 error about OPENROUTER_API_KEY not set, make sure you ran the export command in the same terminal where uvicorn is running, then restart the server.

If you see an authentication error from OpenRouter, verify your key is correct and that you have credits loaded on your OpenRouter account.

Help me figure out why my triage request is failing.

• Once you see a successful JSON triage report, press Ctrl+C in the first terminal to stop the server.

Your triage system now runs as an HTTP API powered by OpenRouter. Next up, you will package it into a Docker image so it can run anywhere without depending on your local machine.

Build the Multi-Stage Docker Image with uv

Your FastAPI triage API works locally, but it depends on your machine's Python environment and installed packages. If you tried to run it on a different machine or a cloud server, you would need to recreate that entire environment from scratch.

Docker solves this by packaging your code, dependencies, and runtime into a single portable image. A naive Dockerfile would produce a 2GB+ image and take minutes to rebuild on every code change. In this step, you will write a multi-stage Dockerfile that uses uv to produce a lean ~300MB image where dependency changes are cached and code changes rebuild in seconds.

In this step, get ready to:

• Write a multi-stage Dockerfile and a .dockerignore file.
• Build the Docker image and test it locally with and without an API key.

Write the Dockerfile and .dockerignore

A multi-stage Docker build uses two separate images. The first stage (the builder) installs all your dependencies. The second stage (the runtime) copies only the finished virtual environment, producing a much smaller final image.

• Create a new file called Dockerfile in the root of your it-triage-crew project folder.
• Add the builder stage by pasting this code:

FROM python:3.12-slim AS builder

COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/

WORKDIR /app

ENV UV_COMPILE_BYTECODE=1
ENV UV_LINK_MODE=copy

# Install dependencies first (cached layer)
RUN --mount=type=cache,target=/root/.cache/uv \
    --mount=type=bind,source=uv.lock,target=uv.lock \
    --mount=type=bind,source=pyproject.toml,target=pyproject.toml \
    uv sync --locked --no-install-project --no-editable

# Copy the project and install it
COPY . /app
RUN --mount=type=cache,target=/root/.cache/uv \
    uv sync --locked --no-editable

What does this builder stage do?

• COPY --from=ghcr.io/astral-sh/uv:latest pulls the uv binary directly from its official container image, so you do not need to install it with pip.
• UV_COMPILE_BYTECODE=1 pre-compiles Python files to bytecode, improving startup time in the final container.
• UV_LINK_MODE=copy ensures files are copied (not symlinked) so they survive being moved to a different stage.
• The first RUN installs only dependencies (--no-install-project) using bind-mounted lockfiles. This layer is cached and only rebuilds when pyproject.toml or uv.lock change.
• The second RUN copies your app code and installs the project itself. Code changes only rebuild this thin layer, not the dependency layer above.

• Below the builder stage in the same file, add the runtime stage:

# Runtime stage
FROM python:3.12-slim

WORKDIR /app

# Copy the virtual environment from builder
COPY --from=builder /app/.venv /app/.venv

# Copy knowledge base files
COPY knowledge/ /app/knowledge/

# Set PATH to use the virtual environment
ENV PATH="/app/.venv/bin:$PATH"

EXPOSE 8000

HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
    CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')"

CMD ["uvicorn", "it_triage_crew.api:app", "--host", "0.0.0.0", "--port", "8000"]

What does the runtime stage do?

• FROM python:3.12-slim starts a brand-new slim image. None of the builder's tools (uv, pip, build artifacts) carry over.
• COPY --from=builder /app/.venv is the key line. It pulls only the finished virtual environment from the builder, keeping the image lean.
• The HEALTHCHECK hits your /health endpoint every 30 seconds. This lets Docker (and later ECS) know if the container is alive.
• The CMD starts uvicorn on port 8000. This is the default command when the container launches.

• Save Dockerfile.

Next, you need a .dockerignore file to keep unnecessary files out of the build context. Without it, Docker would copy your local .venv, .env, and .git folder into the image.

• Create a file called .dockerignore in the root of your project and paste the following:

.venv
__pycache__
*.pyc
.env
output/
.git
.gitignore
*.md
!knowledge/*.md
test_knowledge.py

What does this .dockerignore do?

It excludes files that are either large, sensitive, or irrelevant to the container. The line !knowledge/*.md is a negation. It re-includes the knowledge base markdown files even though *.md excludes all other markdown files.

• Save .dockerignore.
• Confirm both files exist by running:

ls Dockerfile .dockerignore

You should see both filenames listed in the output.

Don't see one of the files?

• Make sure you are in the root of your it-triage-crew folder (the same level as pyproject.toml).
• On macOS, files starting with a dot are hidden by default. The ls command above will still show them.
• help me find where my Dockerfile and .dockerignore should live in my project

✔️ Awesome, I've got everything!

Great. Make sure both files are saved before moving on to the build.

ⓧ I'd like to double check the full code

Your Dockerfile should contain two stages in this order:

• Builder stage (starts with FROM python:3.12-slim AS builder) containing the uv copy, env vars, and two RUN commands for dependency install then project install.
• Runtime stage (starts with FROM python:3.12-slim) containing the COPY --from=builder, knowledge copy, ENV PATH, EXPOSE, HEALTHCHECK, and CMD.

Scroll up and confirm each chunk matches. Your .dockerignore should list 10 lines of exclusions, ending with test_knowledge.py.

Build and run the container

With the Dockerfile ready, you can build the image and test it. Building for the first time downloads the base image and installs all dependencies, so it will take a minute or two.

• Build your Docker image by running:

docker build -t it-triage .

You should see Docker step through both stages, finishing with a message that names the image.

• Check the image size by running:

docker images it-triage

You should see a single image tagged latest at roughly 300MB. Without the multi-stage build, this would be over 2GB.

Build failed?

• If you see an error about uv.lock not found, make sure you have run uv lock in your project root before building.
• If Docker reports that BuildKit is not enabled, make sure Docker Desktop is updated to the latest version.
• help me debug my Docker build failure for the IT triage project

Now it is time to run the container. Before you think about passing your API key, consider what happens when the container starts with no environment variables at all.

What do you think will happen if you POST a ticket to the container without passing OPENROUTER_API_KEY?

• Run the container without any environment variables:

docker run -p 8000:8000 it-triage

• Open a second terminal and POST a test ticket:

curl -X POST http://localhost:8000/triage \
  -H "Content-Type: application/json" \
  -d '{"ticket": "My laptop screen is flickering whenever I open more than 3 browser tabs"}'

You get back a 500 error with the detail: OPENROUTER_API_KEY environment variable not set.

Why did this fail?

Containers are isolated environments. The OPENROUTER_API_KEY you exported in your host terminal does not automatically carry into the container. Runtime secrets must be passed explicitly using the -e flag.

This is the core lesson of container isolation. Your container has no access to your host environment unless you explicitly grant it.

• Stop the running container by pressing Ctrl+C in the first terminal.
• Run the container again, this time passing your API key:

docker run -e OPENROUTER_API_KEY=$OPENROUTER_API_KEY -p 8000:8000 it-triage

Do you think the same curl command will succeed now that the key is available inside the container?

• In your second terminal, run the same curl command again:

curl -X POST http://localhost:8000/triage \
  -H "Content-Type: application/json" \
  -d '{"ticket": "My laptop screen is flickering whenever I open more than 3 browser tabs"}'

This time you get back a structured JSON triage report with category, priority, assigned team, and a drafted response. Your containerized multi-agent system is working.

Still seeing an error?

• If you see a connection refused error, wait a few seconds for uvicorn to start inside the container. Check the first terminal for startup logs.
• If you see an authentication error from OpenRouter, confirm your key is still exported in your host shell by running echo $OPENROUTER_API_KEY. If it is empty, re-export it.
• If the response takes more than 60 seconds, the LLM call may be timing out. Check the container logs for error details.
• help me debug why my Docker container is not returning a triage report

• Stop the container by pressing Ctrl+C in the first terminal.

Your multi-agent triage system is now packaged into a portable Docker image that runs anywhere. Next up, you will push this image to Amazon ECR so ECS Fargate can pull and run it in the cloud.

Push the Image to Amazon ECR

Your Docker image is built and verified locally. Now it needs to live somewhere that ECS Fargate can pull it from when launching your container.

Amazon ECR is the native container registry for AWS. It integrates directly with ECS permissions, so your Fargate tasks can pull images without extra authentication configuration.

In this step, get ready to:

• Create an ECR repository for your Docker image.
• Authenticate Docker to your ECR registry.
• Tag and push the image, then verify it arrived.

Create the ECR repository and push your image

You need your 12-digit AWS account ID for the commands in this substep. You saw it when you ran aws sts get-caller-identity in Step 1.

• Grab your account ID by running this command:

aws sts get-caller-identity --query Account --output text

You should see your 12-digit account ID printed. Keep it handy for the next few commands.

• Create the ECR repository by running this command:

aws ecr create-repository --repository-name it-triage-crew

You should see JSON output including a repositoryUri field that looks like 123456789012.dkr.ecr.us-east-1.amazonaws.com/it-triage-crew.

What did this command do?

It created an empty container image repository in your AWS account. Think of it like creating an empty folder in a file system. The repositoryUri in the output is the address you will push images to and that ECS will pull images from.

Seeing a RepositoryAlreadyExistsException?

If you already created this repository in a previous attempt, the error is harmless. Your existing repository is ready to use. Continue to the next command.

ask

• Authenticate Docker to your ECR registry by running this command:

aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin [[AWS_ACCOUNT_ID="your 12-digit AWS account ID"]].dkr.ecr.us-east-1.amazonaws.com

You should see Login Succeeded printed in your terminal.

What does this command do?

It fetches a temporary authentication token from AWS and pipes it into docker login. This grants your local Docker client permission to push images to your private ECR registry. The token is valid for 12 hours.

You may also see a warning about storing the password unencrypted. This is normal for local development and can be safely ignored.

Login failed or access denied?

Make sure you replaced the account ID placeholder with your actual 12-digit number. Double-check that your AWS CLI credentials have ECR permissions (the default AdministratorAccess policy includes them).

help me fix my ECR login

• Tag your local image for ECR by running this command:

docker tag it-triage:latest [[AWS_ACCOUNT_ID="your 12-digit AWS account ID"]].dkr.ecr.us-east-1.amazonaws.com/it-triage-crew:staging

• Push the tagged image to ECR by running this command:

docker push [[AWS_ACCOUNT_ID="your 12-digit AWS account ID"]].dkr.ecr.us-east-1.amazonaws.com/it-triage-crew:staging

You should see each layer being pushed, followed by a digest and a staging: digest: sha256:... confirmation line. This takes a minute or two depending on your upload speed.

What happened with the tag and push?

Docker images can have multiple tags pointing to the same content. The docker tag command gave your local it-triage:latest image a second name that includes the full ECR registry address and the staging tag.

The docker push command then uploaded the image layers to that ECR address. Only layers that do not already exist in the registry are uploaded, which makes subsequent pushes much faster.

Push denied or connection timeout?

If you see a permission error, your Docker login token may have expired. Re-run the aws ecr get-login-password command from above, then retry the push.

If the push hangs or times out, check your internet connection. Large layers can take a few minutes on slower connections.

help me fix my docker push

Before you run the verification command, what do you expect to see? Think about what fields would confirm your image actually made it to the registry.

• Verify the image is stored in ECR by running this command:

aws ecr describe-images --repository-name it-triage-crew

You should see JSON output listing your image with the staging tag in the imageTag field. This confirms your image is stored in ECR and ready for ECS to pull.

Your image is safely stored in ECR. Next up, you will deploy it to ECS Fargate behind an Application Load Balancer and get a live URL that accepts triage requests.

Deploy to ECS Fargate with an Application Load Balancer

Your Docker image is sitting in ECR, ready to run. The final step is connecting it to the internet so anyone can POST a ticket and get a triage report back.

In this step, you will create the networking layer (security groups and an Application Load Balancer) and deploy your container on ECS Fargate. The ALB gives you a stable DNS name, health checks, and the ability to roll out new versions without downtime.

In this step, get ready to:

• Create an ECS cluster, security groups, and an Application Load Balancer.
• Register a task definition and launch the ECS service.
• POST a ticket to your live ALB endpoint and get a triage report back.

Create the ECS cluster, security groups, and ALB

Before you can run a container on Fargate, you need three things: a cluster to organize your services, security groups to control network access, and a load balancer to route traffic to your container.

• Look up your default VPC ID and subnet IDs by running these commands:

VPC_ID=$(aws ec2 describe-vpcs \
    --filters Name=is-default,Values=true \
    --query "Vpcs[0].VpcId" --output text)
echo $VPC_ID

• You should see a VPC ID like vpc-0abc123def456 printed. Keep this terminal window open so the variable persists.
• Get at least two subnet IDs from different Availability Zones (required for an ALB) by running:

SUBNETS=$(aws ec2 describe-subnets \
    --filters Name=vpc-id,Values=$VPC_ID \
    --query "Subnets[?DefaultForAz==\`true\`].SubnetId" \
    --output text)
echo $SUBNETS

• You should see two or more subnet IDs separated by spaces or tabs.

Why do you need two subnets?

Application Load Balancers require subnets from at least two Availability Zones for high availability. If one zone goes down, the other can still serve traffic.

• Create the ECS cluster by running:

aws ecs create-cluster --cluster-name it-triage-staging

• You should see JSON output with "clusterName": "it-triage-staging" and "status": "ACTIVE".

Seeing an authorization error?

Make sure your IAM user has the AmazonECS_FullAccess policy attached or equivalent permissions. You can check in the IAM console under your user's permissions tab.

Help me troubleshoot this

• Create a security group for the ALB that allows inbound HTTP traffic from anywhere:

ALB_SG=$(aws ec2 create-security-group \
    --group-name it-triage-alb-sg \
    --description "ALB security group for IT triage" \
    --vpc-id $VPC_ID \
    --query "GroupId" --output text)
echo $ALB_SG

• Authorize inbound port 80 from anywhere on the ALB security group:

aws ec2 authorize-security-group-ingress \
    --group-id $ALB_SG \
    --protocol tcp \
    --port 80 \
    --cidr 0.0.0.0/0

• You should see a confirmation with "Return": true in the output.
• Create a second security group for the Fargate tasks that only allows traffic from the ALB:

TASK_SG=$(aws ec2 create-security-group \
    --group-name it-triage-task-sg \
    --description "Fargate task security group" \
    --vpc-id $VPC_ID \
    --query "GroupId" --output text)
echo $TASK_SG

• Authorize inbound port 8000 from the ALB security group only:

aws ec2 authorize-security-group-ingress \
    --group-id $TASK_SG \
    --protocol tcp \
    --port 8000 \
    --source-group $ALB_SG

Why two separate security groups?

The ALB security group opens port 80 to the internet. The task security group only accepts traffic from the ALB on port 8000. This means your container is never directly reachable from the internet. All traffic must flow through the load balancer.

• Create the Application Load Balancer by running:

ALB_ARN=$(aws elbv2 create-load-balancer \
    --name it-triage-alb \
    --subnets $SUBNETS \
    --security-groups $ALB_SG \
    --query "LoadBalancers[0].LoadBalancerArn" \
    --output text)
echo $ALB_ARN

• You should see an ARN printed. The ALB takes a minute or two to provision.
• Create a target group that routes to IP addresses on port 8000 with a health check on /health:

TG_ARN=$(aws elbv2 create-target-group \
    --name it-triage-tg \
    --protocol HTTP \
    --port 8000 \
    --vpc-id $VPC_ID \
    --target-type ip \
    --health-check-path /health \
    --query "TargetGroups[0].TargetGroupArn" \
    --output text)
echo $TG_ARN

• Create a listener on port 80 that forwards traffic to your target group:

aws elbv2 create-listener \
    --load-balancer-arn $ALB_ARN \
    --protocol HTTP \
    --port 80 \
    --default-actions Type=forward,TargetGroupArn=$TG_ARN

• Grab the ALB DNS name so you can use it later for testing:

ALB_DNS=$(aws elbv2 describe-load-balancers \
    --load-balancer-arns $ALB_ARN \
    --query "LoadBalancers[0].DNSName" --output text)
echo $ALB_DNS

• You should see a DNS name like it-triage-alb-123456789.us-east-1.elb.amazonaws.com. Save this value for the final test.

Security group creation failed?

If you see a duplicate group name error, a security group with that name already exists in your VPC. Either delete the existing one or choose a different name.

If the $VPC_ID variable is empty, re-run the VPC lookup command. Make sure you are running all commands in the same terminal session.

Help me debug this

Register a task definition and create the ECS service

The cluster and networking are ready. Now you need to tell ECS exactly how to run your container: how much CPU and memory, which image to pull, and what environment variables to inject.

• Before creating the task definition, ensure the ecsTaskExecutionRole IAM role exists. This role lets ECS pull images from ECR and write logs to CloudWatch. Check if it exists by running:

aws iam get-role --role-name ecsTaskExecutionRole

• If you see role details returned, skip to the next chunk. If you get a NoSuchEntity error, create the role by running these commands:

aws iam create-role \
    --role-name ecsTaskExecutionRole \
    --assume-role-policy-document '{
      "Version": "2012-10-17",
      "Statement": [{
        "Effect": "Allow",
        "Principal": {"Service": "ecs-tasks.amazonaws.com"},
        "Action": "sts:AssumeRole"
      }]
    }'

• Attach the required managed policy to the role:

aws iam attach-role-policy \
    --role-name ecsTaskExecutionRole \
    --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

What does ecsTaskExecutionRole do?

This role gives the ECS agent (not your application) permission to pull container images from ECR and send logs to CloudWatch. Without it, your task cannot start because Fargate cannot access your private ECR image.

• Get your AWS account ID for the task definition by running:

ACCOUNT_ID=$(aws sts get-caller-identity --query "Account" --output text)
echo $ACCOUNT_ID

• Create the task definition file. In your it-triage-crew project folder, create task-definition.json with the following content. This first section defines the task-level settings:

{
  "family": "it-triage-staging",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "512",
  "memory": "1024",
  "executionRoleArn": "arn:aws:iam::ACCOUNT_ID:role/ecsTaskExecutionRole",
  "containerDefinitions": [{
    "name": "it-triage",
    "image": "ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/it-triage-crew:staging",
    "portMappings": [{"containerPort": 8000, "protocol": "tcp"}],

• Continue the file with the environment variables and logging configuration:

    "environment": [
      {"name": "OPENROUTER_API_KEY", "value": "your-openrouter-api-key-here"},
      {"name": "OPENROUTER_MODEL", "value": "openrouter/google/gemini-2.5-flash"}
    ],
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "/ecs/it-triage-staging",
        "awslogs-region": "us-east-1",
        "awslogs-stream-prefix": "ecs",
        "awslogs-create-group": "true"
      }
    },
    "essential": true
  }]
}

What does this task definition configure?

The cpu and memory fields give your container 0.5 vCPU and 1 GB of RAM. This is enough because the LLM inference happens on OpenRouter's servers, not inside your container.

The executionRoleArn grants Fargate permission to pull from ECR. The environment array injects your API key and model choice at runtime. The logConfiguration sends container stdout to CloudWatch so you can debug issues.

• Replace the two ACCOUNT_ID placeholders in the file with your actual account ID (the value printed by echo $ACCOUNT_ID earlier).
• Replace your-openrouter-api-key-here with your actual OpenRouter API key.
• Save task-definition.json.

Plain text secrets are OK for learning

Storing an API key in plain text in a task definition is acceptable for a learning project. In production, you would use AWS Secrets Manager with the secrets field in the container definition instead.

• Register the task definition with ECS by running:

aws ecs register-task-definition \
    --cli-input-json file://task-definition.json

• You should see JSON output with "family": "it-triage-staging" and "revision": 1.

Seeing a JSON parsing error?

Double-check that your task-definition.json is valid JSON. Common mistakes include trailing commas after the last item in an array or missing closing braces. You can validate the file by running python3 -m json.tool task-definition.json.

Help me find the syntax issue

• Create the ECS service, linking it to your cluster, task definition, target group, and security group:

aws ecs create-service \
    --cluster it-triage-staging \
    --service-name it-triage-service \
    --task-definition it-triage-staging \
    --desired-count 1 \
    --launch-type FARGATE \
    --network-configuration "awsvpcConfiguration={subnets=[$SUBNETS],securityGroups=[$TASK_SG],assignPublicIp=ENABLED}" \
    --load-balancers "targetGroupArn=$TG_ARN,containerName=it-triage,containerPort=8000"

What does assignPublicIp=ENABLED do?

Fargate tasks need a public IP to pull images from ECR and call the OpenRouter API. Without it, the container cannot reach the internet and will fail to start.

• Wait for the task to reach RUNNING state. Check the status by running:

aws ecs describe-services \
    --cluster it-triage-staging \
    --services it-triage-service \
    --query "services[0].deployments[0].runningCount"

• Re-run this command every 30 seconds until it returns 1. This usually takes 1-2 minutes while Fargate pulls the image and starts the container.

Task stays at 0 running?

Check the task's stopped reason by running:

aws ecs describe-tasks \
    --cluster it-triage-staging \
    --tasks $(aws ecs list-tasks --cluster it-triage-staging --query "taskArns[0]" --output text) \
    --query "tasks[0].stoppedReason"

Common causes: the ecsTaskExecutionRole is missing the ECR pull policy, the image URI has a typo, or the security group blocks outbound traffic (Fargate needs outbound access to pull images).

Help me debug the stopped task

Before you test, think about this: you are about to POST a ticket to a DNS name on the internet that routes through an ALB, into a Fargate container running your multi-agent CrewAI system. Do you expect it to work on the first try?

• POST a test ticket to your live ALB endpoint by running:

curl -X POST http://$ALB_DNS/triage \
    -H "Content-Type: application/json" \
    -d '{"ticket": "My laptop screen is flickering and sometimes goes black. I cannot present in meetings."}'

After 30-60 seconds (the agents need time to deliberate), you should see a JSON response with a triage_report containing category, priority, assigned team, SLA times, and a drafted response.

What should I see?

A successful response looks like a JSON object with fields like category, priority, assigned_team, sla_response_time, sla_resolution_time, drafted_response, and policy_references. The exact values depend on how your agents interpret the ticket.

Getting a 502 Bad Gateway or timeout?

A 502 usually means the target group health check is failing. Verify your container's /health endpoint works by checking CloudWatch logs. The CrewAI agents take 30-60 seconds to respond, but the health check is instant (it does not call any LLM).

If you get a timeout after 60+ seconds, the ALB has a default idle timeout of 60 seconds. The multi-agent system may need longer. You can increase it with:

aws elbv2 modify-load-balancer-attributes \
    --load-balancer-arn $ALB_ARN \
    --attributes Key=idle_timeout.timeout_seconds,Value=120

Help me debug the 502 or timeout

Your multi-agent IT triage system is live on AWS. Anyone with the ALB URL can now POST a ticket and get back a structured triage report. Next up, you will take this further in the Secret Mission by deploying a separate production environment from the same image.

Secret mission

Deploy a Production Environment

Real teams run separate staging and production environments from the same container image. The only difference is configuration. Can you deploy a production environment that uses a stronger model for higher-quality triage?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. Each running Fargate task plus its ALB costs approximately $0.90/day, so two environments (staging + production) cost around $1.80/day.

Cost warning

Your ECS Fargate tasks and Application Load Balancers incur charges as long as they exist. With one staging environment, expect approximately $0.90/day. If you completed the Secret Mission and have both staging and production running, expect approximately $1.80/day.

If you are not actively using these resources, choose Pause or Delete below to avoid unexpected charges.

Resources you used:

• ECS Cluster it-triage-staging (and it-triage-prod if you completed the Secret Mission).
• ECS Services with running Fargate tasks (one per cluster).
• Application Load Balancers with target groups (one per environment).
• ECR Repository it-triage-crew with pushed images.
• Security groups (ALB and Fargate task groups for each environment).
• CloudWatch Log Groups (/ecs/it-triage-staging and /ecs/it-triage-prod if Secret Mission completed).

Keep everything running

No action needed. Choose this if you are still actively building or testing your triage API.

• Your staging (and optionally production) endpoints remain live and ready to receive triage requests.
• Costs accrue at approximately $0.90/day per environment while services are running.
• Your OpenRouter API key remains exported in your terminal session. It will be cleared when you close the terminal window.

Pause - I'll come back to this later

Shut down the running Fargate tasks to stop compute charges while keeping your infrastructure in place.

• Set the staging ECS service desired count to 0 by running:

aws ecs update-service --cluster it-triage-staging --service it-triage-service --desired-count 0

• If you completed the Secret Mission, repeat for production:

aws ecs update-service --cluster it-triage-prod --service it-triage-service --desired-count 0

ALB charges continue

Setting desired count to 0 stops Fargate task charges, but your ALBs still cost approximately $0.60/day each. To fully pause costs, delete the ALBs as well (see the Delete tab for those commands).

• To resume later, set desired count back to 1:

aws ecs update-service --cluster it-triage-staging --service it-triage-service --desired-count 1

Delete - I don't want to use this again

Remove all project resources from AWS. Run each group of commands in order for the staging environment.

• Delete the ECS service and cluster:

aws ecs delete-service --cluster it-triage-staging --service it-triage-service --force
aws ecs delete-cluster --cluster it-triage-staging

• Delete the Application Load Balancer (this also removes its listeners):

aws elbv2 delete-load-balancer --load-balancer-arn [[STAGING_ALB_ARN="your staging ALB ARN"]]

• Delete the target group:

aws elbv2 delete-target-group --target-group-arn [[STAGING_TG_ARN="your staging target group ARN"]]

• Delete the security groups (ALB group first, then Fargate tasks group):

aws ec2 delete-security-group --group-id [[STAGING_ALB_SG="your staging ALB security group ID"]]
aws ec2 delete-security-group --group-id [[STAGING_TASK_SG="your staging task security group ID"]]

• Delete the ECR repository and all images:

aws ecr delete-repository --repository-name it-triage-crew --force

• Delete the CloudWatch Log Group:

aws logs delete-log-group --log-group-name /ecs/it-triage-staging

Secret Mission resources

If you completed the Secret Mission, repeat the ECS service, cluster, ALB, target group, security group, and log group deletions for your production environment. Replace it-triage-staging with it-triage-prod in the cluster and log group commands, and use your production ARNs and security group IDs.

• Delete the production log group (if Secret Mission completed):

aws logs delete-log-group --log-group-name /ecs/it-triage-prod

That's a Wrap!

That's a Wrap!

Nice work. You just took a local AI prototype and shipped it as a production API on AWS.

You've learned how to:

• Adapt a local CLI-based CrewAI multi-agent system into a production HTTP API using FastAPI, and swap a local LLM for an API-based provider (OpenRouter) so the container runs anywhere without a GPU.
• Build a lean multi-stage Docker image using uv that caches dependencies and rebuilds in seconds when you change code.
• Push the image to Amazon ECR and deploy it to ECS Fargate behind an Application Load Balancer, passing runtime secrets via environment variables.
• Secret Mission: Deployed separate staging and production environments from the same image with different model configurations.

Ready to quiz yourself?