Python Microservice on Kubernetes with Terraform

Deploy a monitored Python microservice on Kubernetes using Terraform and a local AWS emulator.

Introduction

30 Second Summary

Every cloud bill starts with a developer testing something in a real AWS account. What if you could build, deploy, and monitor production-grade infrastructure without spending a single dollar?

In this project, you will build a Python microservice backed by AWS resources, deploy it to a local Kubernetes cluster, and monitor it with Prometheus and Grafana. All of the cloud services run locally using Floci, a free AWS emulator, so your AWS bill stays at zero.

What You'll Build

A Python REST API that stores and retrieves data from DynamoDB and S3, provisioned via Terraform against a local AWS emulator, deployed to Kubernetes with full observability.

By the end of this project, you'll have:

  • A Terraform configuration that provisions an S3 bucket and DynamoDB table against Floci's local AWS emulator.
  • A Dockerized Python microservice with REST endpoints and a Prometheus metrics endpoint exposing request rate, errors, and latency.
  • A Kubernetes deployment with a Prometheus and Grafana monitoring stack showing live metrics on a dashboard.
  • Secret Mission: an SQS-triggered event processor that writes to DynamoDB, instrumented with custom Prometheus metrics.

Are there any prerequisites?

This project assumes you already have Docker Desktop with Kubernetes enabled, Terraform, Git, Python, and VS Code installed. You do not need an active AWS account or any paid subscriptions.

Set Up Floci and Dependencies

Before you can deploy anything, you need the tools that make local cloud development possible. Floci is a local AWS emulator that runs real S3, DynamoDB, and SQS services on your machine without touching a cloud account.

In this step, you'll install Floci alongside the AWS CLI and Helm, then confirm everything connects to your local Kubernetes cluster.

In this step, get ready to:
  • Install Floci CLI, AWS CLI, and Helm.
  • Start Floci and verify AWS services respond locally.
  • Confirm the local Kubernetes cluster is ready for deployments.
Install Floci CLI and AWS CLI

Floci replaces cloud-hosted AWS services with a lightweight local emulator. You'll install it via Scoop, a Windows package manager that keeps CLI tools out of system directories.

  • In VS Code, open a PowerShell terminal by pressing Ctrl+`.
  • Add the Floci Scoop bucket and install the CLI by running these commands:
scoop bucket add floci https://github.com/floci-io/scoop-floci
scoop install floci

What does this do?

The first command registers Floci's package repository (called a "bucket") with Scoop. The second command downloads and installs the Floci CLI binary to your user directory.

  • Verify the Floci CLI installed correctly by running:
floci version

✔️ I see a version number

Floci is installed and ready. Move on to the AWS CLI check below.

ⓧ Command not found

Scoop may not be installed on your machine. Install Scoop first by running these commands in PowerShell:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
irm get.scoop.sh | iex

Close and reopen your PowerShell terminal, then re-run the Floci installation commands above.

  • Check whether the AWS CLI is already installed by running:
aws --version

✔️ I see a version number

AWS CLI is ready. Skip ahead to the Helm check below.

ⓧ Command not found

You need to install the AWS CLI. Download and run the official MSI installer by running:

msiexec.exe /i https://awscli.amazonaws.com/AWSCLIV2.msi

Follow the installer prompts to completion. After it finishes, close and reopen your PowerShell terminal, then run aws --version again to confirm.

  • Check whether Helm is already installed by running:
helm version

✔️ I see a version number

Helm is ready. Move on to the next substep.

ⓧ Command not found

Install Helm via Scoop by running:

scoop install helm

After installation completes, run helm version again to confirm it returns a version number.

Why Floci over LocalStack?

LocalStack now requires an authentication token. Floci is MIT-licensed, free forever, and requires no auth. It starts in milliseconds, uses minimal memory, and runs on the same port (4566) as LocalStack.

This means any tutorial or Terraform config written for LocalStack works with Floci as a drop-in replacement.

Start Floci and verify AWS services

With the CLI tools installed, you can now start the Floci emulator. This launches a lightweight container that serves S3, DynamoDB, and SQS on port 4566.

  • Start the Floci AWS emulator by running:
floci start

What does this command do?

This pulls and starts the Floci Docker container. It emulates AWS services locally so your Terraform and application code can interact with S3, DynamoDB, and SQS without network calls to real AWS.

  • Wait for Floci to be fully ready by running:
floci wait --timeout 60s

You should see a success message indicating Floci is ready to accept connections.

  • Configure your shell's environment variables to point the AWS CLI at Floci by running:
floci env --shell powershell | Invoke-Expression

What does this command do?

This sets AWS_ENDPOINT_URL, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_DEFAULT_REGION in your current session. Every aws command you run will now hit Floci instead of real AWS.

  • Verify Floci is responding by creating a test S3 bucket:
aws s3 mb s3://test-bucket
  • Confirm the bucket was created by listing all S3 buckets:
aws s3 ls

You should see test-bucket in the output. This proves your AWS CLI is communicating with Floci successfully.

Not seeing your bucket?

Make sure floci start completed without errors. Docker Desktop must be running for Floci to launch its container.

If you see a credentials error, re-run floci env --shell powershell | Invoke-Expression to refresh your environment variables.

Help me troubleshoot why my AWS CLI can't connect to Floci on port 4566.

Verify Kubernetes cluster and Helm

Your microservice will run inside Kubernetes. Docker Desktop includes a single-node cluster that is perfect for local development.

  • Confirm your Kubernetes cluster is running by checking the cluster info:
kubectl cluster-info

You should see output showing the Kubernetes control plane address (something like https://127.0.0.1:6443).

  • Verify at least one node is ready by running:
kubectl get nodes

You should see one node with status Ready.

Kubernetes not responding?

Open Docker Desktop and go to Settings then Kubernetes. Make sure the Enable Kubernetes checkbox is checked. Click Apply & Restart if you made changes. Wait 1-2 minutes for the cluster to start.

Help me get Docker Desktop's Kubernetes cluster running on Windows.

  • Confirm Helm can communicate with the cluster by running:
helm version

You should see client version information printed. Helm connects to whichever cluster kubectl points at, so if kubectl cluster-info worked, Helm is ready too.

Your local development stack is fully wired up. Next, you'll use Terraform to provision real S3 and DynamoDB resources against Floci.

Provision Cloud Resources with Terraform

Your local AWS emulator is running and ready to accept requests. Now you need the cloud resources your microservice will depend on.

In production, teams use Terraform to define cloud infrastructure in code files rather than clicking through a web console. This approach is called Infrastructure as Code. You will write a Terraform configuration that provisions an S3 bucket and a DynamoDB table against Floci, giving you the exact same workflow as a real cloud deployment.

In this step, get ready to:
  • Write a Terraform configuration pointing at Floci's local endpoint.
  • Initialize Terraform and apply the configuration to create resources.
  • Verify the S3 bucket and DynamoDB table exist using the AWS CLI.
Write the Terraform configuration

Terraform reads .tf files from a directory to determine what resources to create. You will create a dedicated terraform folder and write a single main.tf file that defines the provider and resources.

  • Create the terraform directory inside your project folder by running this command:
mkdir terraform
  • Create the main.tf file inside the terraform folder by running:
New-Item terraform/main.tf
  • Confirm both exist by running:
ls terraform

You should see main.tf listed in the output.

  • Open terraform/main.tf in VS Code by running:
code terraform/main.tf

The provider configuration tells Terraform where to send API calls. Instead of real AWS, you will point every service endpoint at Floci on localhost:4566.

  • Paste the following provider configuration into terraform/main.tf:
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.98"
    }
  }
}

provider "aws" {
  access_key                  = "test"
  secret_key                  = "test"
  region                      = "us-east-1"
  s3_use_path_style           = true
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true

  endpoints {
    s3       = "http://localhost:4566"
    dynamodb = "http://localhost:4566"
    sqs      = "http://localhost:4566"
    sts      = "http://localhost:4566"
  }
}

What does this configuration do?

The terraform block pins the AWS provider to version ~> 5.98, ensuring consistent behavior across machines.

The provider "aws" block uses mock credentials (test/test) because Floci does not require real AWS keys. The skip_* flags prevent Terraform from calling real AWS metadata endpoints. Every service endpoint points to http://localhost:4566 where Floci is listening.

Now you will define the actual cloud resources your microservice needs. The S3 bucket stores uploaded files and the DynamoDB table stores item records.

  • Add the following resource definitions below the provider block in terraform/main.tf:
resource "aws_s3_bucket" "files" {
  bucket = "microservice-files"
}

resource "aws_dynamodb_table" "items" {
  name         = "items"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "id"

  attribute {
    name = "id"
    type = "S"
  }
}

output "s3_bucket_name" {
  value = aws_s3_bucket.files.bucket
}

output "dynamodb_table_name" {
  value = aws_dynamodb_table.items.name
}

What does this code do?

The aws_s3_bucket resource creates a bucket named microservice-files for storing uploaded files.

The aws_dynamodb_table resource creates a table named items with on-demand billing (PAY_PER_REQUEST) and a string partition key called id. The output blocks print the resource names after Terraform finishes creating them.

  • Save terraform/main.tf.

✔️ Awesome, I've got everything!

Great. Double check you have saved terraform/main.tf before continuing.

ⓧ I'd like to double check the full code

Here is the complete terraform/main.tf file:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.98"
    }
  }
}

provider "aws" {
  access_key                  = "test"
  secret_key                  = "test"
  region                      = "us-east-1"
  s3_use_path_style           = true
  skip_credentials_validation = true
  skip_metadata_api_check     = true
  skip_requesting_account_id  = true

  endpoints {
    s3       = "http://localhost:4566"
    dynamodb = "http://localhost:4566"
    sqs      = "http://localhost:4566"
    sts      = "http://localhost:4566"
  }
}

resource "aws_s3_bucket" "files" {
  bucket = "microservice-files"
}

resource "aws_dynamodb_table" "items" {
  name         = "items"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "id"

  attribute {
    name = "id"
    type = "S"
  }
}

output "s3_bucket_name" {
  value = aws_s3_bucket.files.bucket
}

output "dynamodb_table_name" {
  value = aws_dynamodb_table.items.name
}
Initialize and apply Terraform

Terraform needs to download the AWS provider plugin before it can create resources. The terraform init command handles this setup step.

  • Move into the terraform directory by running:
cd terraform
  • Initialize Terraform by running:
terraform init

You should see output ending with Terraform has been successfully initialized! confirming the AWS provider was downloaded.

Seeing a provider download error?

Make sure you have an internet connection. Terraform downloads the AWS provider from the HashiCorp registry. If you are behind a corporate proxy, configure HTTPS_PROXY in your terminal.

Also verify you are inside the terraform directory (not the project root). Run ls and confirm you see main.tf.

help me fix a Terraform init error

Next, preview what Terraform plans to create. This is a dry run that shows you exactly which resources will be provisioned without making any changes.

  • Preview the planned changes by running:
terraform plan

You should see a plan listing 2 resources to add: the S3 bucket and the DynamoDB table.

Now apply the configuration to actually create the resources against Floci.

  • Apply the Terraform configuration by running:
terraform apply

Terraform will ask for confirmation

Terraform displays the plan and prompts you with "Do you want to perform these actions?" before making changes. Type yes and press Enter to confirm.

You should see output ending with Apply complete! Resources: 2 added, 0 changed, 0 destroyed. along with the output values for the bucket name and table name.

Seeing a connection refused error?

This means Floci is not running on port 4566. Switch back to your other terminal and run floci start followed by floci wait --timeout 60s to restart it.

help me fix a Terraform apply connection error

Verify the provisioned resources

Terraform reported success, but it is good practice to independently confirm resources exist using the AWS CLI.

  • List S3 buckets to confirm the bucket was created by running:
aws s3 ls

You should see microservice-files in the output.

  • Describe the DynamoDB table to confirm it is active by running:
aws dynamodb describe-table --table-name items

Look for "TableStatus": "ACTIVE" in the JSON output. This confirms the table is ready to accept read and write operations.

Not seeing the bucket or table?

Make sure your AWS CLI environment variables still point at Floci. Run floci env --shell powershell | Invoke-Expression to re-export them if your terminal session was restarted.

help me verify my Terraform resources against Floci

Your cloud infrastructure is provisioned and verified. Next up, you will build a Python microservice that reads and writes to these resources.

Build the Python Microservice

Your cloud infrastructure is provisioned and ready. Now you need an application that actually uses those resources.

In this step, you will build a Python REST API that stores data in your DynamoDB table and uploads files to your S3 bucket. The app connects to Floci's local endpoint, so every request hits the same emulated AWS services that Terraform provisioned.

In this step, get ready to:
  • Write a Flask API with REST endpoints backed by DynamoDB and S3.
  • Add Prometheus metrics instrumentation to the API.
  • Containerize the microservice with Docker.
Write the Flask API with boto3

Flask is a lightweight Python web framework that makes it easy to define REST endpoints. Combined with boto3 (the AWS SDK for Python), your app can interact with DynamoDB and S3 using the same code you would write for production AWS.

  • Create the app directory inside your project folder by running this command:
mkdir app
  • Create the app/requirements.txt file with these pinned dependencies:
flask==3.1.1
boto3==1.38.29
prometheus-client==0.22.1

What does this file do?

A requirements.txt file lists the exact library versions your app depends on. Pinning versions ensures every environment runs the same code.

  • flask==3.1.1 is the web framework for your REST endpoints.
  • boto3==1.38.29 is the AWS SDK that talks to DynamoDB and S3.
  • prometheus-client==0.22.1 exposes metrics for monitoring.
  • Save app/requirements.txt.

Now create the main application file. This is where your Flask app, boto3 clients, and route handlers live.

  • Create app/app.py and add the following setup code at the top:
import os
import uuid
import time
import json

import boto3
from flask import Flask, request, jsonify
from prometheus_client import Counter, Histogram, generate_latest, CONTENT_TYPE_LATEST

app = Flask(__name__)

REQUEST_COUNT = Counter(
    "http_requests_total",
    "Total HTTP requests",
    ["method", "endpoint", "status"]
)
REQUEST_LATENCY = Histogram(
    "http_request_duration_seconds",
    "HTTP request latency in seconds",
    ["method", "endpoint"]
)
ERROR_COUNT = Counter(
    "http_errors_total",
    "Total HTTP errors",
    ["method", "endpoint"]
)

ENDPOINT_URL = os.environ.get("AWS_ENDPOINT_URL", "http://localhost:4566")
REGION = os.environ.get("AWS_DEFAULT_REGION", "us-east-1")

dynamodb = boto3.resource(
    "dynamodb",
    endpoint_url=ENDPOINT_URL,
    region_name=REGION,
    aws_access_key_id="test",
    aws_secret_access_key="test",
)
s3 = boto3.client(
    "s3",
    endpoint_url=ENDPOINT_URL,
    region_name=REGION,
    aws_access_key_id="test",
    aws_secret_access_key="test",
)

table = dynamodb.Table("items")

What does this code do?

This sets up the foundation of your microservice.

  • ENDPOINT_URL reads the AWS_ENDPOINT_URL environment variable, defaulting to Floci at http://localhost:4566. This means the same code works against Floci locally or real AWS in production.
  • The dynamodb resource and s3 client both point to Floci's endpoint with mock credentials.
  • The three Prometheus metrics (REQUEST_COUNT, REQUEST_LATENCY, ERROR_COUNT) are defined here and used by every endpoint to track requests.

Next, add the REST endpoints that handle incoming requests.

  • Below the table = dynamodb.Table("items") line, add the health check and item creation endpoints:


@app.route("/health")
def health():
    return jsonify({"status": "healthy"})


@app.route("/items", methods=["POST"])
def create_item():
    start = time.time()
    try:
        data = request.get_json()
        item_id = str(uuid.uuid4())
        item = {"id": item_id, "name": data.get("name", ""), "value": data.get("value", "")}
        table.put_item(Item=item)
        REQUEST_COUNT.labels(method="POST", endpoint="/items", status="200").inc()
        return jsonify(item), 201
    except Exception as e:
        ERROR_COUNT.labels(method="POST", endpoint="/items").inc()
        REQUEST_COUNT.labels(method="POST", endpoint="/items", status="500").inc()
        return jsonify({"error": str(e)}), 500
    finally:
        REQUEST_LATENCY.labels(method="POST", endpoint="/items").observe(time.time() - start)

What does this code do?

  • /health returns a simple JSON response. Kubernetes uses this later to check if your container is alive.
  • /items POST generates a UUID, saves a new item to DynamoDB, and tracks the request with Prometheus counters. The finally block ensures latency is always recorded, even on errors.
  • Below the create_item function, add the list items and file upload endpoints:


@app.route("/items", methods=["GET"])
def list_items():
    start = time.time()
    try:
        response = table.scan()
        items = response.get("Items", [])
        REQUEST_COUNT.labels(method="GET", endpoint="/items", status="200").inc()
        return jsonify(items), 200
    except Exception as e:
        ERROR_COUNT.labels(method="GET", endpoint="/items").inc()
        REQUEST_COUNT.labels(method="GET", endpoint="/items", status="500").inc()
        return jsonify({"error": str(e)}), 500
    finally:
        REQUEST_LATENCY.labels(method="GET", endpoint="/items").observe(time.time() - start)


@app.route("/files/<filename>", methods=["PUT"])
def upload_file(filename):
    start = time.time()
    try:
        content = request.get_data()
        s3.put_object(Bucket="microservice-files", Key=filename, Body=content)
        REQUEST_COUNT.labels(method="PUT", endpoint="/files", status="200").inc()
        return jsonify({"message": f"Uploaded {filename}"}), 200
    except Exception as e:
        ERROR_COUNT.labels(method="PUT", endpoint="/files").inc()
        REQUEST_COUNT.labels(method="PUT", endpoint="/files", status="500").inc()
        return jsonify({"error": str(e)}), 500
    finally:
        REQUEST_LATENCY.labels(method="PUT", endpoint="/files").observe(time.time() - start)

What does this code do?

  • /items GET scans the DynamoDB table and returns all stored items as JSON.
  • /files/<filename> PUT takes raw request data and uploads it to the microservice-files S3 bucket. This is the same bucket Terraform provisioned in the previous step.
  • Both endpoints follow the same pattern: start a timer, try the operation, increment counters, and always record latency in finally.
  • Save app/app.py. You will add the final endpoint in the next substep.

Seeing red squiggles in VS Code?

VS Code may show import errors because the dependencies are not installed in your local Python environment yet. This is expected. The dependencies will be installed inside the Docker container in Substep 3.

help me understand why VS Code shows import errors for flask and boto3

Add Prometheus metrics instrumentation

Your endpoints already record metrics using the counters and histogram defined at the top of the file. The missing piece is a /metrics endpoint that Prometheus can scrape to collect those numbers.

What are RED metrics?

SRE teams monitor three signals for every service: Rate (requests per second), Errors (failed requests), and Duration (how long requests take). These are called RED metrics.

Your REQUEST_COUNT tracks Rate, ERROR_COUNT tracks Errors, and REQUEST_LATENCY tracks Duration. Together they give a complete picture of service health.

  • At the bottom of app/app.py, below the upload_file function, add the metrics endpoint and the main entry point:


@app.route("/metrics")
def metrics():
    return generate_latest(), 200, {"Content-Type": CONTENT_TYPE_LATEST}


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

What does this code do?

  • /metrics calls generate_latest() which serializes all your counters and histograms into the text format that Prometheus expects.
  • The if __name__ == "__main__" block starts the Flask server on port 8080, binding to all interfaces so it is reachable from outside the container.
  • Save app/app.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved both app/requirements.txt and app/app.py.

ⓧ I'd like to double check the full code

import os
import uuid
import time
import json

import boto3
from flask import Flask, request, jsonify
from prometheus_client import Counter, Histogram, generate_latest, CONTENT_TYPE_LATEST

app = Flask(__name__)

REQUEST_COUNT = Counter(
    "http_requests_total",
    "Total HTTP requests",
    ["method", "endpoint", "status"]
)
REQUEST_LATENCY = Histogram(
    "http_request_duration_seconds",
    "HTTP request latency in seconds",
    ["method", "endpoint"]
)
ERROR_COUNT = Counter(
    "http_errors_total",
    "Total HTTP errors",
    ["method", "endpoint"]
)

ENDPOINT_URL = os.environ.get("AWS_ENDPOINT_URL", "http://localhost:4566")
REGION = os.environ.get("AWS_DEFAULT_REGION", "us-east-1")

dynamodb = boto3.resource(
    "dynamodb",
    endpoint_url=ENDPOINT_URL,
    region_name=REGION,
    aws_access_key_id="test",
    aws_secret_access_key="test",
)
s3 = boto3.client(
    "s3",
    endpoint_url=ENDPOINT_URL,
    region_name=REGION,
    aws_access_key_id="test",
    aws_secret_access_key="test",
)

table = dynamodb.Table("items")


@app.route("/health")
def health():
    return jsonify({"status": "healthy"})


@app.route("/items", methods=["POST"])
def create_item():
    start = time.time()
    try:
        data = request.get_json()
        item_id = str(uuid.uuid4())
        item = {"id": item_id, "name": data.get("name", ""), "value": data.get("value", "")}
        table.put_item(Item=item)
        REQUEST_COUNT.labels(method="POST", endpoint="/items", status="200").inc()
        return jsonify(item), 201
    except Exception as e:
        ERROR_COUNT.labels(method="POST", endpoint="/items").inc()
        REQUEST_COUNT.labels(method="POST", endpoint="/items", status="500").inc()
        return jsonify({"error": str(e)}), 500
    finally:
        REQUEST_LATENCY.labels(method="POST", endpoint="/items").observe(time.time() - start)


@app.route("/items", methods=["GET"])
def list_items():
    start = time.time()
    try:
        response = table.scan()
        items = response.get("Items", [])
        REQUEST_COUNT.labels(method="GET", endpoint="/items", status="200").inc()
        return jsonify(items), 200
    except Exception as e:
        ERROR_COUNT.labels(method="GET", endpoint="/items").inc()
        REQUEST_COUNT.labels(method="GET", endpoint="/items", status="500").inc()
        return jsonify({"error": str(e)}), 500
    finally:
        REQUEST_LATENCY.labels(method="GET", endpoint="/items").observe(time.time() - start)


@app.route("/files/<filename>", methods=["PUT"])
def upload_file(filename):
    start = time.time()
    try:
        content = request.get_data()
        s3.put_object(Bucket="microservice-files", Key=filename, Body=content)
        REQUEST_COUNT.labels(method="PUT", endpoint="/files", status="200").inc()
        return jsonify({"message": f"Uploaded {filename}"}), 200
    except Exception as e:
        ERROR_COUNT.labels(method="PUT", endpoint="/files").inc()
        REQUEST_COUNT.labels(method="PUT", endpoint="/files", status="500").inc()
        return jsonify({"error": str(e)}), 500
    finally:
        REQUEST_LATENCY.labels(method="PUT", endpoint="/files").observe(time.time() - start)


@app.route("/metrics")
def metrics():
    return generate_latest(), 200, {"Content-Type": CONTENT_TYPE_LATEST}


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)
Containerize with Docker

A Docker container packages your app and its dependencies into a single portable unit. This ensures it runs the same way on your machine, in Kubernetes, or in production.

  • Create a Dockerfile in your project root (the same level as the app/ and terraform/ folders) with the following multi-stage build:
FROM python:3.12-slim AS builder
WORKDIR /app
COPY app/requirements.txt .
RUN pip install --no-cache-dir --prefix=/install -r requirements.txt

FROM python:3.12-slim
WORKDIR /app
COPY --from=builder /install /usr/local
COPY app/app.py .
EXPOSE 8080
USER 1000
CMD ["python", "app.py"]

What does this Dockerfile do?

A multi-stage build uses one image to install dependencies and a second clean image to run the app. This keeps the final image small.

  • The builder stage installs Python packages into /install without polluting the final image with build tools.
  • The runtime stage copies only the installed packages and your app code.
  • USER 1000 runs the process as a non-root user for security.
  • Save the Dockerfile.
  • Build the Docker image by running this command:
docker build -t floci-microservice:latest .
  • You should see output ending with a line showing Successfully tagged floci-microservice:latest.

Build failing with file not found?

Make sure you are running the docker build command from your project root directory (the folder that contains both the Dockerfile and the app/ folder).

help me fix my Docker build error for the floci-microservice

Now test that the container runs and responds to requests.

  • Start the container with Floci's endpoint by running:
docker run -d -p 8080:8080 -e "AWS_ENDPOINT_URL=http://host.docker.internal:4566" --name floci-test floci-microservice:latest

Why host.docker.internal?

Your container runs in an isolated network. It cannot reach localhost:4566 because that refers to the container itself. host.docker.internal is a special DNS name that Docker Desktop provides. It resolves to your host machine where Floci is running.

  • Test the health endpoint by running:
curl http://localhost:8080/health
  • You should see {"status":"healthy"} in the response.
  • Test the metrics endpoint by running:
curl http://localhost:8080/metrics
  • You should see Prometheus-formatted text output with metric names like http_requests_total and http_request_duration_seconds.
  • Stop and remove the test container by running:
docker stop floci-test
docker rm floci-test

Getting connection refused on port 8080?

Check that the container is actually running with docker ps. If it exited immediately, check the logs with docker logs floci-test. A common cause is Floci not being reachable from the container.

help me debug why my floci-microservice container is not responding

Your microservice is containerized and proven to work. Next up, you will deploy it to Kubernetes and make it a real running service on your cluster.

Deploy to Kubernetes

Your microservice is containerized and ready to run. The next challenge is deploying it to a Kubernetes cluster so it behaves the way it would in a production environment.

Kubernetes manages your containers as Pods, automatically restarts them if they crash, and distributes traffic across replicas. In this step, you will write the manifest files that tell Kubernetes exactly how to run your microservice, then deploy and verify everything is working.

In this step, get ready to:
  • Write Deployment, Service, and ServiceMonitor manifests for your microservice.
  • Deploy the application to your local Kubernetes cluster.
  • Port-forward and confirm the service responds on localhost.
Write Kubernetes manifests

Kubernetes reads YAML files called manifests to understand what resources to create. You need three manifests: a Deployment (how many copies of your container to run and how to configure them), a Service (how other things inside the cluster reach your app), and a ServiceMonitor (how Prometheus discovers and scrapes your metrics endpoint).

  • Create a k8s directory inside your project folder by running this command:
mkdir k8s
  • Confirm the directory exists by running:
ls

You should see k8s listed alongside your app, terraform, and Dockerfile entries.

  • Create the Deployment manifest by pasting the following into a new file at k8s/deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: floci-microservice
  labels:
    app: floci-microservice
spec:
  replicas: 2
  selector:
    matchLabels:
      app: floci-microservice
  template:
    metadata:
      labels:
        app: floci-microservice
    spec:
      containers:
        - name: microservice
          image: floci-microservice:latest
          imagePullPolicy: Never
          ports:
            - containerPort: 8080
          env:
            - name: AWS_ENDPOINT_URL
              value: "http://host.docker.internal:4566"
            - name: AWS_DEFAULT_REGION
              value: "us-east-1"
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 3
            periodSeconds: 5

What does this manifest do?

This Deployment tells Kubernetes to run 2 replicas of your container image. Each replica gets environment variables pointing at Floci via host.docker.internal:4566. That DNS name resolves to your host machine from inside Docker Desktop containers.

The imagePullPolicy: Never setting tells Kubernetes to use the local Docker image you already built instead of pulling from a remote registry. The liveness and readiness probes hit the /health endpoint so Kubernetes can automatically restart unhealthy pods and only route traffic to pods that are ready.

  • Save k8s/deployment.yaml.

Seeing YAML syntax errors?

YAML is whitespace-sensitive. Make sure you are using spaces (not tabs) and that each indentation level is exactly 2 spaces. Check that the env block is indented under containers, not at the same level.

help me debug a YAML indentation error in my Kubernetes Deployment manifest

  • Create the Service manifest by pasting the following into a new file at k8s/service.yaml:
apiVersion: v1
kind: Service
metadata:
  name: floci-microservice
  labels:
    app: floci-microservice
spec:
  selector:
    app: floci-microservice
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
  type: ClusterIP

What does the Service do?

A Service gives your pods a stable internal address. Other resources in the cluster reach your microservice on port 80, and the Service forwards that traffic to port 8080 on the container. ClusterIP means the Service is only accessible from inside the cluster (you will use port-forwarding to access it from your machine).

  • Save k8s/service.yaml.
  • Create the ServiceMonitor manifest by pasting the following into a new file at k8s/servicemonitor.yaml:
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: floci-microservice
  labels:
    release: monitoring
spec:
  selector:
    matchLabels:
      app: floci-microservice
  endpoints:
    - port: "80"
      path: /metrics
      interval: 15s

What is a ServiceMonitor?

A ServiceMonitor is a custom resource from the Prometheus Operator. It tells Prometheus which Services to scrape and where to find their metrics. The release: monitoring label matches the Helm release name you will use in the next step, so Prometheus automatically discovers this monitor.

The interval: 15s setting means Prometheus collects metrics from your /metrics endpoint every 15 seconds.

  • Save k8s/servicemonitor.yaml.

Not sure where to create these files?

All three files live inside the k8s folder you just created. In VS Code, right-click the k8s folder in the Explorer sidebar and select New File to create each one.

help me create YAML manifest files in the correct folder structure for Kubernetes

✔️ Awesome, I've got everything!

Great. Make sure all three files are saved inside your k8s directory before moving on.

ⓧ I'd like to double check the full code

Here is the full content of each manifest file for comparison.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: floci-microservice
  labels:
    app: floci-microservice
spec:
  replicas: 2
  selector:
    matchLabels:
      app: floci-microservice
  template:
    metadata:
      labels:
        app: floci-microservice
    spec:
      containers:
        - name: microservice
          image: floci-microservice:latest
          imagePullPolicy: Never
          ports:
            - containerPort: 8080
          env:
            - name: AWS_ENDPOINT_URL
              value: "http://host.docker.internal:4566"
            - name: AWS_DEFAULT_REGION
              value: "us-east-1"
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 3
            periodSeconds: 5
apiVersion: v1
kind: Service
metadata:
  name: floci-microservice
  labels:
    app: floci-microservice
spec:
  selector:
    app: floci-microservice
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
  type: ClusterIP
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: floci-microservice
  labels:
    release: monitoring
spec:
  selector:
    matchLabels:
      app: floci-microservice
  endpoints:
    - port: "80"
      path: /metrics
      interval: 15s
Deploy the application

With your manifests ready, you can now apply them to your local Kubernetes cluster. Kubernetes reads each file, creates the resources, and starts scheduling your pods.

  • Apply all manifests in the k8s directory by running this command:
kubectl apply -f k8s/

What does this command do?

The kubectl apply -f k8s/ command reads every YAML file in the k8s/ directory and creates (or updates) the corresponding resources in your cluster. You should see output confirming the Deployment, Service, and ServiceMonitor were created.

  • Watch your pods start up by running:
kubectl get pods -w

You should see two pods with names starting with floci-microservice- transition from ContainerCreating to Running. Once both show 1/1 Running, press Ctrl+C to stop watching.

Pods stuck in ImagePullBackOff or ErrImageNeverPull?

This means Kubernetes cannot find the floci-microservice:latest image locally. Make sure you built the Docker image with the exact name floci-microservice:latest in the previous step. Run docker images to confirm it appears in the list.

Also confirm that imagePullPolicy: Never is set in your k8s/deployment.yaml. Without this, Kubernetes tries to pull from a remote registry and fails.

help me fix pods stuck in ImagePullBackOff when using a local Docker image on Docker Desktop Kubernetes

  • Port-forward the Service to access it from your machine by running:
kubectl port-forward svc/floci-microservice 8080:80

What does port-forward do?

This command creates a tunnel from localhost:8080 on your machine to port 80 on the Kubernetes Service. The Service then routes traffic to port 8080 on one of your pods. The command stays running in the foreground.

  • Open a second terminal window. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar, type Terminal or PowerShell, and press Enter.
  • Test the health endpoint by running:
curl http://localhost:8080/health

You should see {"status": "healthy"} in the response.

  • Verify the metrics endpoint by running:
curl http://localhost:8080/metrics

You should see Prometheus-formatted metrics including http_requests_total and http_request_duration_seconds in the output.

Connection refused on localhost:8080?

Make sure the kubectl port-forward command is still running in your first terminal. If it exited, re-run it. Also confirm your pods are in Running state with kubectl get pods.

If pods are running but the health check returns an error, check the pod logs by running kubectl logs followed by the pod name from kubectl get pods.

help me debug why kubectl port-forward is not working for my floci-microservice

Your microservice is live on Kubernetes with health probes keeping it resilient. Next up, you will install Prometheus and Grafana to visualize those metrics flowing from your /metrics endpoint in real-time dashboards.

Set Up Prometheus and Grafana Monitoring

Your microservice is deployed and exposing metrics, but without a monitoring system collecting and visualizing that data, those metrics go unseen.

In this step, you will install a production-grade observability stack using Prometheus and Grafana. Prometheus scrapes the /metrics endpoint your app already exposes, and Grafana turns that raw data into live dashboards.

In this step, get ready to:
  • Install the kube-prometheus-stack Helm chart into your cluster.
  • Access Grafana and create a monitoring dashboard.
  • Generate traffic and observe live metrics updating in real-time.
Install kube-prometheus-stack via Helm

The kube-prometheus-stack Helm chart bundles Prometheus, Grafana, Alertmanager, and the Prometheus Operator into a single install.

  • Add the prometheus-community Helm repository and update it by running these commands:
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

What do these commands do?

The first command registers the prometheus-community chart repository with your local Helm installation. The second command downloads the latest chart index so Helm knows what versions are available.

  • Install the kube-prometheus-stack chart as a release named monitoring in a new monitoring namespace by running:
helm install monitoring prometheus-community/kube-prometheus-stack --namespace monitoring --create-namespace --set grafana.service.type=NodePort

What does this command do?

  • helm install monitoring creates a Helm release called "monitoring".
  • --namespace monitoring --create-namespace puts all resources into a dedicated namespace, creating it if it does not exist.
  • --set grafana.service.type=NodePort exposes Grafana on a node port so you can access it via port-forward.

This install takes 1-2 minutes. Helm will print a success message when it finishes.

  • Verify that all monitoring pods are running by executing:
kubectl get pods -n monitoring

You should see pods for Prometheus, Grafana, Alertmanager, node-exporter, and kube-state-metrics all in a Running or Ready state.

Pods stuck in Pending or CrashLoopBackOff?

  • Make sure Docker Desktop has enough memory allocated (at least 4 GB recommended).
  • Run kubectl describe pod <pod-name> -n monitoring to see specific error messages.
  • If node-exporter pods fail, this is expected on some Windows Docker Desktop setups and does not affect Prometheus or Grafana functionality.

Having trouble getting the monitoring stack running? Help me troubleshoot my kube-prometheus-stack installation.

Connect to Grafana and create a dashboard

With the monitoring stack running, you can now access Grafana and build a dashboard that visualizes the RED metrics (Rate, Errors, Duration) from your microservice.

  • Port-forward Grafana to your local machine by running:
kubectl port-forward svc/monitoring-grafana -n monitoring 3000:80

Why port 3000?

Grafana runs on port 3000 by default. The service exposes it on port 80 inside the cluster, and port-forward maps your local port 3000 to that cluster-internal port 80.

  • Open your browser and navigate to http://localhost:3000.
  • Log in with username admin.
  • Retrieve the password by opening a new terminal and running:
kubectl --namespace monitoring get secrets monitoring-grafana -o jsonpath="{.data.admin-password}" | ForEach-Object { [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($_)) }

What is this command doing?

The kube-prometheus-stack chart generates a random admin password and stores it in a Kubernetes Secret. This command reads the base64-encoded password from the secret and decodes it so you can copy-paste it into the login form.

  • Copy the decoded password from your terminal and paste it into the Grafana login page. Click Log in.

You should see the Grafana home screen with the Prometheus data source already configured.

  • Click the + icon in the top-right area, then select New dashboard.
  • Click Add visualization.
  • Select Prometheus as the data source.
  • In the query editor, enter the following PromQL query for request rate:
rate(http_requests_total[5m])
  • Set the panel title to Request Rate in the panel options on the right.
  • Click Apply to save the panel.
  • Click Add visualization again to add a second panel.
  • Select Prometheus and enter this query for error rate:
rate(http_errors_total[5m])
  • Set the panel title to Error Rate and click Apply.
  • Add a third panel with this query for P95 latency:
histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m]))
  • Set the panel title to P95 Latency and click Apply.
  • Click the save icon (floppy disk) at the top of the dashboard.
  • Name the dashboard Microservice Metrics and click Save.

What are RED metrics?

RED stands for Rate, Errors, and Duration. These are the three signals SRE teams use to monitor any request-driven service. Rate tells you how busy the service is. Errors tell you how often it fails. Duration tells you how slow it is.

The three panels you just created map directly to these signals using the metrics your Flask app exposes via the prometheus_client library.

Dashboard panels showing 'No data'?

  • Prometheus needs time to scrape your microservice. Wait 30-60 seconds and refresh.
  • Verify the ServiceMonitor is being picked up by checking Status > Targets in the Prometheus UI (port-forward to port 9090 on the Prometheus service to check).
  • Confirm that your servicemonitor.yaml has the label release: monitoring which matches the Helm release name.

Still having trouble seeing metrics in Grafana? Help me debug why Prometheus is not scraping my microservice.

Generate load and observe metrics

The dashboard panels will be empty until your microservice receives traffic. A simple loop generating requests will bring those panels to life.

  • Make sure your microservice port-forward is still active. If it stopped, restart it in a new terminal by running:
kubectl port-forward svc/floci-microservice 8080:80
  • Open another new terminal and generate traffic by running this loop:
for ($i=1; $i -le 50; $i++) {
    Invoke-RestMethod -Method POST -Uri http://localhost:8080/items -ContentType "application/json" -Body '{"name": "item-$i", "value": "test"}'
    Invoke-RestMethod -Uri http://localhost:8080/items
    Start-Sleep -Milliseconds 200
}

What does this loop do?

  • It sends 50 POST requests to create items and 50 GET requests to list them.
  • Each pair has a 200ms delay between them to simulate realistic traffic patterns.
  • This generates enough data points for the rate() and histogram_quantile() PromQL functions to produce meaningful graphs.
  • Switch back to your Grafana browser tab.
  • Click the refresh icon or set the auto-refresh interval to 5 seconds using the dropdown at the top-right.

You should see all three panels updating with live data. The Request Rate panel shows increasing requests per second. The P95 Latency panel shows response times. The Error Rate should remain at zero since all requests are succeeding.

Metrics not updating after running the loop?

  • Check that your microservice port-forward (port 8080) is still active in its terminal window.
  • Verify the loop ran successfully by checking for JSON responses in the terminal output.
  • Prometheus scrapes every 15 seconds (as defined in your ServiceMonitor). Wait at least 30 seconds after the loop finishes, then refresh Grafana.

Need help seeing live metrics? Why are my Grafana panels not updating with new data?

Your microservice now has full production-grade observability. Prometheus scrapes metrics every 15 seconds, and Grafana gives you a real-time window into your service's health. Next up, take on the Secret Mission and extend this system with an event-driven SQS processor.

Secret mission

Add an SQS Event Processor

Your microservice handles synchronous HTTP requests. But production systems also need asynchronous event processing. In this mission, you'll add an SQS-triggered worker that consumes messages, writes to DynamoDB, and exposes custom Prometheus metrics on a live Grafana dashboard.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing cloud costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Floci Docker container (running on port 4566).
  • Kubernetes Deployments and Services (floci-microservice, floci-worker, monitoring stack).
  • Helm release monitoring in the monitoring namespace (Prometheus + Grafana).
  • Docker images (floci-microservice:latest and floci-worker:latest).
  • Installed CLI tools (Floci CLI and Helm via Scoop, AWS CLI via MSI installer).
  • Project files on your Desktop.

Keep everything running

No action needed. Choose this if you are still actively building or experimenting with the microservice, monitoring stack, or Terraform configurations. Everything runs locally at zero cost.

  • Floci stays running in the background as a Docker container.
  • Kubernetes Deployments remain active in your local cluster.
  • The Floci CLI, AWS CLI, and Helm remain installed and available on your PATH.

Pause - I'll come back to this later

Shut down running processes to free up memory and CPU while keeping your files and configuration intact.

  • Stop the Floci container by running:
floci stop
  • Delete the Kubernetes resources to free cluster memory by running:
kubectl delete -f k8s/
kubectl delete namespace monitoring

Your project files, Docker images, Terraform state, and installed CLI tools remain on your machine. When you are ready to resume, run floci start and re-apply your manifests.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below to clean up completely.

  • Stop and remove the Floci container by running:
floci stop --remove
  • Delete the Kubernetes application resources by running:
kubectl delete -f k8s/
  • Uninstall the Helm monitoring release by running:
helm uninstall monitoring -n monitoring
  • Delete the monitoring namespace by running:
kubectl delete namespace monitoring
  • Remove the Docker images by running:
docker rmi floci-microservice:latest floci-worker:latest
  • Uninstall the Floci CLI and Helm via Scoop by running:
scoop uninstall floci
scoop uninstall helm
  • Uninstall the AWS CLI. Open Programs and Features in Control Panel, select AWS Command Line Interface, and click Uninstall.
  • Delete the project folder from your Desktop by running:
cd ~\Desktop
Remove-Item -Recurse -Force [[PROJECT_DIR="your project folder name"]]

What about the Scoop bucket?

If you added the Floci Scoop bucket and want to remove it, run:

scoop bucket rm floci

This removes the bucket reference but does not affect any other Scoop packages.

That's a Wrap!

That's a Wrap!

Nice work. You just built a fully observable, infrastructure-as-code-driven microservice running on Kubernetes with local cloud backends.

You've learned how to:

  • Provision cloud infrastructure (S3, DynamoDB) using Terraform with custom endpoint configuration against Floci, a local AWS emulator.
  • Containerize a Python microservice with Docker and deploy it to a local Kubernetes cluster with health probes and resource manifests.
  • Install a production-grade observability stack (Prometheus + Grafana) using Helm and create a live monitoring dashboard showing RED metrics (Rate, Errors, Duration).
  • Secret Mission: Add an event-driven SQS consumer with custom business metrics and end-to-end observability.

Ready to quiz yourself?