Complete the OrderFlow GitOps Loop

Release and roll back a Catalog image through ECR, EKS, and Argo CD GitOps.

Introduction

30 Second Summary

A change can pass every check yet leave the live app running the wrong version. A rollback can disappear moments later when another system restores what it expects.

In this project, you will trace one Catalog change through a complete CI/CD release into Amazon EKS. You will prove that Git controls both deployment and a durable rollback.

What You'll Build

You will watch an Ergonomic Mouse source commit travel through GitHub Actions into Amazon ECR before Argo CD rolls that exact image into the live Catalog service.

By the end of this project, you'll have:

A traceable release where the exact source commit SHA appears in the immutable image tag and the GitOps image reference.
Live Catalog logs that show Ergonomic Mouse after automated tests, Trivy scanning, image publication, and Argo CD synchronization finish.
A rollback proof that shows Kubernetes self-healing restore the Git-declared image. A revert commit then brings Wireless Mouse back as the durable baseline.
Secret Mission: Introduce a deliberately failing Catalog status change. Prove the pipeline gate blocks it before Amazon ECR publication or GitOps movement.

What do I need before starting?

This project continues the preceding OrderFlow EKS, Argo CD, and secure image pipeline projects. You need the retained cluster, worker, repositories, and inherited GitHub Actions credentials from those projects.

Before We Start

This checkpoint locks in your release proof and paid-resource cleanup plan before any hands-on work begins. You are committing to trace one Catalog release through CI/CD, immutable Amazon ECR delivery, GitOps, Argo CD, and Amazon EKS while using Git for the durable rollback.

Verify the Delivery Environment

Your final release proof relies on infrastructure inherited from three earlier projects. An unavailable worker can break the release before it begins.

This step checks the Linux tools plus the shared GitOps repository. A split repository would leave the deployment update outside the path watched by Argo CD.

You then prove Amazon EKS has a Ready worker. You also confirm the worker can pull images from Amazon ECR before the CI/CD loop begins.

In this step, get ready to:

Verify the Linux workstation tools.
Confirm access to the shared GitHub repositories.
Restore the worker boundary and verify Argo CD.

Verify the Linux workstation tools

The delivery loop depends on Git for repository history. Visual Studio Code gives you a consistent view of the inherited files.

AWS CLI v2 selects the inherited cluster. The kubectl command checks its live state.

Open a terminal from your Linux application launcher.
Check all four workstation tools by running these commands:

git --version
code --version
aws --version
kubectl version --client

What do these checks prove?

The first command confirms Git can manage the source and GitOps repositories.
The second command confirms Visual Studio Code is available from your terminal.
The final commands confirm AWS CLI v2 plus the Kubernetes client are available.

✔️ All four commands show versions

Your Linux toolchain is available. Keep this terminal open for the repository and cluster checks.

ⓧ Git or VS Code is missing

Install Git with the package manager for your Linux distribution. Use the matching tab below.

Debian or Ubuntu

Install Git by running this command:

sudo apt install git-all

What does this command do?

The package manager installs Git plus its standard supporting tools. Your distribution records the installation for future updates.

Fedora, RHEL, or CentOS

Install Git by running this command:

sudo dnf install git-all

What does this command do?

The package manager installs Git plus its standard supporting tools. Your distribution records the installation for future updates.

Install Visual Studio Code with Snap by running this command when Snap is available:

sudo snap install --classic code

What does this command do?

Snap installs Visual Studio Code with classic system access. That access lets the editor work with the repositories already stored on your machine.

Use the official Visual Studio Code Linux guide when Snap is unavailable.
Run the four version checks above again after the installations finish.

Still missing a workstation command?

Restart your terminal so it reloads the command search path. Then run the version checks again.

If a command is still unavailable, help me diagnose my Linux Git or Visual Studio Code installation.

ⓧ AWS CLI v2 or kubectl is missing

AWS CLI v2 provides the authenticated connection to AWS. Install it before trying to select the cluster.

Install AWS CLI v2 for your Linux user by running this command:

curl -fsSL https://awscli.amazonaws.com/v2/install.sh | bash

What does this command do?

The command downloads the current-user Linux installer from AWS. It passes that installer to your shell so AWS CLI v2 becomes available without a system-wide package.

Restart your terminal if the AWS command remains unavailable.
Download the matching Linux binary from the official kubectl installation guide.
Install the downloaded Kubernetes client by running this command:

sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

What does this command do?

The command places the downloaded client in a system command directory. Its permissions allow every local user to run it.

Run the four version checks above again after both installations finish.
Keep your kubectl client within one minor version of the inherited EKS control plane.

Still missing a cluster command?

Check that the AWS CLI installer completed without a network error. Confirm the downloaded kubectl binary was in your current directory before using the install command.

For environment-specific help, help me fix my AWS CLI v2 or kubectl installation on Linux. You can also share the non-sensitive error text with the NextWork community.

Resolve the repository handoff and select the cluster

Argo CD watches one public repository for desired state. The pipeline must update that same repository for a release to reach the cluster.

Open GitHub in your web browser.
Sign in with the account used for the earlier OrderFlow projects if prompted.
Select the orderflow-gitops repository.
Confirm the repository is public.
Confirm k8s/overlays/gitops exists in the repository.
Confirm apps/catalog/deployment.yaml exists in the same repository.

Why use one shared repository?

The inherited Argo CD Application already reads from the public orderflow-gitops repository. Keeping the pipeline-managed deployment there gives the controller one source of desired state.

Select the orderflow-catalog repository.
Confirm catalog.sh exists.
Confirm tests/catalog_test.sh exists.
Confirm .github/workflows/catalog-ci.yml exists.
Confirm scripts/update-image.sh exists.

Are the GitOps files split?

The earlier projects may have placed the pipeline files in a differently named repository. Resolve that handoff before continuing.

Copy apps/catalog/deployment.yaml into the public orderflow-gitops repository watched by Argo CD.
Update GITOPS_REPOSITORY in orderflow-catalog to reference the shared repository.
Configure the write-enabled deploy key from the pipeline project for that shared repository.
Confirm both required GitOps paths now exist in one public repository.

If the handoff still looks unclear, help me reconcile my two OrderFlow GitOps repository locations.

Use these tabs to compare the inherited baseline files before any release edits begin.

✔️ I see every baseline file

Your repository handoff is intact. Preserve the existing image reference because the next step connects that exact baseline to Argo CD.

ⓧ I'd like to double check the full code

Compare orderflow-catalog/catalog.sh with this inherited baseline.

#!/bin/sh
set -eu

print_catalog() {
  printf '%s\n' '{"service":"catalog","status":"ok","items":[{"sku":"NW-101","name":"Keyboard"},{"sku":"NW-102","name":"Wireless Mouse"}]}'
}

print_catalog

if [ "${1:-}" = "--once" ]; then
  exit 0
fi

trap 'exit 0' TERM INT
while :; do
  sleep 3600 &
  wait "$!"
done

What should this baseline contain?

The service reports an OK status plus two catalog items. The second item remains Wireless Mouse until the release step changes the source.

Compare orderflow-gitops/apps/catalog/deployment.yaml with this inherited baseline. Keep your actual baseline image URI where the descriptive image placeholder appears.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog
spec:
  replicas: 1
  selector:
    matchLabels:
      app: catalog
  template:
    metadata:
      labels:
        app: catalog
    spec:
      containers:
        - name: catalog
          image: your-baseline-ecr-image-uri-here

What should this manifest contain?

The manifest defines one catalog replica. Its single image: field is the boundary updated by the inherited pipeline.

The repositories describe the release inputs. Your AWS identity now needs permission to select the inherited cluster.

Confirm the active AWS identity by running this command:

aws sts get-caller-identity

What does this command prove?

The response identifies the AWS account plus the active principal. This confirms the terminal has credentials before it requests cluster access.

AWS identity unavailable?

Authenticate with the same AWS access method used in the preceding OrderFlow projects. Avoid pasting credentials into screenshots or project files.

If authentication still fails, help me diagnose my AWS CLI identity without exposing credentials.

Select the inherited EKS cluster by running this command:

aws eks update-kubeconfig --region us-east-1 --name orderflow-foundation

What does this command do?

The command updates your kubeconfig for orderflow-foundation in us-east-1. It also selects that cluster as the current context.

Check the inherited worker state by running this command:

kubectl get nodes

What should this check reveal?

The output shows whether the managed worker is registered with the selected cluster. Use its status to choose the matching recovery path below.

Cluster missing?

A deleted orderflow-foundation cluster is outside this project's rebuild scope. Repeat the EKS foundation plus deployment projects before continuing.

Restore the worker and verify Argo CD

The managed node group supplies the compute that runs each Kubernetes Pod. A registered worker must reach Ready state before Argo CD can reconcile the Catalog workload.

✔️ I see a Ready node

The worker is already available. Continue with the live watch to confirm it remains Ready.

ⓧ No node is Ready

The inherited worker may have been paused to reduce charges. Restore one desired node before continuing.

Restore the orderflow-workers desired capacity by running this command:

aws eks update-nodegroup-config --cluster-name orderflow-foundation --nodegroup-name orderflow-workers --scaling-config minSize=1,maxSize=2,desiredSize=1 --region us-east-1

What does this command change?

The node group keeps a minimum of one worker. It can scale to two workers while requesting one worker now.

The waiter checks AWS every 30 seconds, so quiet gaps are expected while the node group returns to active state.

Wait for the node group update to finish by running this command:

aws eks wait nodegroup-active --cluster-name orderflow-foundation --nodegroup-name orderflow-workers --region us-east-1

What does the waiter confirm?

The command returns after AWS reports the managed node group as active. Kubernetes may still need a short interval to register the worker as Ready.

Node group staying unavailable?

Confirm your AWS identity can manage the inherited cluster. Check that the cluster name plus region match the command exactly.

For a stalled update, help me diagnose the orderflow-workers node group.

Watch the worker registration by running this command:

kubectl get nodes --watch

What does the watch show?

The watch streams node status changes from the selected cluster. It gives you a live signal when the worker reaches Ready state.

Press Ctrl+C after one node becomes Ready.

That is the compute boundary restored. Your cluster now has a worker ready to run the release workload.

A Ready worker still needs permission to download the private Catalog image. The node role supplies that pull boundary.

Display the managed node role by running this command:

aws eks describe-nodegroup --cluster-name orderflow-foundation --nodegroup-name orderflow-workers --region us-east-1 --query 'nodegroup.nodeRole' --output text

What does this output contain?

The output is the full IAM role ARN used by the managed workers. The role name is the portion after the final slash.

Copy the role name after the final slash into your-node-role-name.
Replace your-node-role-name in the command below with the copied role name.
List the managed policies attached to that role by running this command:

aws iam list-attached-role-policies --role-name your-node-role-name

Why inspect the node role?

EKS workers need permission to retrieve private image layers plus authorization tokens from Amazon ECR. AmazonEC2ContainerRegistryPullOnly provides that boundary.

✔️ I see the pull-only policy

The worker can pull private Amazon ECR images. An equivalent custom policy is also valid when it grants the required private pull permissions.

ⓧ The pull permission is missing

Changing a node role permission can feel risky. This change gives the worker pull-only access to private Amazon ECR images.

Attach AmazonEC2ContainerRegistryPullOnly to the recorded node role by following the official EKS node role guide.
Replace your-node-role-name in the command below with the recorded role name.
Confirm the policy is attached by running this command:

aws iam list-attached-role-policies --role-name your-node-role-name

What confirms the permission?

The policy list should now include AmazonEC2ContainerRegistryPullOnly or the equivalent custom policy used by your environment.

Unable to attach the policy?

Your active AWS identity may lack permission to change IAM roles. Use the authorized identity from the earlier EKS foundation project.

If the policy still does not appear, help me verify the ECR pull policy on my EKS node role.

The final check covers the controller side of the delivery chain. Your inherited Argo CD v3.5.2 installation should expose running workloads plus the existing orderflow Application.

Before you run the final checks, do you expect both the worker and the controller to be ready?

Verify the worker plus the inherited Argo CD resources by running these commands:

kubectl get nodes
kubectl -n argocd get statefulsets,deployments,pods
kubectl -n argocd get applications

What do these checks prove?

The node output proves the selected EKS cluster has compute available.
The workload output proves the inherited Argo CD controller components are available in argocd.
The Application output proves the inherited orderflow desired state is still registered.

You should see one Ready node. You should also see the inherited orderflow Application in the argocd namespace.

Your delivery foundation is ready. The repositories, worker boundary, image pull permission, and controller are available for the release loop.

Missing an Argo CD workload or Application?

Confirm kubeconfig still targets orderflow-foundation in us-east-1. Then check that the earlier Argo CD project completed successfully.

For missing controller resources, help me diagnose my inherited Argo CD installation.

Your inherited environment is confirmed from repository to controller. Next, you will connect the pipeline-managed Catalog path to Argo CD.

Connect Releases to Argo CD

Your delivery environment is ready. The worker can pull the existing Catalog image from the private registry.

The pipeline writes releases to apps/catalog/deployment.yaml. The inherited Argo CD Application watches k8s/overlays/gitops.

That mismatch leaves the image handoff stored in Git. A dedicated Application lets Argo CD reconcile the release into Amazon EKS.

In this step, get ready to:

Pin the Catalog Deployment to the orderflow namespace.
Declare a guarded Argo CD Application for apps/catalog.
Reconcile the baseline image into the cluster.

Qualify the Catalog Deployment namespace

A Kubernetes namespace separates related workloads inside a cluster. The standalone Catalog Deployment needs an explicit destination that matches the Argo CD Application.

Press the Super key to open Linux application search.
Type Visual Studio Code into the search field.
Select Visual Studio Code from the results.
Select File from the top menu.
Select Open Folder.
Select your local orderflow-gitops folder.
Select Open.
Select apps/catalog/deployment.yaml from the Explorer sidebar.

Find this existing metadata block:

metadata:
  name: catalog

Why add a namespace?

The current metadata names the Deployment without declaring its namespace. An explicit namespace keeps the Git manifest aligned with the Application destination.

Add the namespace beneath name: catalog so the block looks like this:

metadata:
  name: catalog
  namespace: orderflow

What changed?

The Deployment now declares orderflow as its target namespace. Argo CD can reconcile the manifest into the same namespace declared by the new Application.

Save apps/catalog/deployment.yaml.
Select the Search icon in the left sidebar.
Search for image: within apps/catalog/deployment.yaml.

You should see exactly one image: field. That single field remains the pipeline's image update boundary.

Leave the existing image value unchanged.
Compare the account number in the image URI with the AWS account you verified earlier.
Confirm the registry section contains us-east-1.
Confirm the repository segment identifies orderflow-catalog in Amazon ECR.
Select the Source Control icon in the left sidebar.
Select the changed apps/catalog/deployment.yaml file.

The diff should show one added line for namespace: orderflow. The existing image line should remain unchanged.

Seeing more than one image field?

Remove any duplicate container or image entry from apps/catalog/deployment.yaml. The inherited update script expects exactly one image field.

Compare the file structure with the full reference below if indentation created a second container entry.

help me restore one image field in the Catalog Deployment manifest

Declare the guarded release Application

An Argo CD Application declares a Git source plus a cluster destination. This Application watches only the pipeline-managed apps/catalog path.

The manifest below contains a learner-specific repository placeholder. You replace that placeholder before the Application reaches the cluster.

Select the top-level orderflow-gitops folder in the Explorer sidebar.
Create an argocd folder with the New Folder control.

You should see argocd beside the existing apps and k8s folders.

Select the new argocd folder.
Create catalog-release-application.yaml with the New File control.

The blank file should appear at argocd/catalog-release-application.yaml.

Open the public orderflow-gitops repository in GitHub.
Select the Code menu.
Select the HTTPS option.
Record the public clone URL here: your public GitHub HTTPS clone URL.

Populate argocd/catalog-release-application.yaml by copying this manifest:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: catalog-release
  namespace: argocd
spec:
  project: default
  source:
    repoURL: your-public-github-https-clone-url
    targetRevision: main
    path: apps/catalog
    directory:
      recurse: false
  destination:
    server: https://kubernetes.default.svc
    namespace: orderflow
  syncPolicy:
    automated:
      enabled: true
      prune: false
      selfHeal: true
      allowEmpty: false
    syncOptions:
      - FailOnSharedResource=true

What does this Application control?

The source watches the main branch at apps/catalog.
The destination sends the manifest to the in-cluster Kubernetes API.
The destination namespace keeps the Catalog Deployment inside orderflow.
The selfHeal setting restores live state when it drifts from Git.
The disabled prune setting prevents automated resource deletion.
The FailOnSharedResource=true option stops synchronization when another Application already owns the Deployment.

Replace your-public-github-https-clone-url with your public GitHub HTTPS clone URL.
Save argocd/catalog-release-application.yaml.
Select the Source Control icon in the left sidebar.

You should see changes for apps/catalog/deployment.yaml plus argocd/catalog-release-application.yaml.

A Git commit makes both declarations available to the controller. Publishing them together prevents the Application from reading a namespace-ambiguous Deployment.

Select Terminal from the Visual Studio Code menu.
Select New Terminal.
Publish the two GitOps files to main by running:

git add apps/catalog/deployment.yaml argocd/catalog-release-application.yaml
git commit -m "Connect Catalog releases to Argo CD"
git push origin main

What do these commands publish?

The first command stages only the two GitOps files from this step. The second command records them as one release-boundary change.

The final command sends that commit to the shared repository's main branch. Argo CD can now read the same files you reviewed locally.

Return to the orderflow-gitops repository on GitHub.
Select the main branch.
Confirm the latest commit is named Connect Catalog releases to Argo CD.
Confirm the commit contains both GitOps files.

Push rejected?

Confirm the integrated terminal is inside the local orderflow-gitops repository. A terminal opened from another folder cannot stage these paths.

Confirm your GitHub identity has write access to the public repository. Refresh your authentication before retrying the push.

help me diagnose why the GitOps commit did not push

Reading the reference files

The baseline image placeholder below represents the pipeline-written URI that remains in your local file. The repository URL placeholder represents the public clone URL you inserted.

The source script remains unchanged in this step. Its baseline output still contains Wireless Mouse.

✔️ Awesome, I've got everything!

Confirm the three project files match the intended state for this release connection.

ⓧ I'd like to double check the full code

Compare these complete reference files with the current project state.

#!/bin/sh
set -eu

print_catalog() {
  printf '%s\n' '{"service":"catalog","status":"ok","items":[{"sku":"NW-101","name":"Keyboard"},{"sku":"NW-102","name":"Wireless Mouse"}]}'
}

print_catalog

if [ "${1:-}" = "--once" ]; then
  exit 0
fi

trap 'exit 0' TERM INT
while :; do
  sleep 3600 &
  wait "$!"
done

apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog
  namespace: orderflow
spec:
  replicas: 1
  selector:
    matchLabels:
      app: catalog
  template:
    metadata:
      labels:
        app: catalog
    spec:
      containers:
        - name: catalog
          image: your-baseline-ecr-image-uri-here

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: catalog-release
  namespace: argocd
spec:
  project: default
  source:
    repoURL: your-public-github-https-clone-url
    targetRevision: main
    path: apps/catalog
    directory:
      recurse: false
  destination:
    server: https://kubernetes.default.svc
    namespace: orderflow
  syncPolicy:
    automated:
      enabled: true
      prune: false
      selfHeal: true
      allowEmpty: false
    syncOptions:
      - FailOnSharedResource=true

Apply the Application and verify the baseline

The manifest in Git declares the release path. Applying the Application creates the reconciliation link that reads that path.

Create the reconciliation link by running:

kubectl apply -f argocd/catalog-release-application.yaml

What did this create?

The command submits the catalog-release Application to the argocd namespace. The controller now compares apps/catalog with the live cluster.

A brief wait here is expected. Default Argo CD reconciliation can take up to three minutes before the new Application reports its synchronized state.

Watch the new Application reconcile by running:

kubectl get application catalog-release -n argocd -w

What does synchronized prove?

A synchronized status proves Argo CD read the declared source path. It also proves the desired resource matches the state accepted by the cluster.

Stop the watch with Ctrl+C after the Application reports synchronized.

Shared resource protection blocked the sync?

Check the inherited orderflow Application source under k8s/overlays/gitops for another Catalog Deployment. Remove the duplicate Catalog resource from one Application source before retrying.

Keep one Application responsible for the catalog Deployment. The shared-resource guard exists to prevent two controllers from competing over it.

help me resolve an Argo CD shared-resource conflict for the Catalog Deployment

Synchronization confirms the desired manifest was accepted. Deployment rollout status confirms the Catalog Pod became ready with that manifest.

Confirm the baseline Deployment finishes rolling out by running:

kubectl rollout status deployment/catalog -n orderflow --timeout=180s

What does rollout status confirm?

The Deployment controller waits for the new Pod to become available. A completed rollout proves the namespace-qualified manifest produced a running workload.

If the rollout stalls with ImagePullBackOff, inspect the selected Pod by running:

kubectl -n orderflow describe pod -l app=catalog

What does the Pod description reveal?

The event section shows whether the node could read the full private image reference. It helps separate an incorrect URI from missing registry pull permission.

Catalog image did not pull?

Compare the complete image value in apps/catalog/deployment.yaml with the inherited private registry URI. A missing registry segment or image tag points the Pod at the wrong artifact.

Recheck that the orderflow-workers node role still has private ECR pull permission.

help me diagnose why the Catalog Pod cannot pull its baseline ECR image

Before you check the Pod logs, which Catalog item do you expect the baseline image to print?

Read the latest Catalog log line by running:

kubectl -n orderflow logs deployment/catalog --tail=1

What does this log prove?

The command reads one line from the running Catalog Deployment. That output verifies the reconciled image is executing inside the cluster.

You should see Catalog JSON containing Wireless Mouse. That is the baseline image now controlled through Git and Argo CD.

You closed the missing handoff. A pipeline-managed manifest can now reach the running Catalog Deployment without giving the pipeline cluster credentials.

The reconciliation link is live. Next, you will trace one source commit through the full CI/CD chain into the running cluster.

Ship the Catalog Release

Your guarded Argo CD Application now keeps the baseline Catalog Deployment aligned with Git. The missing proof is whether a real source change can cross every delivery boundary as one traceable release.

In this step, you will send an Ergonomic Mouse change through GitHub Actions, tests, Trivy, an immutable Amazon ECR image, and GitOps. You will then verify that the exact source commit reaches the live Amazon EKS Deployment.

In this step, get ready to:

Create a Catalog source change with a recorded commit SHA.
Trace the source commit through the existing delivery pipeline.
Verify that the exact immutable image runs in the cluster.

Create and identify the release

The source commit SHA gives this release one identity across code, image publication, desired state, and live runtime. Keeping the service name plus status unchanged also proves that the release contains only the intended product rename.

In the Explorer sidebar in Visual Studio Code, select catalog.sh inside the local orderflow-catalog repository.
Inside print_catalog(), find this line:

  printf '%s\n' '{"service":"catalog","status":"ok","items":[{"sku":"NW-101","name":"Keyboard"},{"sku":"NW-102","name":"Wireless Mouse"}]}'

What Is Changing?

This line defines the Catalog JSON printed by the running workload. The existing Wireless Mouse value is the baseline you already observed in the cluster.

Replace Wireless Mouse with Ergonomic Mouse so the line looks like this:

  printf '%s\n' '{"service":"catalog","status":"ok","items":[{"sku":"NW-101","name":"Keyboard"},{"sku":"NW-102","name":"Ergonomic Mouse"}]}'

Why Keep the Rest Unchanged?

The existing test protects the service name plus the "status":"ok" contract. Changing only the product name isolates the release signal you will trace.

Save catalog.sh.

Before you run the test, do you expect the product rename to preserve the Catalog contract?

Verify the source change by running the existing Catalog test:

sh tests/catalog_test.sh

What Does This Test Prove?

The test executes the Catalog script against the inherited contract. A successful result proves that the rename preserved the required healthy status.

The terminal returns to the prompt without a test failure. Your product rename is ready for the release pipeline.

Catalog Test Failing?

Confirm that catalog.sh still contains "status":"ok".
Confirm that only Wireless Mouse changed to Ergonomic Mouse.
Save catalog.sh before rerunning the test.

Help me diagnose the failing Catalog test.

Stage the source change and create its release commit by running:

git add catalog.sh && git commit -m "Rename the Catalog mouse"

What Does This Commit Establish?

The first command stages the updated script. The second command records the exact source state that the pipeline uses to tag the immutable image.

Commit Blocked?

Confirm that your terminal is inside the local orderflow-catalog repository.
Configure your Git author details if Git asks for an identity.

Help me resolve the Catalog commit problem.

Push the release commit to main by running:

git push origin main

What Does This Push Trigger?

The push sends the source commit to the remote main branch. The existing image pipeline starts from that commit.

The source commit is now available to the pipeline. This is the release boundary you will follow through every remaining system.

Push Rejected?

Confirm that your GitHub account has write access to orderflow-catalog.
Confirm that your local branch has not diverged from remote main.

Help me troubleshoot the rejected Catalog push.

Print the full source commit SHA by running:

git rev-parse --verify HEAD

Why Record the Full SHA?

The command verifies the current commit object before printing its full hash. The pipeline uses this identity as the immutable image tag.

Record the printed hash here: your Ergonomic Mouse source commit SHA.

Follow the pipeline handoff

The pipeline owns testing, scanning, artifact publication, and the GitOps update. It does not need Kubernetes or Argo CD credentials because Git carries the deployment handoff.

Return to the orderflow-catalog repository on GitHub.
Select Actions in the repository navigation.
Open the newest Catalog image pipeline run.
Select the publish job.

What Boundary Are You Checking?

The job log shows where each gate sits in the release chain. Its order proves that untested or unscanned source cannot reach image publication.

Confirm that Run Catalog tests finishes successfully.
Confirm that the test finishes before the Trivy scan.
Confirm that the scan finishes before AWS authentication.
Confirm that the immutable image is published or safely reused.
Confirm that Update and push deployment configuration finishes last.
Confirm that no Kubernetes or Argo CD credentials appear in the job.

That successful run proves the separation of duties. CI publishes the artifact plus desired state while Argo CD retains cluster deployment authority.

Switch back to the terminal session for the local orderflow-gitops repository from the previous step.
Pull the pipeline-created GitOps commit by running:

git pull --ff-only

Why Use a Fast-Forward Pull?

This pull updates your local branch only when its history can move straight to the remote commit. That protects the release trail from an accidental merge.

Pull Cannot Fast-Forward?

Confirm that the terminal is inside the local orderflow-gitops repository.
Confirm that the working tree has no uncommitted changes.
Stop before changing history if your local branch has diverged.

Help me resolve the GitOps fast-forward pull safely.

Return to apps/catalog/deployment.yaml in the Visual Studio Code workspace from the previous step.
Confirm that the image: value ends with your Ergonomic Mouse source commit SHA.
Record the complete image URI here: your full Ergonomic Mouse ECR image URI.

✔️ Awesome, I've got everything!

Your Catalog source file now contains the product rename. Your GitOps manifest now references the immutable image produced from that source commit.

ⓧ I'd like to double check the full code

Compare your complete orderflow-catalog/catalog.sh file against this reference.

#!/bin/sh
set -eu

print_catalog() {
  printf '%s\n' '{"service":"catalog","status":"ok","items":[{"sku":"NW-101","name":"Keyboard"},{"sku":"NW-102","name":"Ergonomic Mouse"}]}'
}

print_catalog

if [ "${1:-}" = "--once" ]; then
  exit 0
fi

trap 'exit 0' TERM INT
while :; do
  sleep 3600 &
  wait "$!"
done

Compare your complete orderflow-gitops/apps/catalog/deployment.yaml file against this reference. Your actual image: value contains the full ECR URI produced by the pipeline.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog
  namespace: orderflow
spec:
  replicas: 1
  selector:
    matchLabels:
      app: catalog
  template:
    metadata:
      labels:
        app: catalog
    spec:
      containers:
        - name: catalog
          image: your-ergonomic-mouse-source-commit-sha-ecr-image-uri-here

Confirm that orderflow-gitops/argocd/catalog-release-application.yaml remains unchanged.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: catalog-release
  namespace: argocd
spec:
  project: default
  source:
    repoURL: your-public-github-https-clone-url
    targetRevision: main
    path: apps/catalog
    directory:
      recurse: false
  destination:
    server: https://kubernetes.default.svc
    namespace: orderflow
  syncPolicy:
    automated:
      enabled: true
      prune: false
      selfHeal: true
      allowEmpty: false
    syncOptions:
      - FailOnSharedResource=true

Wait for the exact image and verify the rollout

A synchronized status can still describe the previous release. Waiting for the live Deployment image to equal the URI from GitOps removes that race.

Argo CD's default reconciliation can take up to three minutes. The wait may sit quietly while the controller detects the commit plus starts the new rollout.

Before you run the wait, do you think the live Deployment already matches the new GitOps image?

Wait for the exact live image by replacing your-full-image-uri-here with the complete URI you recorded in this command:

kubectl wait --for=jsonpath='{.spec.template.spec.containers[0].image}'=your-full-image-uri-here deployment/catalog -n orderflow --timeout=300s

What Does This Wait Prove?

The JSONPath reads the live Deployment's container image field. The command succeeds only when that field equals the complete immutable URI from the GitOps manifest.

The command exits successfully once Argo CD has reconciled the exact release image. You have now matched live state to the desired image from Git.

Exact Image Wait Timed Out?

Confirm that you replaced the placeholder with the complete image URI from apps/catalog/deployment.yaml.
Confirm that the image URI includes the recorded source SHA as its tag.
Check the catalog-release Application for a synchronization problem.

Help me diagnose why the live Catalog Deployment has not reached the exact GitOps image.

Verify rollout health plus inspect the active image by running these commands:

kubectl rollout status deployment/catalog -n orderflow --timeout=180s
kubectl -n orderflow describe deployment catalog

What Do These Checks Confirm?

The rollout status command waits for the new ReplicaSet to become available.
The Deployment description exposes the complete image currently assigned to the Pod template.

The rollout reaches completion. The Deployment description shows an image URI ending with your Ergonomic Mouse source commit SHA.

Before you read the newest log line, which mouse name do you expect the new Pod to print?

Read the latest Catalog output by running:

kubectl -n orderflow logs deployment/catalog --tail=1

What Does the Log Complete?

The log comes from the Pod running the image that Kubernetes just rolled out. It completes the trace from source text to live workload behavior.

You will see Catalog JSON containing Ergonomic Mouse. Strong work. One source SHA has now crossed tests, scanning, immutable publication, GitOps, Argo CD synchronization, and the live Deployment.

Still Seeing Wireless Mouse?

Confirm that the Deployment description shows the same complete image URI as apps/catalog/deployment.yaml.
Confirm that the live image tag matches the recorded source SHA.
Check that the rollout reached completion before reading the log.

Help me trace why the Catalog log still shows the baseline product.

Your source SHA is now live through the GitOps path. Next, you will test whether a cluster-only rollback can survive Argo CD self-healing.

Test the Cluster-Only Rollback

Your Catalog release is live on Amazon EKS. The logs now show Ergonomic Mouse.

A direct Kubernetes rollback changes the live Deployment using cluster history. Git still declares the current release.

This creates an authority conflict with Argo CD self-healing. You will watch that conflict resolve before checking which release remains active.

In this step, get ready to:

Confirm the Catalog Deployment has enough revision history for a rollback.
Watch the live image while a cluster-only rollback runs.
Verify Argo CD restores the image declared by Git.

Establish a visible image watch

Deployment revisions preserve earlier Pod templates. The rollback needs at least two revisions so Kubernetes has a previous image to restore.

Confirm the Catalog Deployment has at least two revisions by running this command:

kubectl rollout history deployment/catalog -n orderflow

What does this show?

The command lists the recorded revisions for the Catalog Deployment.
Each revision represents a saved Pod template that Kubernetes can restore.

You should see at least two revision numbers in the output.

Only see one revision?

Stop here if the output contains only one revision.
Confirm the earlier baseline image and the current Ergonomic Mouse image both completed a rollout before retrying this test.

Ask for help if the rollout history still contains one revision: help me determine why my catalog Deployment has only one rollout revision

The image watch exposes every change to the live Deployment. Keeping it visible makes a short-lived rollback easier to catch.

Start the live image watch in your first terminal by running this command:

kubectl get deployment/catalog -n orderflow -o custom-columns='IMAGE:.spec.template.spec.containers[0].image' --watch

What does the watch show?

The custom column displays only the image currently assigned to the Catalog Pod template.
The watch keeps listening for changes instead of exiting after the first result.

The first value should be the full ECR image URI ending with your recorded Ergonomic Mouse source commit SHA.

Trigger the cluster-only rollback

The image watch now shows the live side of the delivery system. The next command changes that live state without changing the GitOps manifest.

Start a second terminal session while leaving the image watch visible in the first terminal.

Before you trigger the rollback, consider this question: which image do you expect to remain once both controllers have responded.

Roll the live Catalog Deployment back to its previous revision by running this command in the second terminal:

kubectl rollout undo deployment/catalog -n orderflow

What just changed?

The rollback asks Kubernetes to restore the previous Deployment revision.
The command changes live cluster state only.
The GitOps manifest continues to declare the immutable Ergonomic Mouse image.

Argo CD checks Git for drift on a reconciliation interval that can take up to three minutes.

Return to the image watch in the first terminal.
Watch for the previous image to appear briefly.
Keep watching until the image ending with your recorded source SHA returns.
Stop the watch after the Git-declared image returns.

You should see the previous image first. The Ergonomic Mouse release image then returns when Argo CD self-heals the drift.

Did the image sequence look different?

Repeat the rollout history check if the previous image changed too quickly to catch visually.
Confirm selfHeal: true remains in argocd/catalog-release-application.yaml if the release image does not return.
Confirm the catalog-release Application remains available if no reconciliation occurs.

Use this prompt to investigate an unexpected result: help me troubleshoot why Argo CD did not restore the Git-declared Catalog image after kubectl rollout undo

Verify Git wins the conflict

The image watch proves that the rollback was temporary. A rollout check and a fresh log line confirm the final workload is healthy.

Before the final check, predict whether the logs will show the previous product name or the Git-declared product name.

Verify the final Catalog rollout and read its latest log line by running these commands:

kubectl rollout status deployment/catalog -n orderflow --timeout=180s
kubectl -n orderflow logs deployment/catalog --tail=1

What does this verify?

The rollout status waits for the restored Git-declared Deployment to become healthy.
The log command reads one line from the active Catalog workload.

The rollout completes successfully. The final JSON contains Ergonomic Mouse.

That proves the authority boundary: a cluster-only rollback is temporary while Git continues to declare the newer release.

Still seeing the baseline output?

Confirm the stopped image watch ended on the URI containing your recorded source SHA.
Confirm the GitOps manifest still references the Ergonomic Mouse image.
Repeat the final rollout check after Argo CD finishes reconciling.

Get targeted help with the remaining drift: help me compare my GitOps image with the live catalog Deployment after Argo CD self-healing

✔️ Awesome, I've got everything!

Your repository files remain unchanged. The rollback test touched live cluster state before Argo CD restored the state declared in Git.

ⓧ I'd like to double check the full code

Compare orderflow-catalog/catalog.sh with the unchanged source file below.

#!/bin/sh
set -eu

print_catalog() {
  printf '%s\n' '{"service":"catalog","status":"ok","items":[{"sku":"NW-101","name":"Keyboard"},{"sku":"NW-102","name":"Ergonomic Mouse"}]}'
}

print_catalog

if [ "${1:-}" = "--once" ]; then
  exit 0
fi

trap 'exit 0' TERM INT
while :; do
  sleep 3600 &
  wait "$!"
done

Compare orderflow-gitops/apps/catalog/deployment.yaml with the unchanged GitOps manifest below.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog
  namespace: orderflow
spec:
  replicas: 1
  selector:
    matchLabels:
      app: catalog
  template:
    metadata:
      labels:
        app: catalog
    spec:
      containers:
        - name: catalog
          image: your-ergonomic-mouse-source-commit-sha-ecr-image-uri-here

Compare orderflow-gitops/argocd/catalog-release-application.yaml with the unchanged Argo CD Application below.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: catalog-release
  namespace: argocd
spec:
  project: default
  source:
    repoURL: your-public-github-https-clone-url
    targetRevision: main
    path: apps/catalog
    directory:
      recurse: false
  destination:
    server: https://kubernetes.default.svc
    namespace: orderflow
  syncPolicy:
    automated:
      enabled: true
      prune: false
      selfHeal: true
      allowEmpty: false
    syncOptions:
      - FailOnSharedResource=true

You have proved that changing the cluster alone cannot create a durable rollback. Next, you will move the desired state backward through a new Git commit.

Roll Back Through Git

Your cluster-only rollback has already been reversed by Argo CD. That result proves the live Kubernetes Deployment follows the image still declared in Git.

A durable rollback starts where desired state lives. Reverting the pipeline-created GitOps commit gives Argo CD a previous image to reconcile through a new commit.

In this step, get ready to:

Identify the pipeline-created release commit in the shared GitOps repository.
Revert the release commit without rewriting main.
Prove the reverted baseline is active across GitOps and the live Deployment.

Identify the GitOps release commit

The release commit changed apps/catalog/deployment.yaml to the Ergonomic Mouse image. Its full commit hash gives git revert an exact change to reverse.

Switch back to the terminal for your local orderflow-gitops repository.
Bring your local main branch up to date by running:

git pull --ff-only

What does this command do?

The --ff-only option updates your local branch only when Git can move it forward without creating a merge commit. This protects the clean release history you are about to inspect.

Find the latest commit that changed apps/catalog/deployment.yaml by running:

git log -1 --format=%H -- apps/catalog/deployment.yaml

What does this command show?

The path filter limits the history search to apps/catalog/deployment.yaml. The output is the full hash of the pipeline-created release commit.

Record the printed hash here: your-release-commit-hash.
Confirm the working tree is clean by running:

git status --short

What should you see?

A clean working tree produces no output in the short status format. Blank output confirms the revert cannot accidentally include local file changes.

Seeing local changes?

Stop before reverting if the status command lists any files. Preserve or discard those unrelated changes using the method that fits your work.

Run the status check again when the repository is clean.

help me prepare a clean Git working tree before reverting a commit

Create a forward-moving rollback commit

A revert preserves the existing release history by adding a new commit that reverses the selected change. The restored baseline becomes the latest desired state on main.

This keeps the shared branch history intact. The rollback remains visible to everyone who uses the repository.

Create the rollback commit by substituting your-release-commit-hash for your-release-commit-hash in this command:

git revert --no-edit your-release-commit-hash

What does this command do?

The named commit identifies the release change that Git reverses.
The --no-edit option accepts the generated revert message without opening an editor.
The resulting commit restores the baseline content of apps/catalog/deployment.yaml.

Git creates a new revert commit in your local history. The original release commit remains available as part of the audit trail.

Publish the revert commit to the shared branch by running:

git push origin main

What happens after the push?

The remote main branch receives the new desired state. Argo CD can now reconcile the baseline image from Git.

Return to the main history in the shared orderflow-gitops repository on GitHub to confirm a new revert commit is present.
Switch back to apps/catalog/deployment.yaml in Visual Studio Code to find the restored image: field.
Record the complete image URI here: your-baseline-ecr-image-uri-here.

GitHub now shows a forward-moving revert commit. The manifest points to the earlier immutable Amazon ECR image.

Push rejected?

Confirm your GitHub access still has write permission for the shared repository. A protected branch may require the same contribution path you used in the earlier pipeline project.

help me troubleshoot a rejected Git push for the OrderFlow GitOps rollback

Wait for the exact rollback image

A successful rollback requires the live Deployment image to equal the URI restored in Git. This exact-image check prevents an older synchronization result from looking like success.

Argo CD may take a few minutes to detect the commit. The wait stays active while the controller replaces the running Pod.

Before you run the wait, which image should the live Deployment reach now that Git has changed?

Wait for the live image by replacing your-baseline-image-uri-here with your-baseline-ecr-image-uri-here in this command:

kubectl wait --for=jsonpath='{.spec.template.spec.containers[0].image}'=your-baseline-image-uri-here deployment/catalog -n orderflow --timeout=300s

What does this command check?

The JSONPath selects the live container image from the catalog Deployment. The command exits successfully only when that value equals the complete baseline URI.

The completed wait proves Argo CD has moved the Deployment specification back to the Git-declared baseline image.

Did the image wait time out?

Check that the command contains the complete baseline URI from apps/catalog/deployment.yaml. A shortened repository name or missing image tag cannot match the live field.

Confirm the catalog-release Application can still read the latest commit from the shared repository.

help me diagnose why the Catalog Deployment did not reach the reverted image URI

Confirm the replacement Pods finish rolling out by running:

kubectl rollout status deployment/catalog -n orderflow --timeout=180s

What does rollout status prove?

The rollout check waits for the Deployment to complete its update. Success confirms the baseline image is running in a healthy replacement Pod.

Inspect the active Deployment image by running:

kubectl -n orderflow describe deployment catalog

What should you compare?

Find the active image in the Deployment details. It should match your-baseline-ecr-image-uri-here from the reverted manifest.

Before you read the final log line, which product name should the baseline image print?

Read the latest Catalog output by running:

kubectl -n orderflow logs deployment/catalog --tail=1

What does the log prove?

The log comes from the Pod created by the reverted Deployment. Its Catalog JSON verifies the behavior packaged inside the baseline image.

You should see Catalog JSON containing Wireless Mouse. That is the durable rollback complete: Git declares the baseline image again.

Your orderflow-catalog source still contains Ergonomic Mouse. The GitOps repository deliberately deploys the earlier baseline artifact.

Still seeing Ergonomic Mouse?

Compare the live Deployment image with the complete URI in apps/catalog/deployment.yaml. Different values show that reconciliation has not completed.

If the URIs match, confirm the baseline image tag is the one restored by the revert commit.

help me diagnose why the Catalog logs still show Ergonomic Mouse after the GitOps revert

✔️ Awesome, I've got everything!

Your GitOps manifest now contains the restored baseline image URI. The live Deployment uses that same immutable image.

ⓧ I'd like to double check the full code

Compare your apps/catalog/deployment.yaml file with this reference. Keep the actual baseline URI restored by your revert commit in the image: field.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: catalog
  namespace: orderflow
spec:
  replicas: 1
  selector:
    matchLabels:
      app: catalog
  template:
    metadata:
      labels:
        app: catalog
    spec:
      containers:
        - name: catalog
          image: your-baseline-ecr-image-uri-here

Secret mission

Prove a Broken Change Stops Before Deployment

A successful release proves that your delivery path can move forward. This challenge proves that a failing test stops a broken source commit before image publication or deployment.

Clean Up Your Resources

The live Amazon EKS environment continues to generate AWS charges. Decide whether to keep it running, pause the worker, or delete the final series environment.

Cost warning

The standard-support Amazon EKS control plane costs $0.10 per cluster-hour. Worker capacity, storage, public IPv4 usage, and data transfer can add charges.

Choose Pause or Delete if you have finished the release proof. Deleting the final environment stops the widest range of ongoing charges.

Resources you used:

The paid orderflow-foundation Amazon EKS cluster with its orderflow-workers managed node group.
The catalog-release and inherited orderflow applications managed by Argo CD v3.5.2.
The orderflow namespace with its project PersistentVolumes and inherited Amazon EBS volumes.
The public IPv4 address and other AWS infrastructure attached to the final series environment.
The orderflow-catalog Amazon ECR repository plus any retained orderflow/* repositories.

Keep everything running

No action is needed. Choose this if you are still testing the release loop or continuing the OrderFlow series.

Keep the orderflow-workers node group at one desired node.
Retain the catalog-release and orderflow applications.
Keep the orderflow-catalog and shared orderflow-gitops GitHub repositories as your source history.
Expect the control plane, worker, storage, public IPv4 address, ECR storage, and possible data transfer to keep generating charges.

Pause - I'll come back to this later

A paused worker releases its compute capacity while preserving the cluster and project history. The EKS control plane plus retained storage continue to generate charges.

Scale the orderflow-workers node group to zero by running this command:

aws eks update-nodegroup-config --cluster-name orderflow-foundation --nodegroup-name orderflow-workers --scaling-config minSize=0,maxSize=2,desiredSize=0 --region us-east-1

The AWS CLI sets the minimum and desired worker capacity to zero. The cluster control plane stays available.

Scale-in can take several minutes. A quiet terminal during the wait is normal.

Wait for the node group update to finish by running this command:

aws eks wait nodegroup-active --cluster-name orderflow-foundation --nodegroup-name orderflow-workers --region us-east-1

The command returns when the node group reaches ACTIVE status.

Keep both GitHub repositories so you can resume from the same source and desired-state history.
Keep both Argo CD applications so synchronization resumes when worker capacity returns.

Worker refusing to scale down?

Confirm your AWS identity still has access to orderflow-foundation in us-east-1. Check that the node group name is orderflow-workers.

Ask for help with the exact command output: Help me diagnose why my OrderFlow EKS node group did not scale to zero.

Delete - I don't want to use this again

Deleting the final series environment is permanent. Your GitHub repositories remain available, so the release history and manifests stay intact.

Record the project volume handles before deletion: your EBS volume handles.
Delete the catalog-release Application from its saved manifest by running this command:

kubectl delete -f argocd/catalog-release-application.yaml

This removes the Application that reconciles apps/catalog into the cluster.

Application deletion failed?

Confirm the terminal is connected to orderflow-foundation. Run the command from the orderflow-gitops checkout that contains argocd/catalog-release-application.yaml.

Ask for help without sharing AWS credentials: Help me diagnose why kubectl could not delete my catalog-release Application manifest.

Delete the inherited orderflow Application using the retained cleanup instructions from the preceding Argo CD project.
Delete the orderflow namespace using the retained cleanup instructions from the preceding deployment project.
Confirm the project PersistentVolumes have disappeared before continuing.
Confirm the volumes matching your EBS volume handles have disappeared before continuing.

Cluster deletion can take several minutes. The waiting period gives AWS time to remove the control plane and managed node group cleanly.

Delete the orderflow-foundation cluster from its retained configuration by running this command:

eksctl delete cluster -f cluster.yaml --disable-nodegroup-eviction --wait

The wait option keeps the command running until cluster deletion finishes. This cleanup also removes the managed node group.

Cluster deletion stopped?

Confirm cluster.yaml is the retained configuration for orderflow-foundation. Check that the AWS identity still has permission to delete the inherited environment.

Ask for help with any remaining resources: Help me diagnose why my eksctl cluster deletion did not complete.

Replace your-repository-name with orderflow-catalog before running this command:

aws ecr delete-repository --repository-name your-repository-name --force --region us-east-1

The --force option deletes the repository even when it still contains images.

Repeat the ECR deletion command for each retained orderflow/* repository that you no longer need.
Keep the orderflow-catalog and orderflow-gitops GitHub repositories as the release record.
Remove the inherited orderflow-catalog-ci IAM role with the cleanup path from the pipeline project if no workflow still needs it.
Remove the GitHub OIDC provider with the inherited cleanup path only when no other workflow in the AWS account uses it.

ECR repository still present?

Confirm the repository name matches the value shown in us-east-1. Repeat the command separately for every repository you want to remove.

Ask for help without exposing AWS credentials: Help me diagnose why my ECR repository deletion failed.

Nice Work!

You did it! Your OrderFlow delivery loop now carries a source commit to a live Amazon EKS rollout through CI/CD and GitOps.

You've learned how to:

Connected the pipeline-managed apps/catalog path to Amazon EKS through a guarded Argo CD Application. Kept cluster credentials out of GitHub Actions.
Confirmed the Catalog source SHA passed its tests. Followed the same SHA through Trivy. Matched it to an immutable Amazon ECR image. Verified Argo CD rolled that image out to Amazon EKS.
Proved that a cluster-only Kubernetes rollback cannot outlast Argo CD self-healing. Restored Wireless Mouse through a forward-moving revert commit in Git.
Secret Mission: Proved the pipeline gate blocked the degraded status change before AWS authentication. Confirmed Amazon ECR stayed on the baseline. Confirmed apps/catalog/deployment.yaml stayed on the baseline. Recovered main through a new revert commit. Saw the pipeline return to green.

Ready to quiz yourself?