GitOps Across Clusters with Argo CD

Build two-cluster GitOps with Argo CD for promotion, self-healing, and recovery.

Introduction

30 Second Summary

A software change can look safe in testing. The same change can cause trouble when it reaches the live version.

In this project, you will build a production-style GitOps platform across two local Kubernetes clusters. Each cluster runs its own Argo CD control plane while Git controls the release lifecycle.

What You'll Build

You will show two browser pages moving through a controlled release, with the production page returning to its Git-approved state after drift and a failed update.

By the end of this project, you'll have:

  • Two isolated release pages that let you compare development with production while each page runs from its own local Kubernetes cluster.
  • Git-controlled promotion that moves release 1.1.0 into development before production. Your commit history provides the audit trail.
  • Recovery proof showing Argo CD flag an unauthorized production hotfix as drift. You will see self-healing restore Git's version before a Git revert repairs a faulty release.
  • Secret Mission: Add production guardrails with an AppProject that permits only the approved Git repository, in-cluster destination, release-demo namespace, and required Namespace resource.

Are there any prerequisites?

You need a Mac with at least 4 GB of RAM, a supported macOS release, and a GitHub account. The guide covers every required local tool from installation through verification.

Before We Start

This is your moment to commit to the release you will promote before any hands-on work begins. You will also define why production must return to the state declared in Git after drift or failure.

Build the Two-Cluster Argo CD Platform

Development and production need independent failure boundaries. Separate Kubernetes clusters let one environment fail without taking the other environment with it.

In this step, you will prepare your Mac with Docker Desktop plus the required command-line tools. You will create two local clusters before installing an independent Argo CD control plane in each one.

In this step, get ready to:
  • Install the required macOS workstation tools.
  • Authenticate with GitHub before creating the development and production clusters.
  • Install an independent Argo CD control plane in each cluster.
Install and verify the workstation tools

Docker Desktop runs the containers that become your local clusters. Homebrew installs the remaining command-line tools with consistent versions.

  • Press Cmd+Space to open Spotlight.
  • Type Terminal into Spotlight.
  • Press Enter to open Terminal.
  • Check the current Docker installation by running this command:
docker version

This command asks the Docker client for its version details. It also contacts the Docker engine to confirm that Docker Desktop is running.

✔️ I use the required Docker Desktop release

Your Docker installation is ready. Make sure Docker Desktop stays running while you create both clusters.

  • Press Cmd+Space to open Spotlight.
  • Type Docker into Spotlight.
  • Press Enter to start Docker Desktop.
  • Wait for Docker Desktop to finish starting.

ⓧ I use an older Docker Desktop release

Install Docker Desktop 4.88.1 over the older release. This keeps the local cluster environment aligned with the project.

The subscription terms can feel weighty. Personal learning is covered by Docker Personal at $0. Qualifying larger enterprises and government entities need a paid subscription.

  • Download Docker Desktop 4.88.1 for your Mac chip from Docker's Mac installation page.
  • Open Docker.dmg from your Downloads folder.
  • Drag Docker.app into /Applications.
  • Approve the replacement if macOS asks about the existing application.
  • Press Cmd+Space to open Spotlight.
  • Type Docker into Spotlight.
  • Press Enter to start Docker Desktop.
  • Accept the applicable subscription terms.

ⓧ Docker Desktop is not installed

Install Docker Desktop 4.88.1 before continuing. The installation gives kind the container engine it needs to build local clusters.

The subscription terms can look intimidating. Personal learning is covered by Docker Personal at $0. Qualifying larger enterprises and government entities need a paid subscription.

  • Download Docker Desktop 4.88.1 for your Mac chip from Docker's Mac installation page.
  • Open Docker.dmg from your Downloads folder.
  • Drag Docker.app into /Applications.
  • Press Cmd+Space to open Spotlight.
  • Type Docker into Spotlight.
  • Press Enter to start Docker Desktop.
  • Accept the applicable subscription terms.
  • Wait for Docker Desktop to finish starting.
  • Confirm that the Docker engine is available by running this command again:
docker version

You should see details for both the client and the server. That result confirms Docker Desktop is running and ready to host your clusters.

  • Check whether Homebrew is already installed by running this command:
brew --version

This command prints the installed Homebrew release. The required release for this project is 6.0.0.

✔️ I see the required Homebrew version

Homebrew is ready to install the rest of your toolchain. Continue with the command-line tool checks below.

ⓧ I see an older Homebrew version

Update Homebrew before installing the project tools. This refreshes Homebrew itself plus its available package information.

  • Update Homebrew by running this command:
brew update

This command downloads the current Homebrew metadata and updates the Homebrew installation. Let it finish before checking the version again.

ⓧ Homebrew is not installed

The installer may request your Mac password. It also pauses to explain its changes before asking you to continue.

  • Install Homebrew by running this command:
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

This command downloads the official installer before running it with Bash. Follow any post-install shell instructions printed in Terminal so that new sessions can find brew.

  • Confirm the Homebrew installation by running this command:
brew --version

You should see Homebrew 6.0.0 in the output. That confirms the package manager is ready.

The remaining toolchain includes Git for version history. It also includes the GitHub CLI plus the Kubernetes and Argo CD command-line clients.

  • Check all five command-line tools by running these commands:
git --version
gh --version
kubectl version --client
kind version
argocd version --client

These commands only inspect the local clients. Together they reveal which tools are ready before Homebrew changes anything.

✔️ Every tool shows the required version

Your command-line toolchain is already ready. Continue to GitHub authentication.

ⓧ One or more tools show an older version

Install the current Homebrew formulae to align every tool with the project. Homebrew handles each package independently.

  • Install the required tool releases by running these commands:
brew install git
brew install gh
brew install kubernetes-cli
brew install kind
brew install argocd

These commands install Git, GitHub CLI, kubectl, kind, and the Argo CD CLI from their Homebrew formulae. Homebrew prints the result of each installation before moving to the next command.

ⓧ One or more commands are unavailable

Install the missing tools through Homebrew. Running the full set also confirms that every dependency is present.

  • Install the required tool releases by running these commands:
brew install git
brew install gh
brew install kubernetes-cli
brew install kind
brew install argocd

These commands install Git, GitHub CLI, kubectl, kind, and the Argo CD CLI from their Homebrew formulae. Each successful installation adds its command to your toolchain.

  • Verify the complete command-line toolchain by running the checks again:
git --version
gh --version
kubectl version --client
kind version
argocd version --client

You should see Git 2.55.0, GitHub CLI 2.98.0, kubectl 1.37.0, kind 0.33.0, and Argo CD CLI 3.5.2.

A tool still is not available?

Close Terminal before opening a fresh Terminal session. A new session reloads the command paths added during installation.

If one tool still fails, rerun its matching Homebrew install command and inspect the final output for the failed package.

Help me diagnose which Homebrew tool is missing from my command path.

Authenticate and create two pinned clusters

The next step publishes the GitOps configuration to GitHub. Authenticating now lets the GitHub CLI create that repository without interrupting the cluster workflow later.

Each kind cluster uses the same pinned Kubernetes node image. Separate kind-dev and kind-prod contexts keep every cluster command explicit.

  • Start GitHub's browser authentication flow by running this command:
gh auth login --web --git-protocol https

What does this command do?

The web option authenticates your GitHub account through a browser flow. The Git protocol option configures repository operations to use HTTPS.

  • Complete the authorization flow in your browser.
  • Return to Terminal after GitHub confirms the authorization.

Your terminal should return after the browser flow completes. That confirms the GitHub CLI can act through your account.

Browser authentication did not complete?

Keep Terminal open while completing the browser flow. Restart the command if the browser session expires before authorization finishes.

Help me complete GitHub CLI browser authentication on macOS.

The cluster image is pinned by tag plus digest. The digest makes both environments use the same Kubernetes 1.37.0 node contents on Apple Silicon or Intel Macs.

Creating both clusters can take several minutes because Docker downloads the node image during the first run. The five-minute wait gives each control plane time to become ready.

  • Create the development and production clusters by running these commands:
kind create cluster --name dev --image kindest/node:v1.37.0@sha256:a1ed56cfb0e7b93589bdf97c8cd566405a265939e3620fc4f5de89adff580ae5 --wait 5m
kind create cluster --name prod --image kindest/node:v1.37.0@sha256:a1ed56cfb0e7b93589bdf97c8cd566405a265939e3620fc4f5de89adff580ae5 --wait 5m

What do these commands create?

  • The first command creates the dev cluster with the kind-dev kubectl context.
  • The second command creates the prod cluster with the kind-prod kubectl context.
  • The shared image digest gives both clusters an identical Kubernetes node build.

Before you list the clusters, do you expect to see one shared environment or two independently named environments?

  • List the created kind clusters by running this command:
kind get clusters

You should see dev and prod on separate lines. That proves Docker is running two independent local clusters.

A cluster did not become ready?

Confirm that Docker Desktop is still running. A stopped Docker engine prevents kind from creating or starting cluster nodes.

If only one cluster appears, rerun the creation command for the missing cluster. Keep the cluster name and pinned image unchanged.

Help me troubleshoot a kind cluster that did not become ready within five minutes.

Install Argo CD in both clusters

Each cluster needs its own reconciliation control plane. Installing Argo CD locally inside both clusters lets development and production continue operating independently.

Why use one control plane per cluster?

A centralized Argo CD instance would need network access plus credentials for both local clusters. Cluster-local control planes avoid that credential path while preserving independent environment failure boundaries.

This topology keeps the two-cluster lesson visible. Each Argo CD installation reconciles only the cluster where it runs.

  • Create the Argo CD namespaces and install the pinned release by running these commands:
kubectl --context kind-dev create namespace argocd
kubectl --context kind-dev apply -n argocd --server-side --force-conflicts -f https://raw.githubusercontent.com/argoproj/argo-cd/v3.5.2/manifests/install.yaml
kubectl --context kind-prod create namespace argocd
kubectl --context kind-prod apply -n argocd --server-side --force-conflicts -f https://raw.githubusercontent.com/argoproj/argo-cd/v3.5.2/manifests/install.yaml

What does this installation do?

  • Each namespace isolates the Argo CD components under argocd inside its cluster.
  • Each apply command installs the standard Argo CD v3.5.2 manifest into the selected context.
  • Server-side apply lets the Kubernetes API server manage the manifest fields.

The deployments can take a few minutes to become available while their container images download. Give each cluster the full five-minute timeout if needed.

  • Wait for both Argo CD server deployments to become available by running these commands:
kubectl --context kind-dev wait --for=condition=Available deployment/argocd-server -n argocd --timeout=5m
kubectl --context kind-prod wait --for=condition=Available deployment/argocd-server -n argocd --timeout=5m

Each command waits for the selected argocd-server Deployment to satisfy its Available condition. Both successful waits prove the independent control planes can serve requests.

An Argo CD server did not become available?

Check that the affected cluster still appears in the kind cluster list. Confirm that Docker Desktop has enough memory available for both clusters.

Rerun the matching wait command after the Argo CD images finish downloading. Keep the context aligned with the cluster that failed.

Help me troubleshoot an Argo CD server Deployment that is not Available in one kind cluster.

Before the final check, do you expect both cluster names plus a separate Argo CD deployment table for each context?

  • Verify both clusters and control planes by running these commands:
kind get clusters
kubectl --context kind-dev get deployments -n argocd && kubectl --context kind-prod get deployments -n argocd

You should see dev and prod in the cluster list. You should also see available Argo CD deployments in both command tables.

Your Two-Cluster Platform State

Created with name 'dev', pinned node image kindest/node:v1.37.0@sha256:a1ed56cfb0e7b93589bdf97c8cd566405a265939e3620fc4f5de89adff580ae5, accessible through context kind-dev, with Argo CD v3.5.2 installed in namespace argocd.

Created with name 'prod', pinned node image kindest/node:v1.37.0@sha256:a1ed56cfb0e7b93589bdf97c8cd566405a265939e3620fc4f5de89adff580ae5, accessible through context kind-prod, with Argo CD v3.5.2 installed in namespace argocd.

Your two cluster-local control planes are ready. Next up, you will give each one a Git-managed root Application and watch both environments deploy their first visible release.

Bootstrap Both Clusters from Git

Your two independent Argo CD control planes are running. However, installed controllers have no declared workloads to reconcile.

In this step, Git becomes the source of truth for your GitOps platform. A root Application in each cluster watches its own repository path.

In this step, get ready to:
  • Create the repository structure for both environments.
  • Publish the public repository to GitHub.
  • Bootstrap both clusters from Git.
Create the GitOps repository structure

The App of Apps pattern gives each cluster one root Application. That root Application creates the child Application responsible for the visible release.

Why keep separate environment paths?

The development root watches clusters/dev/apps. The production root watches clusters/prod/apps.

Each child Application also points to a separate release path. This structure makes production promotion an explicit Git change.

  • Press Cmd+Space to open Spotlight Search.
  • Type Terminal.
  • Press Enter to open Terminal.
  • Move to your Desktop by running this command:
cd ~/Desktop

What does this command do?

This command moves your Terminal session to the Desktop. Your project folder will be easy to find in Finder.

  • Create the argo-two-cluster-gitops folder with its nested environment directories by running:
mkdir -p argo-two-cluster-gitops/bootstrap argo-two-cluster-gitops/clusters/dev/apps argo-two-cluster-gitops/clusters/prod/apps argo-two-cluster-gitops/environments/dev argo-two-cluster-gitops/environments/prod
cd argo-two-cluster-gitops

What does this command do?

The first line creates the complete repository structure. The second line moves your Terminal session into argo-two-cluster-gitops.

  • Confirm the top-level folders exist by running:
ls

You should see bootstrap, clusters, and environments in the output.

Missing one of the folders?

Check that your Terminal prompt ends with argo-two-cluster-gitops. Run the folder creation command again if one of the three top-level folders is missing.

Ask for help with the exact folder structure if the output still differs: help me create the required GitOps repository folders on macOS

Every Application manifest needs the address of your public repository. Record your GitHub username here before you create those files: your-github-username.

Create the root and child Applications

The root Applications watch the child Application manifests. The child Applications watch the release manifests for their own environments.

  • Create bootstrap/dev-root.yaml with Nano by running:
nano bootstrap/dev-root.yaml

Nano opens a new plain-text file at bootstrap/dev-root.yaml.

  • Paste this development root Application into Nano:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: dev-root
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-github-username/argo-two-cluster-gitops.git
    targetRevision: main
    path: clusters/dev/apps
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: true

What does this manifest do?

  • The dev-root Application watches clusters/dev/apps on the main branch.
  • The in-cluster server address keeps this root Application inside the development cluster.
  • Automated pruning removes resources that disappear from the watched Git path.
  • Root self-healing keeps the child Application definition aligned with Git.
  • Replace your-github-username with your-github-username.
  • Press Ctrl+O to save the file.
  • Press Enter to confirm the filename.
  • Press Ctrl+X to exit Nano.
  • Create bootstrap/prod-root.yaml with Nano by running:
nano bootstrap/prod-root.yaml

Nano opens the matching root Application file for production.

  • Paste this production root Application into Nano:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: prod-root
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-github-username/argo-two-cluster-gitops.git
    targetRevision: main
    path: clusters/prod/apps
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: true

What changes for production?

The production root is named prod-root. It watches clusters/prod/apps.

The destination remains the local cluster where this Application runs. This preserves the independent production control plane.

  • Replace your-github-username with your-github-username.
  • Press Ctrl+O to save the file.
  • Press Enter to confirm the filename.
  • Press Ctrl+X to exit Nano.
  • Create clusters/dev/apps/release-demo.yaml with Nano by running:
nano clusters/dev/apps/release-demo.yaml

Nano opens the development child Application file.

  • Paste this development child Application into Nano:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: release-demo
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-github-username/argo-two-cluster-gitops.git
    targetRevision: main
    path: environments/dev
  destination:
    server: https://kubernetes.default.svc
    namespace: release-demo
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: false
    syncOptions:
      - CreateNamespace=true

Why is self-healing disabled here?

This child Application deploys the development release from environments/dev. The CreateNamespace=true option creates the release-demo namespace.

The child starts with selfHeal: false. A later step uses that policy to make configuration drift visible before enabling automatic repair.

  • Replace your-github-username with your-github-username.
  • Press Ctrl+O to save the file.
  • Press Enter to confirm the filename.
  • Press Ctrl+X to exit Nano.
  • Create clusters/prod/apps/release-demo.yaml with Nano by running:
nano clusters/prod/apps/release-demo.yaml

Nano opens the production child Application file.

  • Paste this production child Application into Nano:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: release-demo
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-github-username/argo-two-cluster-gitops.git
    targetRevision: main
    path: environments/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: release-demo
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: false
    syncOptions:
      - CreateNamespace=true

How is production isolated?

This child watches only environments/prod. It deploys into the release-demo namespace of its own cluster.

Development changes cannot reach this Application until the production path receives its own Git commit.

  • Replace your-github-username with your-github-username.
  • Press Ctrl+O to save the file.
  • Press Enter to confirm the filename.
  • Press Ctrl+X to exit Nano.

Application file missing or empty?

Make sure you pressed Ctrl+O before leaving Nano. Confirm the path shown at the top of Nano matches the manifest you are editing.

Check that all four repoURL values contain your GitHub username. Help me compare my four Argo CD Application manifests

Create the visible release manifests

Each release file contains a ConfigMap for the web page. It also contains a Deployment plus a Service.

Both environments begin at release 1.0.0. Their labels make the two clusters easy to distinguish in a browser.

  • Create environments/dev/release.yaml with Nano by running:
nano environments/dev/release.yaml

Nano opens the development release manifest.

  • Paste the development page configuration into Nano:
apiVersion: v1
kind: ConfigMap
metadata:
  name: release-page
data:
  index.html: |
    <!doctype html>
    <html lang="en">
      <head><meta charset="utf-8"><title>Development Release</title></head>
      <body>
        <h1>Release 1.0.0</h1>
        <p>Environment: development</p>
        <p>Delivered by Argo CD from Git.</p>
      </body>
    </html>

What does this configuration store?

The ConfigMap stores the complete development page as index.html. The page shows release 1.0.0 with the development label.

  • Move to a new line below the closing HTML tag.
  • Paste the development Deployment below the ConfigMap:
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: release-demo
  template:
    metadata:
      labels:
        app: release-demo
    spec:
      containers:
        - name: nginx
          image: nginx:1.30.4-alpine@sha256:97d490c12ba55b4946b01546d1c3ed324e8d41ab1c9fcb2a616aa470620e5b46
          ports:
            - containerPort: 80
          volumeMounts:
            - name: page
              mountPath: /usr/share/nginx/html
              readOnly: true
      volumes:
        - name: page
          configMap:
            name: release-page

What does this Deployment run?

The Deployment runs one NGINX container from the pinned image. It mounts the release-page ConfigMap at the web server document path.

The Deployment selector matches the Pod label app: release-demo. This allows Kubernetes to manage the correct Pod.

  • Move to a new line below the ConfigMap volume.
  • Paste the development Service below the Deployment:
---
apiVersion: v1
kind: Service
metadata:
  name: release-demo
spec:
  selector:
    app: release-demo
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

What does this Service expose?

The Service selects the release Pod through app: release-demo. It sends traffic from Service port 80 to container port 80.

  • Press Ctrl+O to save environments/dev/release.yaml.
  • Press Enter to confirm the filename.
  • Press Ctrl+X to exit Nano.
  • Create environments/prod/release.yaml with Nano by running:
nano environments/prod/release.yaml

Nano opens the production release manifest.

  • Paste the production page configuration into Nano:
apiVersion: v1
kind: ConfigMap
metadata:
  name: release-page
data:
  index.html: |
    <!doctype html>
    <html lang="en">
      <head><meta charset="utf-8"><title>Production Release</title></head>
      <body>
        <h1>Release 1.0.0</h1>
        <p>Environment: production</p>
        <p>Delivered by Argo CD from Git.</p>
      </body>
    </html>

What identifies production?

The production ConfigMap uses the same release number. Its title plus environment label identify the production cluster.

  • Move to a new line below the closing HTML tag.
  • Paste the production Deployment below the ConfigMap:
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: release-demo
  template:
    metadata:
      labels:
        app: release-demo
    spec:
      containers:
        - name: nginx
          image: nginx:1.30.4-alpine@sha256:97d490c12ba55b4946b01546d1c3ed324e8d41ab1c9fcb2a616aa470620e5b46
          ports:
            - containerPort: 80
          volumeMounts:
            - name: page
              mountPath: /usr/share/nginx/html
              readOnly: true
      volumes:
        - name: page
          configMap:
            name: release-page

Why use the same image?

Both environments use the same pinned NGINX image. The environment-specific page content comes from each cluster's own ConfigMap.

  • Move to a new line below the ConfigMap volume.
  • Paste the production Service below the Deployment:
---
apiVersion: v1
kind: Service
metadata:
  name: release-demo
spec:
  selector:
    app: release-demo
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

How does production receive traffic?

This Service targets only Pods in the production cluster with the matching label. A local port-forward later connects your browser to this Service.

  • Press Ctrl+O to save environments/prod/release.yaml.
  • Press Enter to confirm the filename.
  • Press Ctrl+X to exit Nano.

Seeing malformed YAML later?

Check that each --- separator begins at the far left. Confirm that spaces under each YAML field match the reference code.

Avoid tabs inside YAML files. Help me find indentation problems in my release manifest

Your six repository files are now in place. Use this comparison before you publish them.

✔️ Awesome, I've got everything!

Great. Confirm that all six files are saved before you continue.

ⓧ I'd like to double check the full code

Compare each local file with the matching reference below. Replace every your-github-username value with your own GitHub username in your saved files.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: dev-root
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-github-username/argo-two-cluster-gitops.git
    targetRevision: main
    path: clusters/dev/apps
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: true

This is the complete development root Application.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: prod-root
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-github-username/argo-two-cluster-gitops.git
    targetRevision: main
    path: clusters/prod/apps
  destination:
    server: https://kubernetes.default.svc
    namespace: argocd
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: true

This is the complete production root Application.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: release-demo
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-github-username/argo-two-cluster-gitops.git
    targetRevision: main
    path: environments/dev
  destination:
    server: https://kubernetes.default.svc
    namespace: release-demo
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: false
    syncOptions:
      - CreateNamespace=true

This is the complete development child Application.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: release-demo
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-github-username/argo-two-cluster-gitops.git
    targetRevision: main
    path: environments/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: release-demo
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: false
    syncOptions:
      - CreateNamespace=true

This is the complete production child Application.

apiVersion: v1
kind: ConfigMap
metadata:
  name: release-page
data:
  index.html: |
    <!doctype html>
    <html lang="en">
      <head><meta charset="utf-8"><title>Development Release</title></head>
      <body>
        <h1>Release 1.0.0</h1>
        <p>Environment: development</p>
        <p>Delivered by Argo CD from Git.</p>
      </body>
    </html>
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: release-demo
  template:
    metadata:
      labels:
        app: release-demo
    spec:
      containers:
        - name: nginx
          image: nginx:1.30.4-alpine@sha256:97d490c12ba55b4946b01546d1c3ed324e8d41ab1c9fcb2a616aa470620e5b46
          ports:
            - containerPort: 80
          volumeMounts:
            - name: page
              mountPath: /usr/share/nginx/html
              readOnly: true
      volumes:
        - name: page
          configMap:
            name: release-page
---
apiVersion: v1
kind: Service
metadata:
  name: release-demo
spec:
  selector:
    app: release-demo
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

This is the complete development release manifest.

apiVersion: v1
kind: ConfigMap
metadata:
  name: release-page
data:
  index.html: |
    <!doctype html>
    <html lang="en">
      <head><meta charset="utf-8"><title>Production Release</title></head>
      <body>
        <h1>Release 1.0.0</h1>
        <p>Environment: production</p>
        <p>Delivered by Argo CD from Git.</p>
      </body>
    </html>
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: release-demo
  template:
    metadata:
      labels:
        app: release-demo
    spec:
      containers:
        - name: nginx
          image: nginx:1.30.4-alpine@sha256:97d490c12ba55b4946b01546d1c3ed324e8d41ab1c9fcb2a616aa470620e5b46
          ports:
            - containerPort: 80
          volumeMounts:
            - name: page
              mountPath: /usr/share/nginx/html
              readOnly: true
      volumes:
        - name: page
          configMap:
            name: release-page
---
apiVersion: v1
kind: Service
metadata:
  name: release-demo
spec:
  selector:
    app: release-demo
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

This is the complete production release manifest.

Publish the repository to GitHub

Argo CD needs a remote repository that both clusters can read. GitHub hosts that shared desired state.

  • Initialize the main branch.
  • Stage the six repository files.
  • Create the bootstrap commit by running:
git init -b main
git add .
git commit -m "Bootstrap two-cluster GitOps"

What do these Git commands do?

  • The first command creates a repository on the main branch.
  • The second command stages every manifest in the current folder.
  • The third command records the complete two-cluster bootstrap as one snapshot.
  • Create the public remote repository by running:
gh repo create argo-two-cluster-gitops --public --source=. --remote=origin --push

What does this command create?

The GitHub CLI creates the public argo-two-cluster-gitops repository. It adds the repository as origin.

The command also pushes your local main branch. Argo CD can now read the manifests from GitHub.

Repository creation failed?

Confirm your Terminal prompt is inside the argo-two-cluster-gitops folder. A repository with the same name in your GitHub account can also block creation.

Your earlier GitHub authentication should still be active. Help me troubleshoot the GitHub CLI repository creation command

Before you inspect the remote, what repository address do you expect origin to show?

  • Display the configured remote by running:
git remote -v

You should see your argo-two-cluster-gitops GitHub URL listed for fetch plus push.

  • Compare the username in the remote URL with every repoURL value.

Remote and manifests do not match?

Return to each of the four Application manifests. Replace any remaining your-github-username text with your GitHub username.

Commit the corrected files before bootstrapping the clusters. Help me find a repository URL mismatch in my Argo CD manifests

Bootstrap both clusters and compare the releases

Each cluster needs one direct bootstrap action. After that point, its root Application reads Git plus creates the child Application.

  • Apply the development root Application to kind-dev by running:
kubectl --context kind-dev apply -n argocd -f bootstrap/dev-root.yaml

What does this bootstrap command do?

This creates dev-root in the development Argo CD namespace. The root then reads clusters/dev/apps from Git.

  • Apply the production root Application to kind-prod by running:
kubectl --context kind-prod apply -n argocd -f bootstrap/prod-root.yaml

What happens in production?

This creates prod-root in the production Argo CD namespace. That root reads only clusters/prod/apps.

Each root now reconciles its own cluster. The separation gives development plus production independent failure boundaries.

  • Wait for the development release Deployment to become available by running:
kubectl --context kind-dev wait --for=condition=Available deployment/release-demo -n release-demo --timeout=5m

You should see confirmation that the development Deployment met the Available condition.

  • Wait for the production release Deployment to become available by running:
kubectl --context kind-prod wait --for=condition=Available deployment/release-demo -n release-demo --timeout=5m

You should see confirmation that the production Deployment met the Available condition.

Deployment did not become available?

Confirm the repository is public. Check that each root path matches an existing folder in the pushed repository.

A username mismatch in repoURL prevents Argo CD from reading the manifests. Help me diagnose why my Argo CD release-demo Deployment was not created

The port-forward commands keep their Terminal windows busy while they route traffic. Leave both windows running until you finish comparing the pages.

  • Start the development port-forward by running:
kubectl --context kind-dev -n release-demo port-forward service/release-demo 8081:80

This forwards local port 8081 to the development Service. Press Ctrl+C when you later want to stop it.

  • Press Cmd+N to open another Terminal window.
  • Start the production port-forward in the new window by running:
kubectl --context kind-prod -n release-demo port-forward service/release-demo 8082:80

This forwards local port 8082 to the production Service. Keep this Terminal window running too.

Before you open the pages, what environment label do you expect each local port to show?

  • Open http://localhost:8081 in your browser.
  • Confirm the page shows release 1.0.0 with the development environment label.
  • Open http://localhost:8082 in another browser tab.
  • Confirm the page shows release 1.0.0 with the production environment label.

That is the GitOps handoff complete. Both clusters now render their own Git-declared release through separate Argo CD control planes.

Browser page not loading?

Keep each port-forward command running in its own Terminal window. A stopped command closes the matching local connection.

Check that development uses port 8081. Check that production uses port 8082.

If the page still fails, help me troubleshoot my two Kubernetes Service port-forwards

Your shared repository now controls two isolated release pages. Next, you will promote a visible release through Git before exposing an unauthorized production change.

Promote a Release and Expose Drift

Your two release pages now show the same 1.0.0 build from independent Kubernetes clusters. Argo CD keeps each cluster aligned with its own Git path.

A production workflow needs an auditable route from development to production. It also needs to expose direct cluster edits as untracked configuration drift.

In this step, get ready to:
  • Promote release 1.1.0 to development through a Git commit.
  • Promote the tested release to production through a separate Git commit.
  • Patch production directly to reveal configuration drift.
Promote release 1.1.0 to development

A promotion changes the release declared in Git. Argo CD reads that change from the development path without receiving a direct deployment command.

  • In environments/dev/release.yaml, find this heading line:
        <h1>Release 1.0.0</h1>

This heading is the visible release number served by the development workload.

  • Replace the heading with the development 1.1.0 line shown below:
        <h1>Release 1.1.0</h1>

Why Does One Line Promote the Release?

  • The <h1> element makes the promoted version visible in your browser.
  • The development Application watches environments/dev for changes.
  • Keeping every other line unchanged preserves the pinned image plus the existing workload configuration.
  • Save environments/dev/release.yaml.
  • Prepare to enter Promote development to release 1.1.0 when Git opens your configured commit editor.
  • Start the development commit by running these commands:
git add .
git commit

What Do These Commands Do?

  • The first command stages the development file change.
  • The second command opens your configured editor for the commit message.
  • Saving the message creates an auditable snapshot of the development promotion.
  • Enter Promote development to release 1.1.0 in the commit editor.
  • Save the commit message.
  • Close the editor.
  • Publish the development commit by running this command:
git push

What Does This Command Do?

This sends the new commit to the existing origin remote. Argo CD can now discover the development promotion from the repository.

Before you refresh, which environment should move to 1.1.0 first?

  • Switch back to the development release page at http://localhost:8081.
  • Refresh the development page until its heading updates.
  • Switch back to the production release page at http://localhost:8082.
  • Refresh the production page once.

Good progress. Development now displays release 1.1.0.

Production still displays release 1.0.0. The separate repository paths have kept the promotion contained.

Development Still Shows the Old Release?

Confirm that you changed environments/dev/release.yaml. Check that the commit completed before you pushed it.

The mounted ConfigMap updates eventually. Wait briefly before refreshing the development page again.

Help me diagnose why the development release page is not showing the Git-declared version.

✔️ Awesome, I've got everything!

Development now shows the promoted release. Make sure environments/dev/release.yaml is saved.

ⓧ I'd like to double check the full code

apiVersion: v1
kind: ConfigMap
metadata:
  name: release-page
data:
  index.html: |
    <!doctype html>
    <html lang="en">
      <head><meta charset="utf-8"><title>Development Release</title></head>
      <body>
        <h1>Release 1.1.0</h1>
        <p>Environment: development</p>
        <p>Delivered by Argo CD from Git.</p>
      </body>
    </html>
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: release-demo
  template:
    metadata:
      labels:
        app: release-demo
    spec:
      containers:
        - name: nginx
          image: nginx:1.30.4-alpine@sha256:97d490c12ba55b4946b01546d1c3ed324e8d41ab1c9fcb2a616aa470620e5b46
          ports:
            - containerPort: 80
          volumeMounts:
            - name: page
              mountPath: /usr/share/nginx/html
              readOnly: true
      volumes:
        - name: page
          configMap:
            name: release-page
---
apiVersion: v1
kind: Service
metadata:
  name: release-demo
spec:
  selector:
    app: release-demo
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  • Compare this reference with your saved environments/dev/release.yaml file.
Promote the tested release to production

Development now proves that the release page works in its first environment. A separate production commit records the moment that tested release crosses the environment boundary.

  • In environments/prod/release.yaml, find this heading line:
        <h1>Release 1.0.0</h1>

This line keeps production on its current declared release until you explicitly promote it.

  • Replace the heading with the production 1.1.0 line shown below:
        <h1>Release 1.1.0</h1>

Why Use a Separate Production Commit?

The second commit creates a distinct promotion record for production. Your Git history can show when each environment received the release.

  • Save environments/prod/release.yaml.
  • Prepare to enter Promote production to release 1.1.0 when Git opens your configured commit editor.
  • Start the production commit by running these commands:
git add .
git commit

What Happens to the Production Change?

  • The production manifest becomes the only new staged change.
  • The commit captures the production promotion as its own Git snapshot.
  • Enter Promote production to release 1.1.0 in the commit editor.
  • Save the commit message.
  • Close the editor.
  • Publish the production commit by running this command:
git push

What Does This Push Trigger?

The push updates the production path in the shared repository. The production Argo CD control plane can now reconcile release 1.1.0 independently.

Before you refresh production, do you expect its heading to match development now?

  • Switch back to the production page at http://localhost:8082.
  • Refresh the production page until its heading updates.
  • Switch back to the development page at http://localhost:8081.
  • Refresh the development page once.

That is controlled promotion working. Both pages now display release 1.1.0 after separate Git commits.

Production Still Shows Release 1.0.0?

Confirm that the changed file is environments/prod/release.yaml. Check that the production commit reached the remote repository.

Wait briefly for the mounted ConfigMap to update. Refresh http://localhost:8082 again after the Application reconciles.

Help me diagnose why production has not received the promoted Git release.

✔️ Awesome, I've got everything!

Both environments now show the promoted release. Make sure environments/prod/release.yaml is saved.

ⓧ I'd like to double check the full code

apiVersion: v1
kind: ConfigMap
metadata:
  name: release-page
data:
  index.html: |
    <!doctype html>
    <html lang="en">
      <head><meta charset="utf-8"><title>Production Release</title></head>
      <body>
        <h1>Release 1.1.0</h1>
        <p>Environment: production</p>
        <p>Delivered by Argo CD from Git.</p>
      </body>
    </html>
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: release-demo
  template:
    metadata:
      labels:
        app: release-demo
    spec:
      containers:
        - name: nginx
          image: nginx:1.30.4-alpine@sha256:97d490c12ba55b4946b01546d1c3ed324e8d41ab1c9fcb2a616aa470620e5b46
          ports:
            - containerPort: 80
          volumeMounts:
            - name: page
              mountPath: /usr/share/nginx/html
              readOnly: true
      volumes:
        - name: page
          configMap:
            name: release-page
---
apiVersion: v1
kind: Service
metadata:
  name: release-demo
spec:
  selector:
    app: release-demo
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  • Compare this reference with your saved environments/prod/release.yaml file.
Create an out-of-band production hotfix

Production currently matches the release declared in Git. A direct patch changes the live Kubernetes ConfigMap without changing the repository.

Before you apply the patch, do you think the live page can change while Git remains untouched?

  • Patch the live production ConfigMap by running this command:
kubectl --context kind-prod -n release-demo patch configmap release-page --type merge -p '{"data":{"index.html":"<h1>Emergency production hotfix outside Git</h1>"}}'

What Does This Command Change?

  • The --context kind-prod option targets the production cluster.
  • The -n release-demo option targets the workload namespace.
  • The merge patch replaces the live index.html value inside the release-page ConfigMap.
  • Switch back to the production page at http://localhost:8082.
  • Refresh the production page until the hotfix heading appears.

You will see Emergency production hotfix outside Git on the production page.

That mismatch is the intended shortfall. Git still declares the full production release page while the cluster serves the direct hotfix.

Before you ask Argo CD, do you expect it to treat the live hotfix as aligned with Git?

  • Check the production child Application by running this command:
kubectl --context kind-prod -n argocd get application release-demo

What Does This Status Prove?

The production Application reports OutOfSync. Argo CD has compared the live ConfigMap with the repository declaration.

The hotfix remains because selfHeal is still false for the production child Application. Argo CD exposes the drift without overwriting it.

  • Refresh the development page at http://localhost:8081.
  • Refresh the production page at http://localhost:8082.

Development still shows release 1.1.0. Production shows the emergency hotfix while its Application remains OutOfSync.

Drift Is Not Visible Yet?

If the page still shows release 1.1.0, wait briefly for the mounted ConfigMap to update. Refresh the production page again.

If the Application remains aligned, confirm that selfHeal is still false in clusters/prod/apps/release-demo.yaml.

Help me diagnose why the production hotfix is not producing visible Argo CD drift.

You have promoted one release through two Git-controlled environments. Next, you will turn drift detection into automatic self-healing.

Enable GitOps Self-Healing

You have promoted release 1.1.0 through Git. Production still displays the direct hotfix because self-healing is disabled.

GitOps self-healing gives Argo CD permission to replace that live drift with the declared files in Git. Git becomes the enforceable source of truth for production.

In this step, get ready to:
  • Enable self-healing in both child Application manifests.
  • Commit the policy change to Git.
  • Verify that Argo CD removes the production hotfix.
Enable self-healing in both Applications

The child Application manifests already automate synchronization and pruning. Setting selfHeal to true authorizes Argo CD to reconcile live drift.

  • Open the existing local argo-two-cluster-gitops repository in your code editor.
  • Select clusters/dev/apps/release-demo.yaml in your code editor's file browser.
  • Locate selfHeal: false under the automated sync policy.
  • Replace false with true.
  • Confirm the development sync policy matches this reference:
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

What does this policy control?

  • The enabled: true setting keeps automated synchronization active.
  • The prune: true setting removes managed resources that disappear from Git.
  • The selfHeal: true setting lets Argo CD reverse live changes that differ from Git.
  • The CreateNamespace=true option keeps automatic namespace creation enabled.
  • Save clusters/dev/apps/release-demo.yaml.

✔️ Awesome, I've got everything!

Good work. The development child Application now has Git-managed self-healing enabled.

ⓧ I'd like to double check the full code

  • Compare your saved clusters/dev/apps/release-demo.yaml file with this reference:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: release-demo
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-github-username/argo-two-cluster-gitops.git
    targetRevision: main
    path: environments/dev
  destination:
    server: https://kubernetes.default.svc
    namespace: release-demo
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

What should you compare?

  • The child Application still targets the development environment.
  • The automated policy now contains selfHeal: true.
  • The destination remains the release-demo namespace in the local cluster.
  • Select clusters/prod/apps/release-demo.yaml in your code editor's file browser.
  • Locate selfHeal: false under the automated sync policy.
  • Replace false with true.
  • Save clusters/prod/apps/release-demo.yaml.

✔️ Awesome, I've got everything!

The production child Application now carries the same self-healing policy as development.

ⓧ I'd like to double check the full code

  • Compare your saved clusters/prod/apps/release-demo.yaml file with this reference:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: release-demo
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: https://github.com/your-github-username/argo-two-cluster-gitops.git
    targetRevision: main
    path: environments/prod
  destination:
    server: https://kubernetes.default.svc
    namespace: release-demo
  syncPolicy:
    automated:
      enabled: true
      prune: true
      selfHeal: true
    syncOptions:
      - CreateNamespace=true

What should you compare?

  • The child Application still targets the production environment.
  • The automated policy now contains selfHeal: true.
  • The source repository remains the shared public GitOps repository.
Commit the policy to Git

The local edits do not affect either cluster until they reach the public repository. A commit records the policy change before Argo CD applies it.

  • Open your code editor's integrated terminal for the existing argo-two-cluster-gitops repository.
  • Create the self-healing policy commit by running this command:
git add clusters && git commit -m "Enable GitOps self-healing"

What does this command do?

  • The git add clusters command stages both child Application changes.
  • The commit saves those staged changes with an auditable policy message.

You should see a new commit named Enable GitOps self-healing with two changed files.

Commit not created?

Confirm both Application files are saved. Each file must contain selfHeal: true.

Make sure the terminal is using your existing argo-two-cluster-gitops repository.

Help me diagnose why Git cannot commit my two self-healing changes.

  • Publish the policy commit to the shared repository by running this command:
git push origin main

What does this push trigger?

The push updates the declared state on the remote main branch. Each root Application can now deliver the updated child policy to its own cluster.

You should see the remote main branch accept the new commit. The repository now carries the policy that repairs drift.

Push not reaching the repository?

Confirm your internet connection is active. Check that the terminal is still authenticated with the GitHub account that owns the repository.

Help me troubleshoot why my self-healing commit will not push to the main branch.

Watch production converge on Git

Argo CD now compares the live production ConfigMap with its declared version in Git. It replaces the direct hotfix without a repair command against the workload.

This convergence may need several refreshes while Kubernetes updates the mounted ConfigMap. The existing production port-forward keeps the result visible in your browser.

Before you check, do you expect production to remain OutOfSync or converge to Git?

  • Check the production child Application by running this command:
kubectl --context kind-prod -n argocd get application release-demo

What does this check prove?

The command reads the release-demo child Application from the production Argo CD control plane. When reconciliation finishes, the Application settles on Synced.

  • Repeat the status check until the Application reports Synced.

Argo CD has now accepted the Git policy. The direct production drift is reconciled.

  • Refresh the existing production page at http://localhost:8082.

You will see Release 1.1.0 above Environment: production. The emergency hotfix text is gone.

  • Refresh the existing development page at http://localhost:8081.

You will still see Release 1.1.0 above Environment: development.

Production still shows the hotfix?

Refresh the production page several times. Mounted ConfigMap content updates after Kubernetes propagates the reconciled data.

Confirm the production Application manifest contains selfHeal: true and that the policy commit reached the remote repository.

Help me find why production remains OutOfSync after enabling self-healing.

That closes the drift gap: production now converges on Git without a direct repair command. Next, you will test the same workflow against a faulty production commit.

Recover Production Through Git

Production is healthy again because Argo CD reconciled the direct ConfigMap hotfix back to the Git declaration. This confirms that self-healing protects the declared production state.

Automated sync can also deliver an unsafe Git change quickly. Recovery needs an auditable repository change so the cluster converges on the declared state again.

In this step, get ready to:
  • Commit an intentionally invalid production image.
  • Observe the failed production rollout.
  • Recover production through an auditable Git commit.
Commit a faulty production image

Production currently uses a digest-pinned NGINX image. Changing that declaration gives automated sync a faulty desired state to apply.

  • In the repository from earlier, select environments/prod/release.yaml from your editor's file tree.
  • Find the current production image line:
          image: nginx:1.30.4-alpine@sha256:97d490c12ba55b4946b01546d1c3ed324e8d41ab1c9fcb2a616aa470620e5b46

What are you locating?

This line pins the production container to an exact image digest. Git currently declares this image as the production source of truth.

  • Replace the current image line with the intentionally invalid image reference below:
          image: nginx:1.161

Why use this image?

The nginx:1.161 image reference is deliberately invalid. Committing it tests how your automated production workflow behaves when Git contains an unsafe declaration.

  • Save environments/prod/release.yaml.
  • Publish the faulty production declaration by running these commands:
git add environments/prod/release.yaml
git commit -m "Deploy faulty production image"
git push origin main

What do these commands do?

  • The first command stages only the production release manifest.
  • The second command records the faulty desired state in repository history.
  • The third command publishes that commit to the remote GitHub repository.

Your terminal confirms that the new commit reached the remote repository. Argo CD can now reconcile the faulty declaration.

Did the push fail?

  • Confirm your terminal is inside the argo-two-cluster-gitops repository.
  • Complete GitHub authentication again if your previous session has expired.

Ask for help with the failed publish: help me diagnose why my Git push failed in the argo-two-cluster-gitops repository.

Observe the failed rollout

Argo CD now treats the faulty commit as the desired production state. Kubernetes attempts to replace the running Pod using that image declaration.

Take a moment to predict the Pod state that the new image produces.

  • Inspect the production Pods and child Application by running these commands:
kubectl --context kind-prod -n release-demo get pods
kubectl --context kind-prod -n argocd get application release-demo

What do these checks reveal?

  • The first command lists the Pods created by the production release-demo Deployment.
  • The second command reports the health of the production child Application.

You will see a new Pod in ImagePullBackOff. The previous replica remains available under the Deployment's rolling update behavior.

The production child Application also reflects the unhealthy rollout. This is the visible consequence of treating every Git commit as deployable desired state.

You should still see production release 1.1.0 because the previous replica remains available. That is the failure evidence you needed: the new rollout is broken while the last working replica continues serving the page.

Do you not see the failed Pod?

  • Wait for Argo CD to process the pushed commit.
  • Run the production Pod check again.
  • Confirm the faulty commit appears in your remote repository.

Get help tracing the reconciliation: help me find why the faulty production image has not reached my kind-prod cluster.

Revert the faulty Git commit

A direct cluster repair would leave Git declaring the broken image. A Git revert creates a new commit that reverses the faulty declaration while preserving the incident in repository history.

Argo CD consumes that recovery commit through the same automated path as any other release. Production returns to the pinned image without a direct rollout rollback.

  • Create the auditable recovery commit by running this command:
git revert --no-edit HEAD

What does this command do?

This command creates a new commit that reverses the current HEAD commit. The --no-edit option accepts the generated revert message without opening an editor.

  • Publish the recovery commit by running this command:
git push origin main

What happens after the push?

The recovery commit restores the pinned production image in Git. Argo CD detects the new declaration and starts replacing the failed Pod automatically.

✔️ Awesome, I've got everything!

The revert has restored the final production manifest. Your Git history still contains both the faulty commit and its recovery commit.

ⓧ I'd like to double check the full code

apiVersion: v1
kind: ConfigMap
metadata:
  name: release-page
data:
  index.html: |
    <!doctype html>
    <html lang="en">
      <head><meta charset="utf-8"><title>Production Release</title></head>
      <body>
        <h1>Release 1.1.0</h1>
        <p>Environment: production</p>
        <p>Delivered by Argo CD from Git.</p>
      </body>
    </html>
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: release-demo
  template:
    metadata:
      labels:
        app: release-demo
    spec:
      containers:
        - name: nginx
          image: nginx:1.30.4-alpine@sha256:97d490c12ba55b4946b01546d1c3ed324e8d41ab1c9fcb2a616aa470620e5b46
          ports:
            - containerPort: 80
          volumeMounts:
            - name: page
              mountPath: /usr/share/nginx/html
              readOnly: true
      volumes:
        - name: page
          configMap:
            name: release-page
---
apiVersion: v1
kind: Service
metadata:
  name: release-demo
spec:
  selector:
    app: release-demo
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

Before you check, predict whether production has converged on the recovery commit yet.

  • Verify the production Pod and both child Applications by running these commands:
kubectl --context kind-prod -n release-demo get pods
kubectl --context kind-prod -n argocd get application release-demo
kubectl --context kind-dev -n argocd get application release-demo

What proves recovery?

  • The production replacement Pod reaches Running.
  • The production child Application reports Healthy and Synced.
  • The development child Application remains Healthy and Synced.

You should see production release 1.1.0 served from the restored pinned image. You have recovered production through Git while preserving an auditable record of the failure.

Is production still recovering?

  • Confirm the recovery commit reached the remote repository.
  • Run the production Pod check again after Argo CD processes the commit.
  • Compare environments/prod/release.yaml with the full-code tab above.

Get help tracing the recovery: help me diagnose why production has not returned to a Running Pod with a Healthy and Synced Application.

That is production recovery working as designed. Git records the failure plus its reversal while Argo CD restores the declared release automatically.

Secret mission

Lock Production to an AppProject

Production is healthy, but its child Application still uses Argo CD's permissive default project. Add a production AppProject that limits the approved repository, destination, namespace, and cluster-scoped resources.

Clean Up Your Resources

Clean Up Your Resources

Your local Kubernetes clusters and public GitHub repository cost $0 for personal learning under Docker Personal and GitHub Free. Choose whether to keep them running, pause the local processes, or delete the resources entirely.

Cost warning

Qualifying larger enterprises and government entities need a paid Docker Desktop subscription. Personal use and education remain covered by Docker Personal.

Resources you used:

  • Two local Kubernetes clusters named dev and prod, containing both Argo CD control planes and the release workloads.
  • Two local port-forward processes serving development on http://localhost:8081 and production on http://localhost:8082.
  • The public argo-two-cluster-gitops GitHub repository.

Keep everything running

No action is needed. Choose this option if you want both release pages to remain available while you continue experimenting.

  • Keep Docker Desktop running so both local clusters remain available.
  • Keep both port-forward terminals running so the development and production pages remain accessible.
  • Retain the public argo-two-cluster-gitops repository so Git remains the desired state for both clusters.

Pause - I'll come back to this later

Shut down the running processes to free local memory. Your cluster data and Git repository remain available for your return.

  • Stop the development port-forward by pressing Ctrl+C in its terminal.
  • Stop the production port-forward by pressing Ctrl+C in its terminal.
  • Quit Docker Desktop from the macOS menu bar.

Both port-forward terminals return to a shell prompt. Docker Desktop releases the cluster memory while the local data remains on disk.

Delete - I don't want to use this again

This option removes the local platform for a fresh start. Deleting both clusters looks drastic, but the Git repository stays untouched until you choose to remove it.

  • Stop the development port-forward by pressing Ctrl+C in its terminal.
  • Stop the production port-forward by pressing Ctrl+C in its terminal.
  • Delete both local clusters by running these commands:
kind delete cluster --name dev
kind delete cluster --name prod

What do these commands remove?

Each command removes one local cluster. Deleting both clusters also removes their Argo CD control planes and release workloads.

  • Confirm that both clusters are gone by running this command:
kind get clusters

What should I see?

The cluster list should no longer include dev or prod. This proves both local environments are gone.

Repository deletion is permanent. Keep it if you want the Git history for future reference.

  • Return to the public argo-two-cluster-gitops repository from earlier.
  • Open the repository settings.
  • Use the repository deletion controls to permanently delete argo-two-cluster-gitops.
  • Confirm that argo-two-cluster-gitops no longer appears in your GitHub repositories.
  • Quit Docker Desktop from the macOS menu bar.

Nice Work!

You made it! Your two-cluster Kubernetes GitOps platform now runs independent Argo CD control planes for development and production.

You've learned how to:

  • Bootstrap isolated environments from one Git repository using the App of Apps pattern.
  • Promote visible release 1.1.0 from development to production through auditable commits.
  • Restore the Git-declared production state after unauthorized drift plus a faulty image update.
  • Secret Mission: Lock production to the release-demo-prod AppProject. Its guardrails restrict the repository plus the deployment destination. Only the required cluster-scoped Namespace resource remains permitted.

Ready to quiz yourself?