Federate Kubernetes Clusters with Karmada

Introduction

30 Second Summary

When companies run their applications across multiple cloud providers, they gain resilience and avoid vendor lock-in. The real challenge is managing deployments across all of them without losing your mind.

In this project, you will build a local multi-cluster Kubernetes federation using Karmada. You will deploy an application once to a central control plane and use policies to distribute, customize, and failover workloads across multiple clusters.

What You'll Build

A local multi-cloud Kubernetes simulation where you operate as a platform engineer managing workloads across multiple clusters from a single Karmada control plane.

By the end of this project, you'll have:

• A Karmada control plane running on a local kind cluster with two member clusters registered and federated.
• A working nginx deployment propagated across both clusters using a PropagationPolicy.
• Per-cluster customization using OverridePolicy to set different replica counts per cluster, with verified automated failover when a cluster goes offline.
• Secret Mission: Implement weighted replica scheduling with a 70/30 split across clusters.

Are there any prerequisites?

This project requires a Linux machine (physical or virtual) with at least 8GB of RAM. You will run three Docker-based clusters simultaneously, which needs that memory headroom.

No prior Kubernetes experience is required, but familiarity with the terminal and running commands will help you move faster.

Install Docker and Kubernetes Tools

Before you can run multiple Kubernetes clusters on your machine, you need a container runtime and a set of CLI tools that create and manage those clusters.

In this step, you will install four tools: Docker (the container engine), kubectl (the Kubernetes command-line interface), kind (which creates lightweight clusters inside Docker containers), and karmadactl (the Karmada federation CLI).

In this step, get ready to:

• Install Docker Engine and configure it for non-root access.
• Install kubectl and kind for creating local Kubernetes clusters.
• Install karmadactl for managing the Karmada control plane.

Install Docker Engine

Docker is the container runtime that kind uses to simulate Kubernetes nodes. Each "node" in your cluster is actually a Docker container running on your machine.

• Download the official Docker install script by running this command:

curl -fsSL https://get.docker.com -o get-docker.sh

• Run the install script with sudo:

sudo sh get-docker.sh

What does this script do?

The convenience script detects your Linux distribution automatically and installs Docker Engine along with its dependencies. It works across Ubuntu, Debian, Fedora, CentOS, and RHEL.

• Add your user to the docker group so you can run Docker without sudo:

sudo usermod -aG docker $USER

• Activate the new group membership in your current session by running:

newgrp docker

Why do I need the docker group?

By default, Docker requires root privileges. Adding yourself to the docker group lets you run Docker commands without prefixing sudo every time.

This is required for kind to work properly, since kind creates clusters by talking to the Docker daemon directly.

• Verify Docker works by running the hello-world container:

docker run hello-world

✔️ I see 'Hello from Docker!'

Docker is installed and working correctly. Move on to the next substep.

ⓧ I see a permission denied error

The group change has not taken effect yet. Try logging out of your terminal session completely and logging back in.

After logging back in, run docker run hello-world again.

ⓧ Command not found or connection error

Docker may not have installed properly or the daemon is not running. Start the Docker daemon with:

sudo systemctl start docker

Then run docker run hello-world again. If you still see "command not found", re-run the install script above.

Still stuck? Help me troubleshoot my Docker installation

Install kubectl and kind

kubectl lets you interact with any Kubernetes cluster. kind creates lightweight clusters inside Docker containers, perfect for local development and testing.

• Download and install kubectl by running these commands:

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

What do these commands do?

• The first line downloads the latest stable kubectl binary for Linux.
• The second line installs it to /usr/local/bin/ with the correct permissions so any user can run it.

• Verify kubectl is installed by running:

kubectl version --client

✔️ I see a client version number

kubectl is installed and on your PATH. Move on to installing kind below.

ⓧ Command not found

Make sure /usr/local/bin is in your PATH. Check by running:

echo $PATH

If /usr/local/bin is missing from the output, add it with:

export PATH=$PATH:/usr/local/bin

Then run kubectl version --client again to confirm.

Need more help? Help me get kubectl working

• Now install kind v0.32.0 by running these commands:

[ $(uname -m) = x86_64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.32.0/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind

What do these commands do?

• The first line downloads the kind binary for your architecture (x86_64).
• The second line makes it executable.
• The third line moves it to a system-wide location on your PATH.

• Verify kind is installed at the correct version by running:

kind version

✔️ I see v0.32.0

kind is installed at the correct version. Move on to the next substep.

ⓧ I see an older version

You have an older version of kind installed. Re-run the download commands to replace it with v0.32.0:

[ $(uname -m) = x86_64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.32.0/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind

Run kind version again to confirm it now shows v0.32.0.

ⓧ Command not found

The binary may not be on your PATH. Verify the file exists:

ls /usr/local/bin/kind

If the file is not there, re-run the three installation commands above. If it is there but still not found, ensure /usr/local/bin is in your PATH.

Still having trouble? Help me install kind v0.32.0

Install karmadactl

karmadactl is the CLI for managing Karmada federations. You will use it to initialize the control plane and join member clusters in the next step.

• Install karmadactl v1.17.2 using the official install script:

curl -s https://raw.githubusercontent.com/karmada-io/karmada/master/hack/install-cli.sh | sudo INSTALL_CLI_VERSION=1.17.2 bash

What does this script do?

• It detects your operating system and architecture.
• It downloads the karmadactl binary for version 1.17.2 from the Karmada GitHub releases.
• It installs the binary to /usr/local/bin/karmadactl.

• Verify karmadactl is installed at the correct version by running:

karmadactl version

✔️ I see v1.17.2

karmadactl is installed at the correct version. You are ready to proceed.

ⓧ I see an older version

You have a different version installed. Re-run the install script with the pinned version to overwrite it:

curl -s https://raw.githubusercontent.com/karmada-io/karmada/master/hack/install-cli.sh | sudo INSTALL_CLI_VERSION=1.17.2 bash

Run karmadactl version again to confirm it now shows v1.17.2.

ⓧ Command not found

The install script may have failed silently. Check if the binary exists:

ls /usr/local/bin/karmadactl

If the file is not there, re-run the install script. Make sure you have an internet connection and that curl is installed on your system.

Need help? Help me install karmadactl v1.17.2

All four tools are installed and verified. Next up, you will use them to create a multi-cluster Kubernetes environment with a Karmada control plane.

Create the Karmada Control Plane and Member Clusters

With all your tools installed, you can now build the multi-cluster infrastructure that simulates a real multi-cloud environment.

In this step, you will create three kind clusters and wire them together under a single Karmada control plane. One cluster hosts the Karmada API server, and the other two act as member clusters representing different cloud providers.

In this step, get ready to:

• Create a host cluster and initialize the Karmada control plane on it.
• Create two member clusters and register them with Karmada.
• Explore the cluster topology from the Karmada API server.

Create the host cluster and initialize Karmada

The Karmada control plane needs a Kubernetes cluster to run on. This is the "host cluster" that holds the Karmada API server, scheduler, and controllers.

• Create the host cluster by running this command:

kind create cluster --name karmada-host

What does this command do?

This creates a lightweight Kubernetes cluster running inside a Docker container, named karmada-host. The kind tool uses kubeadm under the hood to bootstrap the cluster and automatically sets your kubectl context to kind-karmada-host.

• Wait for the cluster creation to finish. You should see a message confirming the kubectl context is set.

Cluster creation failed?

If you see a Docker-related error, confirm Docker is running with docker ps. If you get a permissions error, make sure you completed the newgrp docker step from Step 1.

If the cluster times out during creation, your machine may be low on resources. Close other applications and try again.

Help me troubleshoot my kind cluster creation failure.

Now that the host cluster is running, you can initialize the Karmada control plane on it.

• Initialize Karmada on the host cluster by running:

sudo karmadactl init

What does this command do?

This deploys the full Karmada control plane (API server, controller manager, scheduler, webhook, etcd) as pods inside the karmada-host cluster. It requires sudo because it writes configuration files and certificates to /etc/karmada/.

The most important file it creates is /etc/karmada/karmada-apiserver.config. This is the kubeconfig that lets you talk directly to the Karmada API server.

• Wait for the initialization to complete. This may take a few minutes as it downloads container images and starts multiple components.
• Once done, verify the Karmada components are running by checking pods in the karmada-system namespace:

kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get pods -n karmada-system

• You should see pods for karmada-apiserver, karmada-controller-manager, karmada-scheduler, karmada-webhook, and etcd in a Running state.

Pods not in Running state?

If pods are stuck in ContainerCreating or Pending, wait another minute. The images may still be downloading. Re-run the command to check again.

Help me debug why my Karmada pods are not starting.

Create and register member clusters

Member clusters represent the "clouds" in your federation. You will create two kind clusters to simulate an AWS cluster and a GCP cluster.

• Create the first member cluster (simulating AWS) by running:

kind create cluster --name member1

• Create the second member cluster (simulating GCP) by running:

kind create cluster --name member2

What's happening with kubeconfig?

Each kind create cluster command adds a new context to your ~/.kube/config file. After creating both member clusters, your kubeconfig now has three contexts: kind-karmada-host, kind-member1, and kind-member2.

With both member clusters running, you can register them with the Karmada control plane using Push mode.

• Register member1 with Karmada by running:

sudo karmadactl join member1 --kubeconfig=/etc/karmada/karmada-apiserver.config --cluster-kubeconfig=$HOME/.kube/config --cluster-context=kind-member1

• Register member2 with Karmada by running:

sudo karmadactl join member2 --kubeconfig=/etc/karmada/karmada-apiserver.config --cluster-kubeconfig=$HOME/.kube/config --cluster-context=kind-member2

What does karmadactl join do?

The karmadactl join command registers a cluster to the Karmada control plane in Push mode. In this mode, the Karmada control plane pushes workloads directly to the member clusters.

The --kubeconfig flag points to the Karmada API server config. The --cluster-kubeconfig flag points to the member cluster's kubeconfig file. The --cluster-context flag tells Karmada which context in that file corresponds to the member cluster.

• Verify both clusters are registered and ready by running:

kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters

• You should see both member1 and member2 listed with Push mode and True in the READY column.

Clusters not showing as Ready?

If a cluster shows False in the READY column, wait 30 seconds and run the command again. The cluster health check takes a moment to propagate.

If the join command itself failed, double-check the context name matches exactly. Run kubectl config get-contexts to see all available contexts.

Help me debug why my member clusters are not Ready in Karmada.

Explore the cluster topology

You now have a working hub-spoke architecture. The Karmada API server is the hub, and both member clusters are the spokes.

• View the registered clusters and their details from the Karmada control plane:

kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters

What is the hub-spoke architecture?

In this topology, you interact with a single API server (the Karmada API server) to manage workloads across all member clusters. You never deploy directly to member clusters. Instead, you submit manifests to the Karmada API server and use policies to control where workloads land.

This is the same pattern used in production multi-cloud setups. One control plane governs multiple clusters across different providers, regions, or availability zones.

• To see each cluster individually, you can switch kubectl contexts. View member1 by running:

kubectl --context kind-member1 get nodes

• View member2 by running:

kubectl --context kind-member2 get nodes

• Each command should show a single node (the kind container acting as a Kubernetes node) in a Ready state.

Your multi-cluster federation is up and running. Next up, you will deploy an application to this control plane and use policies to propagate it across both member clusters.

Deploy Across Clusters with PropagationPolicy

Your clusters are federated and ready. Now it is time to deploy an actual application across them.

In a traditional single-cluster setup, you apply a Deployment directly to the cluster and pods start running. With Karmada, you apply the Deployment to the control plane instead. Karmada then distributes it to member clusters based on policies you define.

In this step, get ready to:

• Create an nginx Deployment on the Karmada control plane.
• Write and apply a PropagationPolicy to distribute it across clusters.
• Verify the deployment is running on both member clusters.

Create an nginx Deployment on the Karmada control plane

In Karmada's model, you submit workloads to the karmada-apiserver. The control plane holds the manifest as a template until a PropagationPolicy tells it where to send it.

• Create a file called deployment.yaml by running:

cat <<EOF > deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.27
        ports:
        - containerPort: 80
EOF

What does this manifest do?

This is a standard Kubernetes Deployment. It creates 2 replicas of an nginx:1.27 container listening on port 80.

There is nothing Karmada-specific about this file. You use the exact same manifests you already know from single-cluster Kubernetes.

• Apply the Deployment to the karmada-apiserver by running:

kubectl --kubeconfig /etc/karmada/karmada-apiserver.config apply -f deployment.yaml

• Check the deployment status by running:

kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get deployments

You should see the nginx deployment listed, but with 0 pods ready. Without a PropagationPolicy, Karmada holds the manifest idle on the control plane.

Why are there zero ready pods?

Karmada intentionally does not auto-distribute workloads. A Deployment posted to the karmada-apiserver without a matching PropagationPolicy sits idle. This design gives you explicit control over which clusters receive which workloads.

Seeing permission denied?

If you get an error accessing /etc/karmada/karmada-apiserver.config, the file may be owned by root. Run sudo chmod 644 /etc/karmada/karmada-apiserver.config to make it readable.

help me fix the permission error when accessing the karmada-apiserver kubeconfig

Write and apply a PropagationPolicy

A PropagationPolicy is Karmada's way of mapping workloads to clusters. You define which resources to target and which clusters should receive them.

• Create a file called propagationpolicy.yaml by running:

cat <<EOF > propagationpolicy.yaml
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: nginx-propagation
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: nginx
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2
EOF

What does this policy do?

The resourceSelectors field tells Karmada which workloads this policy applies to. Here it targets the nginx Deployment specifically.

The placement.clusterAffinity.clusterNames field lists which member clusters should receive the workload. Karmada will push the full Deployment (all 2 replicas) to both member1 and member2.

• Apply the PropagationPolicy to the karmada-apiserver by running:

kubectl --kubeconfig /etc/karmada/karmada-apiserver.config apply -f propagationpolicy.yaml

Karmada immediately begins propagating the nginx Deployment to both member clusters.

• Wait a few seconds, then check the deployment status again by running:

kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get deployments

You should now see 2/2 pods ready. The karmada-apiserver aggregates status from all member clusters.

Still seeing 0 ready pods?

Give it 30-60 seconds for the member clusters to pull the nginx:1.27 image. If it stays at 0, confirm your policy file has no typos by checking that name: nginx under resourceSelectors matches the Deployment name exactly.

help me debug why my PropagationPolicy is not distributing pods

Verify deployment on member clusters

The karmada-apiserver shows aggregated status, but you can also inspect each member cluster directly to confirm pods are running.

• Check nginx pods on member1 by running:

kubectl --context kind-member1 get pods

You should see 2 nginx pods in a Running state.

• Check nginx pods on member2 by running:

kubectl --context kind-member2 get pods

You should see 2 nginx pods running here as well. Both clusters received the full Deployment.

• Return to the karmada-apiserver view and check the aggregated status by running:

kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get deployments nginx -o wide

Why verify from multiple perspectives?

The karmada-apiserver gives you a single-pane-of-glass view across all clusters. Checking individual member clusters confirms that the workload was actually scheduled and is running. In production, you would rely on the aggregated view. For learning, seeing both levels builds confidence that federation is working.

Pods not showing on a member cluster?

Make sure you are using the correct context name. Kind clusters use the pattern kind-<cluster-name>, so kind-member1 and kind-member2. List available contexts by running kubectl config get-contexts.

help me find the right kubectl context for my kind member cluster

✔️ Awesome, I've got everything!

Great. Double check you have both deployment.yaml and propagationpolicy.yaml saved.

ⓧ I'd like to double check the full code

Here is deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.27
        ports:
        - containerPort: 80

Here is propagationpolicy.yaml:

apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
  name: nginx-propagation
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: nginx
  placement:
    clusterAffinity:
      clusterNames:
        - member1
        - member2

Your application is now running across two clusters from a single deployment command. Next up, you will customize the deployment per-cluster using OverridePolicy to give each cluster a different replica count.

Customize Per-Cluster with OverridePolicy

Your nginx deployment is running identically on both member clusters. In production, clusters often serve different purposes.

Karmada's OverridePolicy lets you customize deployments per-cluster without changing the original manifest. In this step, you'll give member1 more replicas (simulating a primary cloud) while keeping member2 lean.

In this step, get ready to:

• Write an OverridePolicy that sets different replica counts per cluster.
• Apply the policy and verify each cluster runs the correct number of pods.

Write an OverridePolicy for per-cluster customization

An OverridePolicy intercepts resources as they propagate to member clusters and modifies specific fields. You define which clusters get which changes, and the original manifest on the control plane stays untouched.

• Create a new file called overridepolicy.yaml with the following content:

apiVersion: policy.karmada.io/v1alpha1
kind: OverridePolicy
metadata:
  name: member1-override
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: nginx
  overrideRules:
    - targetCluster:
        clusterNames:
          - member1
      overriders:
        plaintext:
          - path: "/spec/replicas"
            operator: replace
            value: 3
    - targetCluster:
        clusterNames:
          - member2
      overriders:
        plaintext:
          - path: "/spec/replicas"
            operator: replace
            value: 1

What does this policy do?

This single OverridePolicy contains two override rules that target different clusters.

• resourceSelectors picks the nginx Deployment as the resource to modify.
• The first overrideRules entry targets member1 and replaces /spec/replicas with 3.
• The second entry targets member2 and sets replicas to 1.
• The plaintext overrider works like kubectl patch. It modifies any field by JSON path.

Why different replica counts per cluster?

In real multi-cloud setups, a primary region (like AWS) might handle 75% of traffic while a secondary region (like GCP) handles the rest. You give the primary cluster more capacity and keep the secondary lighter for cost efficiency.

• Save overridepolicy.yaml.
• Confirm the file exists by running:

ls overridepolicy.yaml

You should see overridepolicy.yaml in the output.

File not showing up?

• Make sure you saved the file in the same directory where you created deployment.yaml and propagationpolicy.yaml.
• Check for typos in the filename. It must be exactly overridepolicy.yaml.

Help me find my overridepolicy.yaml file

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before continuing.

ⓧ I'd like to double check the full code

apiVersion: policy.karmada.io/v1alpha1
kind: OverridePolicy
metadata:
  name: member1-override
spec:
  resourceSelectors:
    - apiVersion: apps/v1
      kind: Deployment
      name: nginx
  overrideRules:
    - targetCluster:
        clusterNames:
          - member1
      overriders:
        plaintext:
          - path: "/spec/replicas"
            operator: replace
            value: 3
    - targetCluster:
        clusterNames:
          - member2
      overriders:
        plaintext:
          - path: "/spec/replicas"
            operator: replace
            value: 1

Apply and verify overrides

• Apply the OverridePolicy to the Karmada control plane by running:

kubectl apply -f overridepolicy.yaml

You should see output confirming the policy was created:

overridepolicy.policy.karmada.io/member1-override created

What just happened?

Karmada received the OverridePolicy and immediately applied the overrides to the already-propagated nginx Deployment. member1 is now scaling up to 3 replicas and member2 is scaling down to 1.

• Verify member1 now runs 3 nginx pods by running:

kubectl get pods -l app=nginx --context kind-member1

You should see 3 nginx pods all in Running status.

• Verify member2 now runs only 1 nginx pod by running:

kubectl get pods -l app=nginx --context kind-member2

You should see exactly 1 nginx pod in Running status.

• Confirm the original Deployment on the karmada-apiserver still shows 2 replicas by running:

kubectl get deployment nginx

The READY column shows 2/2. The source manifest on the control plane is unchanged.

Why does the control plane still show 2 replicas?

Karmada treats the Deployment on the karmada-apiserver as the "source of truth." OverridePolicy modifies the copies that land on member clusters, not the original. This means you can always see the intended baseline by checking the control plane, regardless of per-cluster customizations.

Not seeing the right number of pods?

• Wait 10-15 seconds after applying the policy. Karmada needs a moment to reconcile the override with the existing deployment.
• Make sure you applied the OverridePolicy to the karmada-apiserver context, not directly to a member cluster.
• Check the OverridePolicy exists with kubectl get overridepolicy. You should see member1-override listed.

Help me debug why my OverridePolicy isn't changing replica counts

You now have per-cluster customization working. member1 runs 3 replicas (simulating a high-capacity primary cloud) while member2 runs just 1 (a lean secondary). Next up, you'll test what happens when one of those clusters goes offline entirely.

Test Multi-Cluster Failover

Your multi-cluster deployment is live with customized replica counts across both clusters. But what happens when one of those clusters goes down?

In production multi-cloud environments, cluster failures are inevitable. The real question is whether your platform can automatically recover without human intervention. In this step, you will simulate a cluster failure and watch Karmada automatically reschedule workloads to the surviving cluster.

In this step, get ready to:

• Simulate a cluster failure by stopping the member1 Docker container.
• Observe Karmada automatically rescheduling workloads to member2.
• Restore the failed cluster and watch workloads rebalance.

Simulate a cluster failure

Each kind cluster runs as a Docker container on your machine. Stopping that container simulates the cluster going offline, just like an entire cloud region becoming unavailable.

• Stop the member1 cluster by running this command:

docker stop member1-control-plane

What does this command do?

The docker stop command halts the Docker container named member1-control-plane. This is the container that runs the entire member1 Kubernetes cluster. From Karmada's perspective, member1 just disappeared from the network.

• Wait about 30-40 seconds for Karmada to detect the failure, then check the cluster status by running:

kubectl get clusters

You should see member1's READY column change from True to False. The member2 cluster remains True.

How does Karmada detect the failure?

Karmada uses a lease-based health check to monitor member clusters. When a cluster stops responding, Karmada applies cluster-level taints (cluster.karmada.io/not-ready or cluster.karmada.io/unreachable) to mark it as unhealthy.

This works just like how Kubernetes taints unhealthy Nodes. Once a cluster is tainted, Karmada's scheduler evicts workloads from it and reschedules them elsewhere.

Still seeing Ready: True for member1?

Karmada's health check interval means it can take up to 40-60 seconds to detect the failure. Wait a full minute and run kubectl get clusters again.

If it still shows True after 90 seconds, verify the container actually stopped by running docker ps and confirming member1-control-plane is not listed.

Help me troubleshoot why Karmada is not detecting my member1 cluster as unhealthy after I stopped the Docker container.

Observe automatic rescheduling

With member1 offline, Karmada needs to ensure your nginx workloads keep running. It does this by rescheduling to the remaining healthy cluster.

• Switch your context to member2 and check how many nginx pods are running:

kubectl get pods -l app=nginx --context kind-member2

You should see additional replicas running on member2 to compensate for the lost member1 capacity.

• Switch back to the karmada-apiserver context and view the resource binding to understand how Karmada made the scheduling decision:

kubectl get resourcebindings --context karmada-apiserver

What is a ResourceBinding?

A ResourceBinding is Karmada's internal record of which clusters a resource has been scheduled to. When member1 went down, Karmada updated the binding to target only member2. This is the scheduler's decision log.

• Describe the nginx deployment's resource binding to see the scheduling details:

kubectl describe resourcebinding nginx-deployment --context karmada-apiserver

Look for the Clusters field in the binding status. It should now list only member2, confirming Karmada moved all workloads away from the failed cluster.

Not seeing extra pods on member2?

The rescheduling process can take 1-2 minutes after the cluster status changes. Run the kubectl get pods command again after waiting.

Also confirm you are looking at the right context by running kubectl config current-context to verify which cluster you are querying.

Help me understand why Karmada is not rescheduling my nginx pods to member2 after member1 went down.

Restore the failed cluster

Now let's bring member1 back online and watch Karmada rebalance workloads across both clusters again.

• Restart the member1 cluster container by running:

docker start member1-control-plane

• Wait about 30-40 seconds, then check the cluster status again:

kubectl get clusters

You should see member1's READY column return to True. Both clusters are now healthy.

What happens when a cluster comes back?

When Karmada detects the cluster is healthy again, it removes the taints. The scheduler then re-evaluates the placement and rebalances workloads according to your PropagationPolicy and OverridePolicy.

This means member1 should eventually go back to running 3 replicas (from your OverridePolicy) and member2 back to 1 replica.

• Verify workloads have rebalanced by checking pods on both clusters:

kubectl get pods -l app=nginx --context kind-member1
kubectl get pods -l app=nginx --context kind-member2

You should see member1 running 3 nginx pods and member2 running 1 nginx pod, matching your OverridePolicy configuration from the previous step.

Member1 is Ready but pods haven't rebalanced?

Rebalancing can take 2-3 minutes as Karmada goes through its reconciliation loop. Run the pod checks again after waiting.

If member1 remains at 0 pods after several minutes, check that your OverridePolicy is still applied by running kubectl get overridepolicies --context karmada-apiserver.

Help me troubleshoot why workloads are not rebalancing back to member1 after it recovered.

You just proved that Karmada handles the full disaster recovery lifecycle: detection, failover, and rebalancing. Next up, you will take this further with weighted replica scheduling in the Secret Mission.

Secret mission

Weighted Replica Scheduling

Your current setup duplicates the full Deployment to each cluster, then overrides replicas after the fact. Real production systems do something smarter. They let the scheduler split a total replica count across clusters by weight, so you declare "10 replicas, 70% here, 30% there" and the scheduler handles the math. Can you replace your OverridePolicy with Karmada's built-in ReplicaScheduling to achieve a 70/30 split?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally using Docker containers, so there are no ongoing cloud costs.

Resources you used:

• 3 kind clusters running as Docker containers (karmada-host, member1, member2). These use approximately 4-6GB of RAM while running.
• Local YAML manifest files (deployment.yaml, propagationpolicy.yaml, overridepolicy.yaml, propagationpolicy-weighted.yaml, deployment-scaled.yaml).
• Karmada configuration directory at /etc/karmada/.

Keep everything running

No action needed. Choose this if you are still actively experimenting with Karmada policies or want to try different scheduling strategies.

• Your 3 kind clusters will continue running as Docker containers, using approximately 4-6GB of RAM.
• All YAML files remain in your working directory for future reference.
• You can resume working with the clusters at any time by switching kubectl contexts.

Pause - I'll come back to this later

Stop the Docker containers to free up memory. You can restart them later without losing your cluster state.

• Stop all three kind cluster containers by running:

docker stop karmada-host-control-plane member1-control-plane member2-control-plane

• When you are ready to resume, restart the containers by running:

docker start karmada-host-control-plane member1-control-plane member2-control-plane

• Your YAML files remain untouched and ready to reapply.

Delete - I don't want to use this again

Remove all project resources completely and free up disk space and memory.

• Delete all three kind clusters by running:

kind delete cluster --name karmada-host && kind delete cluster --name member1 && kind delete cluster --name member2

• Remove the Karmada configuration directory by running:

sudo rm -rf /etc/karmada/

• Remove your local YAML manifest files by running:

rm -f deployment.yaml propagationpolicy.yaml overridepolicy.yaml propagationpolicy-weighted.yaml deployment-scaled.yaml

• Verify the clusters are gone by running:

kind get clusters

• You should see no clusters listed in the output.

That's a Wrap!

That's a Wrap!

Nice work! You just built a fully functional multi-cluster Kubernetes federation on your local machine using Karmada. This is the exact pattern platform engineers use to manage workloads across AWS, GCP, and Azure from a single control plane.

You've learned how to:

• Built a 3-cluster Kubernetes federation using Karmada as the multi-cloud control plane, with one host cluster and two member clusters simulating different cloud providers.
• Deployed workloads to the karmada-apiserver and distributed them across clusters using PropagationPolicy, then customized replica counts per-cluster with OverridePolicy.
• Tested automated failover by taking a cluster offline and verifying that Karmada rescheduled workloads to the remaining healthy cluster automatically.
• Secret Mission: Implemented weighted replica scheduling to split 10 replicas across clusters in a 70/30 ratio using Karmada's Divided scheduling type.

Ready to quiz yourself?