K8s by Example: Canary Deployments

Canary deployments release new versions to a small subset of traffic first. If the canary is healthy, gradually increase traffic until full rollout. If problems occur, roll back with minimal impact. Use for: risk mitigation, A/B testing, feature validation, compliance requirements.
canary-basic.yaml

Basic canary with two Deployments sharing a Service. Both have the same app label so the Service routes to both. Adjust replica counts to control traffic split (9 stable + 1 canary = 10% canary traffic).

 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-stable
spec:
  replicas: 9
  selector:
    matchLabels:
      app: my-app
      version: stable
  template:
    metadata:
      labels:
        app: my-app
        version: stable
    spec:
      containers:
        - name: app
          image: my-app:v1.0.0
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-canary
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
      version: canary
  template:
    metadata:
      labels:
        app: my-app
        version: canary
    spec:
      containers:
        - name: app
          image: my-app:v1.1.0
canary-service.yaml

The Service selects Pods by the app label only, routing to both stable and canary versions. Traffic distribution is proportional to replica count. For precise control, use Ingress or service mesh.

 
apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  selector:
    app: my-app
  ports:
    - port: 80
      targetPort: 8080
canary-ingress.yaml

Ingress-based canary with nginx-ingress annotations. The canary-weight annotation sends a percentage of traffic to the canary backend. More precise than replica-based splitting.

 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-stable
spec:
  ingressClassName: nginx
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: app-stable
                port:
                  number: 80
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "10"
spec:
  ingressClassName: nginx
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: app-canary
                port:
                  number: 80
canary-header.yaml

Header-based canary routing. Only requests with specific headers go to canary. Useful for internal testing before public rollout. Combine with weight for staged rollout.

 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: app-canary-header
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "X-Canary"
    nginx.ingress.kubernetes.io/canary-by-header-value: "true"
spec:
  ingressClassName: nginx
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: app-canary
                port:
                  number: 80
terminal

For automated canary with analysis, install Argo Rollouts. It provides the Rollout CRD which replaces Deployment and adds canary/blue-green strategies with automatic promotion and rollback.

 
$ kubectl apply -n argo-rollouts --create-namespace -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml
namespace/argo-rollouts created
customresourcedefinition.apiextensions.k8s.io/rollouts.argoproj.io created
serviceaccount/argo-rollouts created
clusterrole.rbac.authorization.k8s.io/argo-rollouts created
deployment.apps/argo-rollouts created
canary-analysis.yaml

Argo Rollouts provides automated canary analysis. Define metrics to check (error rate, latency) and thresholds. The rollout automatically progresses or rolls back based on analysis results.

 
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: my-app
spec:
  replicas: 10
  selector:
    matchLabels:
      app: my-app
  template:
    metadata:
      labels:
        app: my-app
    spec:
      containers:
        - name: app
          image: my-app:v1.1.0
  strategy:
    canary:
      steps:
        - setWeight: 10
        - pause: {duration: 5m}
        - setWeight: 30
        - pause: {duration: 5m}
        - setWeight: 50
        - pause: {duration: 5m}
      analysis:
        templates:
          - templateName: success-rate
        startingStep: 1
analysis-template.yaml

AnalysisTemplate defines how to measure canary health. Query Prometheus for error rates, compare against thresholds. If the analysis fails, the rollout automatically aborts.

 
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
  name: success-rate
spec:
  metrics:
    - name: success-rate
      interval: 1m
      successCondition: result[0] >= 0.95
      failureLimit: 3
      provider:
        prometheus:
          address: http://prometheus.monitoring.svc:9090
          query: |
            sum(rate(http_requests_total{
              app="my-app",
              status=~"2.."
            }[5m])) /
            sum(rate(http_requests_total{
              app="my-app"
            }[5m]))
terminal

Monitor canary progress with kubectl or the Argo Rollouts CLI. Watch traffic shifting, analysis results, and rollout status. Manually promote or abort if needed.

 
$ kubectl argo rollouts get rollout my-app -w
Name:            my-app
Status:          рее Paused
Strategy:        Canary
  Step:          2/6
  SetWeight:     30
  ActualWeight:  30

$ kubectl argo rollouts promote my-app

$ kubectl argo rollouts abort my-app

$ kubectl get pods -l app=my-app --show-labels
NAME                        LABELS
my-app-stable-7d8f9-abc12   app=my-app,version=stable
my-app-canary-5f6g7-xyz34   app=my-app,version=canary

Index | GitHub | Use arrow keys to navigate |