How to Do Canary Deployments with Argo Rollouts 1.10 (2026)
This guide is for engineers who already run services on Kubernetes and want canary releases that shift real traffic gradually and roll back on their own when the error rate climbs. By the end you will have Argo Rollouts v1.10.1 (released 12 October 2026) installed with Helm chart 2.43.8, the Gateway API traffic router plugin v0.17.0 controlling HTTPRoute weights through Envoy Gateway v1.9.2, a Rollout with weighted steps and a manual gate, a Prometheus AnalysisTemplate that aborts bad releases, and a list of the errors you will probably hit. I checked every command, field and version against the official docs and release pages on 20 October 2026, and rendered the Helm values locally with helm template to confirm the output. You need a cluster running a supported Kubernetes version, kubectl, Helm 3, and cluster-admin rights for the install. If you want the why before the how, read the companion post on feature flags vs canary vs blue-green.
How an Argo Rollouts canary works
A Rollout is a drop-in replacement for a Deployment with extra strategies. When you change the pod template, the controller creates a new canary ReplicaSet and walks through your steps: set a traffic weight, pause, run an analysis, repeat. With a traffic router configured, weights are applied at the gateway or mesh, so setWeight: 10 really means 10 percent of requests, not 10 percent of pods. If an AnalysisRun fails, the controller aborts the rollout, sets the canary weight back to zero and marks the Rollout Degraded.
Step 1: Install a Gateway API implementation
Argo Rollouts supports Istio, SMI, ALB, Traefik, Apisix and Ingress NGINX natively, and Gateway API through a plugin. Ingress NGINX was retired in March 2026, so for a new setup use Gateway API. Any conformant implementation works; the plugin's provider status page lists tested ones, including Envoy Gateway (tested at 1.7.2), Istio, Cilium, Kong, Traefik and NGINX Gateway Fabric. This guide uses Envoy Gateway v1.9.2, the current release, following its quickstart. The chart also installs the Gateway API CRDs.
helm install eg oci://docker.io/envoyproxy/gateway-helm \
--version v1.9.2 -n envoy-gateway-system --create-namespace
kubectl wait --timeout=5m -n envoy-gateway-system \
deployment/envoy-gateway --for=condition=Available
Create a GatewayClass and a Gateway with one HTTP listener:
# gateway.yaml
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: eg
namespace: default
spec:
gatewayClassName: eg
listeners:
- name: http
protocol: HTTP
port: 80
kubectl apply -f gateway.yaml
kubectl get gateway eg
# NAME CLASS ADDRESS PROGRAMMED AGE
# eg eg 203.0.113.10 True 30s
If you migrated from Ingress NGINX recently, the migration guide covers translating annotations into routes.
Step 2: Install Argo Rollouts 1.10.1 with the Gateway API plugin
The plain-manifest install is one line (kubectl apply -n argo-rollouts -f https://github.com/argoproj/argo-rollouts/releases/latest/download/install.yaml), but latest is not something you want in production. Use the Helm chart and pin it. Chart 2.43.8 ships app version v1.10.1.
The plugin is a separate binary the controller downloads at startup. Its installation docs list the RBAC it needs: get, list, update and patch on httproutes and grpcroutes. The chart's built-in Gateway API rule (enabled by default under providerRBAC.providers.gatewayAPI) grants get, list, watch and update but not patch, so add it explicitly:
# rollouts-values.yaml (chart argo-rollouts 2.43.8, app v1.10.1)
controller:
replicas: 2 # leader election is on by default
metrics:
enabled: true # exposes controller metrics on a Service
trafficRouterPlugins:
- name: "argoproj-labs/gatewayAPI"
# pick the binary for your node architecture: linux-amd64 or linux-arm64
location: "https://github.com/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi/releases/download/v0.17.0/gatewayapi-plugin-linux-amd64"
providerRBAC:
additionalRules:
- apiGroups: ["gateway.networking.k8s.io"]
resources: ["httproutes", "grpcroutes"]
verbs: ["get", "list", "update", "patch"]
dashboard:
enabled: true # optional read/write UI; restrict access
helm repo add argo https://argoproj.github.io/argo-helm
helm repo update
helm install argo-rollouts argo/argo-rollouts \
--version 2.43.8 -n argo-rollouts --create-namespace \
-f rollouts-values.yaml
kubectl -n argo-rollouts rollout status deployment/argo-rollouts
kubectl -n argo-rollouts get configmap argo-rollouts-config -o yaml | grep -A3 trafficRouterPlugins
# trafficRouterPlugins: |-
# - location: https://github.com/argoproj-labs/.../v0.17.0/gatewayapi-plugin-linux-amd64
# name: argoproj-labs/gatewayAPI
Then check the controller logs for the download line:
kubectl -n argo-rollouts logs deployment/argo-rollouts | grep -i plugin
# level=info msg="Downloading plugin argoproj-labs/gatewayAPI from: https://github.com/..."
# level=info msg="Download complete, it took <N>s"
If your nodes cannot reach GitHub, use the plugin's container image (ghcr.io/argoproj-labs/rollouts-plugin-trafficrouter-gatewayapi) as an init container that copies the binary into an emptyDir, and point location at file:///plugins/rollouts-plugin-trafficrouter-gatewayapi, as the installation docs show.
Step 3: Install the kubectl plugin
The kubectl argo rollouts plugin is optional but you will want it for get, promote, abort and retry. The installation page offers Homebrew or a direct download; pin the version for CI.
# macOS
brew install argoproj/tap/kubectl-argo-rollouts
# Linux, pinned
curl -LO https://github.com/argoproj/argo-rollouts/releases/download/v1.10.1/kubectl-argo-rollouts-linux-amd64
chmod +x kubectl-argo-rollouts-linux-amd64
sudo mv kubectl-argo-rollouts-linux-amd64 /usr/local/bin/kubectl-argo-rollouts
kubectl argo rollouts version
Step 4: Create the Services and HTTPRoute
A canary with traffic routing needs two Services (stable and canary) and a route that lists both as backends. The controller rewrites each Service's selector to add the rollouts-pod-template-hash of the right ReplicaSet, and the plugin rewrites the backend weights.
# route.yaml
apiVersion: v1
kind: Service
metadata:
name: rollouts-demo-stable
namespace: default
spec:
selector:
app: rollouts-demo
ports:
- name: http
port: 80
targetPort: http
---
apiVersion: v1
kind: Service
metadata:
name: rollouts-demo-canary
namespace: default
spec:
selector:
app: rollouts-demo
ports:
- name: http
port: 80
targetPort: http
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: rollouts-demo-route
namespace: default
spec:
parentRefs:
- name: eg
rules:
- matches:
- path:
type: PathPrefix
value: /
backendRefs:
- name: rollouts-demo-stable
port: 80
- name: rollouts-demo-canary
port: 80
kubectl apply -f route.yaml
kubectl get httproute rollouts-demo-route
Step 5: Write the Rollout with canary steps
This uses the official argoproj/rollouts-demo image, which serves a page that polls /color, so you can watch the split in a browser. The steps go 10, 25, 50 percent with timed pauses, then stop at an indefinite pause that needs a human promote. If you already have a Deployment, you can keep it and reference it with spec.workloadRef (with scaleDown: onsuccess), as the migration docs describe.
# rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: rollouts-demo
namespace: default
spec:
replicas: 5
revisionHistoryLimit: 3
selector:
matchLabels:
app: rollouts-demo
template:
metadata:
labels:
app: rollouts-demo
spec:
containers:
- name: rollouts-demo
image: argoproj/rollouts-demo:blue
ports:
- name: http
containerPort: 8080
resources:
requests:
cpu: 10m
memory: 32Mi
strategy:
canary:
canaryService: rollouts-demo-canary
stableService: rollouts-demo-stable
abortScaleDownDelaySeconds: 30 # default is 30; keeps aborted pods briefly for debugging
trafficRouting:
plugins:
argoproj-labs/gatewayAPI:
httpRoute: rollouts-demo-route
namespace: default
steps:
- setWeight: 10
- pause: {duration: 2m}
- setWeight: 25
- pause: {duration: 5m}
- setWeight: 50
- pause: {} # wait for: kubectl argo rollouts promote rollouts-demo
kubectl apply -f rollout.yaml
kubectl argo rollouts get rollout rollouts-demo
# Status: ✔ Healthy
# Strategy: Canary
# Step: 6/6
# SetWeight: 100
# ActualWeight: 100
The first apply goes straight to 100 percent because there is no previous version to protect. Now release a new colour and watch the weights move:
kubectl argo rollouts set image rollouts-demo rollouts-demo=argoproj/rollouts-demo:yellow
kubectl argo rollouts get rollout rollouts-demo --watch
# In another terminal, confirm the plugin rewrote the route:
kubectl get httproute rollouts-demo-route -o jsonpath='{.spec.rules[0].backendRefs[*].weight}'
# 90 10
To view it, port-forward the Envoy service and open http://localhost:8888:
export ENVOY_SERVICE=$(kubectl get svc -n envoy-gateway-system \
--selector=gateway.envoyproxy.io/owning-gateway-namespace=default,gateway.envoyproxy.io/owning-gateway-name=eg \
-o jsonpath='{.items[0].metadata.name}')
kubectl -n envoy-gateway-system port-forward service/${ENVOY_SERVICE} 8888:80
When the rollout reaches the indefinite pause at 50 percent, promote it. --full skips all remaining steps and analysis, so keep it for emergencies.
kubectl argo rollouts promote rollouts-demo
Step 6: Add a Prometheus AnalysisTemplate
Pauses only buy time; analysis makes the decision. The Prometheus provider runs a PromQL query on an interval and evaluates successCondition with the expr language. Two defaults matter: failureLimit defaults to 0, so a single failed measurement fails the run, and consecutiveErrorLimit defaults to 4, so a fifth consecutive error (an unreachable Prometheus, a bad query) also fails the metric.
The demo image has no /metrics endpoint, so this template assumes your own service exports a request counter such as http_requests_total with a code label (the labels Go's promhttp.InstrumentHandlerCounter produces). To measure only canary pods, pass the canary's pod template hash in as an argument and make sure the hash exists as a Prometheus label. With the Prometheus Operator, a PodMonitor can copy the pod label across:
# podmonitor.yaml (Prometheus Operator, monitoring.coreos.com/v1)
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
name: checkout
namespace: default
spec:
selector:
matchLabels:
app: checkout
podTargetLabels:
- rollouts-pod-template-hash # becomes rollouts_pod_template_hash
podMetricsEndpoints:
- port: http
path: /metrics
# analysis-success-rate.yaml
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: success-rate
namespace: default
spec:
args:
- name: canary-hash
metrics:
- name: success-rate
interval: 1m
initialDelay: 1m # let the canary take traffic before measuring
failureLimit: 1 # tolerate one bad minute, fail on the second
# Empty result (no traffic) counts as failure: safer than passing silently.
successCondition: len(result) > 0 && result[0] >= 0.99
provider:
prometheus:
address: http://prometheus-operated.monitoring.svc:9090
timeout: 30
query: |
sum(rate(http_requests_total{namespace="default",
rollouts_pod_template_hash="{{args.canary-hash}}",code!~"5.."}[2m]))
/
sum(rate(http_requests_total{namespace="default",
rollouts_pod_template_hash="{{args.canary-hash}}"}[2m]))
Wire it into the Rollout as background analysis, starting at the first weight step, with the hash supplied by the controller:
# excerpt of spec.strategy.canary in your service's Rollout
analysis:
templates:
- templateName: success-rate
startingStep: 1 # start once setWeight: 10 has applied
args:
- name: canary-hash
valueFrom:
podTemplateHashValue: Latest
Run the query in the Prometheus UI against a real canary hash before you trust it. A typo in a label name returns an empty vector, which this template correctly treats as failure, and you will spend an afternoon wondering why every release aborts. If you already have SLO burn-rate alerts, reuse those recording rules here rather than inventing a second definition of "healthy".
Step 7: Watch an automated rollback
You can see an abort without any metrics pipeline by using the Job provider, which passes if the Job exits 0 and fails otherwise. This smoke test hits the canary Service directly twenty times and fails on any 5xx. The demo's bad- images return a high share of 500s on /color.
# analysis-smoke.yaml
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: smoke
namespace: default
spec:
metrics:
- name: canary-smoke
provider:
job:
spec:
backoffLimit: 0
template:
spec:
restartPolicy: Never
containers:
- name: curl
image: curlimages/curl:8.22.0
command: ["/bin/sh", "-c"]
args:
- |
for i in $(seq 1 20); do
code=$(curl -s -o /dev/null -w '%{http_code}' -X POST \
http://rollouts-demo-canary.default.svc/color)
echo "attempt $i: $code"
[ "$code" -ge 500 ] && exit 1
sleep 1
done
Add an inline analysis step to the demo Rollout right after the first weight, then release a bad image:
steps:
- setWeight: 10
- analysis:
templates:
- templateName: smoke
- pause: {duration: 2m}
# ...remaining steps unchanged
kubectl apply -f analysis-smoke.yaml -f rollout.yaml
kubectl argo rollouts set image rollouts-demo rollouts-demo=argoproj/rollouts-demo:bad-red
kubectl argo rollouts get rollout rollouts-demo --watch
# Status: ✖ Degraded
# Message: RolloutAborted: Rollout aborted update to revision 3: Metric "canary-smoke" assessed Failed ...
The route goes back to 100 0 and stable pods keep serving. A Degraded Rollout stays that way until you act: either set the image back to the stable one (it becomes the new desired state with no canary) or fix the bug, push a new image and let a fresh canary start. kubectl argo rollouts retry rollout rollouts-demo re-runs the same revision, which is useful after a flaky analysis and useless after a real bug.
Step 8: Run it under Argo CD (and the Istio and NGINX variants)
If Argo CD manages the HTTPRoute, it will see the plugin's weight changes as drift and, with self-heal on, put them back mid-canary. The plugin docs recommend ignoring the weights:
# in the Argo CD Application spec
spec:
ignoreDifferences:
- group: gateway.networking.k8s.io
kind: HTTPRoute
jqPathExpressions:
- .spec.rules[].backendRefs[].weight
The full GitOps setup is in the Argo CD guide. If you run Istio, use the native integration instead of the plugin: trafficRouting.istio.virtualService.name plus routes, with a VirtualService whose route has stable and canary destinations at weights 100 and 0, per the Istio traffic docs. The native trafficRouting.nginx.stableIngress integration still works, but it depends on a controller that no longer receives fixes, so treat it as a migration target rather than a design.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Controller logs show plugin download failures, Rollout stuck with traffic router errors | Nodes cannot reach GitHub, or wrong architecture in the URL | Use the init-container install with the ghcr image, or fix linux-amd64 vs linux-arm64; restart the controller after changing the ConfigMap |
forbidden errors on httproutes in controller logs |
Missing patch (or a namespaced Role for a route in another namespace) |
Add the providerRBAC.additionalRules above, or a Role in the route's namespace |
| Weights snap back to the Git values during a canary | Argo CD self-heal reverting the plugin's edits | Add the ignoreDifferences block for backendRefs[].weight |
setWeight: 10 sends far more or less traffic than 10% |
No traffic router configured, so weight is approximated by replica count | Configure trafficRouting; check ActualWeight in kubectl argo rollouts get |
AnalysisRun Error, then metric fails after a few minutes |
Query errors or empty vector with result[0]; five consecutive errors exceed the default consecutiveErrorLimit of 4 |
Test the PromQL, use len(result) > 0 && ..., check Prometheus address and auth |
| Every release aborts at the first analysis | Label mismatch (for example rollouts_pod_template_hash not scraped), or no traffic to the canary yet |
Add podTargetLabels, add initialDelay, verify the query returns data for the canary hash |
Rollout Paused with an Inconclusive analysis |
No success or failure condition matched, or a Job could not pull its image | Fix the condition or image, then kubectl argo rollouts promote or retry rollout |
Rollout Degraded and nothing happens afterwards |
Expected after an abort; the controller waits for a new desired state | Roll the image back, or push a fix; use retry rollout only for flaky analysis |
| Stable Service sends traffic to both versions | Something else edits the Service selector, or canaryService and stableService are the same |
Use two distinct Services owned only by the Rollout |
FAQ
What is the difference between Argo Rollouts and Flagger?
Both automate canary and blue-green releases on Kubernetes with metric-driven promotion. Argo Rollouts replaces the Deployment with a Rollout CRD and pairs naturally with Argo CD; Flagger keeps your Deployment and generates the canary objects around it, and comes from the Flux ecosystem. Flagger's current release is v1.45.0.
Does Argo Rollouts work without a service mesh?
Yes. Without traffic routing, it approximates weights by scaling replica counts. With the Gateway API plugin, any conformant gateway such as Envoy Gateway, Cilium, Kong or Traefik gives you exact request-level weights without a mesh.
How does Argo Rollouts roll back automatically?
When an AnalysisRun fails, because a metric exceeded its failureLimit or consecutiveErrorLimit, the controller aborts the update, sets the canary weight to zero and marks the Rollout Degraded. Stable pods keep serving throughout. The canary pods are scaled down after abortScaleDownDelaySeconds, 30 seconds by default.
Should I use kubectl argo rollouts promote --full in production?
Only as a deliberate override. --full skips every remaining step, pause and analysis, which is exactly the protection you set up. A plain promote advances past the current pause and keeps the rest of the checks.
In production, the first thing I check when a canary misbehaves is not the Rollout but the analysis query: does it return data for the canary hash, right now, in the Prometheus UI. Most "Argo Rollouts is broken" incidents I have looked at in general were a label or a scrape config. The second is the route weights on the actual HTTPRoute, because GitOps fighting the plugin looks like a flaky canary. After that, I keep an indefinite pause before 100 percent on anything customer-facing, at least until the analysis has a few weeks of trustworthy history. The reference docs are the Argo Rollouts documentation, the Gateway API plugin docs and the Gateway API specification.
Related: Progressive Delivery in 2026: Feature Flags vs Canary vs Blue-Green, CI/CD in 2026: Agents Opened 1M PRs in 5 Months and Kubernetes as the Agent Control Plane in 2026.
I'm Ajay Kumar — I build and operate PandaStack, an open-source Firecracker microVM cloud for AI agents, and write about the infrastructure and tooling around it.
Need this kind of infrastructure work? See what I do or email hello@ajayk.sh.
Related
How to Fix 502 Bad Gateway and 504 Gateway Timeout Errors (2026)
Fix 502 Bad Gateway and 504 Gateway Timeout step by step: find which hop generated the error, read NGINX upstream errors, fix keepalive and idle timeout mismatches (ALB 60s, GCP 600s), Kubernetes readiness and rollout gaps, Gateway API timeouts and Cloudflare 520 to 526.
15 minOct 20, 2026How to Set Up Argo CD for GitOps: Step-by-Step Guide (2026)
Install Argo CD 3.5.5 with Helm chart 10.10.5, structure a GitOps repo, bootstrap with app-of-apps and ApplicationSet, configure sync policies, waves and hooks, add SSO and RBAC, deploy Helm and Kustomize apps, and fix OutOfSync, Degraded and ComparisonError.
14 minOct 20, 2026How to Set Up Backstage as an Internal Developer Portal (2026)
Step-by-step Backstage 1.55 setup for 2026: create-app on Node 22 or 24, PostgreSQL, GitHub sign-in, catalog-info.yaml, software templates, TechDocs, a Kubernetes deploy with the official Helm chart, alternatives, and fixes for common errors.
14 min