Weighted Canary Rollout with Istio

Walk me through how you'd canary a new version of a service with Istio. Say you want to start at 5% traffic to v2 and ramp up.

Mid-Level questions

Service Meshintermediate

// interview question

Walk me through how you'd canary a new version of a service with Istio. Say you want to start at 5% traffic to v2 and ramp up.

Sample answer

First, both versions need to be running with distinct labels — `version: v1` on the stable pods and `version: v2` on the new ones. Then I write a DestinationRule that declares both subsets so Istio knows what v1 and v2 mean. Next, a VirtualService with a weighted route: 95 to v1, 5 to v2. To ramp, I bump the weight values and re-apply. Most teams script this or wire it into Flagger or Argo Rollouts so the ramp is automated based on success rate and latency metrics from Prometheus. The key thing to verify after each step is that the subset labels actually match real pods. A common failure mode is shifting weight to v2 when v2 has zero healthy pods because the pod label is `app=reviews-v2` instead of `version: v2`. You'll see 503s spike on the percentage of traffic you shifted. Also remember weights are percentages of matched traffic, not absolute. If your VirtualService has a header match before the weighted route, the weights only apply to requests that fell through to that rule.

Why this matters

This is the most common real-world Istio use case. The interviewer wants to hear that the candidate understands the two-CRD pattern, the label-matching gotcha, and ideally that they know weights are not an SLO-aware progressive delivery system on their own — that's what Flagger or Argo Rollouts add on top. Bonus points for mentioning sticky sessions or session affinity considerations.

Code examples

Initial 95/5 split with DestinationRule subsets

yaml

apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  subsets:
    - name: v1
      labels: { version: v1 }
    - name: v2
      labels: { version: v2 }
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts: [reviews]
  http:
    - route:
        - destination: { host: reviews, subset: v1 }
          weight: 95
        - destination: { host: reviews, subset: v2 }
          weight: 5

Verify the split is hitting both subsets

bash

# Send 100 requests and count which version answered
for i in $(seq 1 100); do
  kubectl exec deploy/curl -- curl -s http://reviews/version
done | sort | uniq -c

# Expected output (roughly):
#   94 v1
#    6 v2

Flagger Canary for automated ramp

yaml

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: reviews
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: reviews
  service:
    port: 9080
  analysis:
    interval: 1m
    threshold: 5
    stepWeight: 10
    maxWeight: 50
    metrics:
      - name: request-success-rate
        thresholdRange: { min: 99 }

Common mistakes to avoid

Forgetting to label the v2 pods with `version: v2`, so traffic shifts to an empty subset
Manually editing weights in production instead of using Flagger or Argo Rollouts
Assuming weights apply globally when an earlier match rule already routed the request

Likely follow-ups

How would you stick a specific user or header to v2 while everyone else stays on v1?
What metrics would you watch during the ramp, and where would you get them?
How does this change if your service holds in-memory session state?

Answer out loud first, then check yourself against the model answer.

Practice all Mid-Level questions More Service Mesh questions

#istio#service-mesh#traffic-management#canary#progressive-delivery

Also worth your time on this topic

Checklist

Istio Traffic Management Checklist: Routing, Retries, and Circuit Breaking

How to configure traffic management policies in Istio so your services can do canary releases, retry transient failures, and shed load when a downstream service goes bad. Covers VirtualService, DestinationRule, retries, timeouts, circuit breakers, and outlier detection.

60-90 minutes

Interview

VirtualService vs DestinationRule

In Istio, what's the difference between a VirtualService and a DestinationRule? When would you use each?

junior

Article

Istio Traffic Management: Routing, Retries, and Circuit Breaking

Configure weighted routing, automatic retries, and circuit breakers in Istio with copy-paste YAML examples and real kubectl output you can verify on your own cluster.

Weighted Canary Rollout with Istio

More Service Mesh interview questions

Also worth your time on this topic

Istio Traffic Management Checklist: Routing, Retries, and Circuit Breaking

VirtualService vs DestinationRule

Istio Traffic Management: Routing, Retries, and Circuit Breaking