nginx-ingress 的灰度发布

基于Request Header的流量切分

nginx.ingress.kubernetes.io/canary-by-header：用于通知Ingress将请求路由到Canary Ingress中指定的服务的标头。当请求标头设置always为时，它将被路由到Canary。当标头设置never为时，它将永远不会被路由到Canary。对于任何其他值，将忽略标头，并通过优先级将请求与其他Canary规则进行比较。

nginx.ingress.kubernetes.io/canary-by-header-value：要匹配的标头值，用于通知Ingress将请求路由到Canary Ingress中指定的服务。当请求标头设置为此值时，它将被路由到Canary。对于任何其他标头值，标头将被忽略，并且请求与其他Canary规则的优先级进行比较。此注释必须与canary-by-header一起使用。nginx.ingress.kubernetes.io/canary-by-header注释允许自定义标头值而不是使用硬编码值的扩展。如果nginx.ingress.kubernetes.io/canary-by-header注释未定义，则没有任何效果。

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "new"
  labels:
    app: demo
  name: demo-ingress
  namespace: demo-canary
spec:
  rules:
  - host: canary.example.com
    http:
      paths:
      - backend:
          serviceName: demo-canary
          servicePort: 80
        path: /

上面这个示例，当你请求头加入new: always的时候就会访问demo-canary，当标头设置never为时，则不会访问。例如下面的这个curl

curl -s -H “new: always” http://canary.example.com
下面这个示例使用的是自定义的标头值

例2：使用canary-by-header-value

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "new"
    nginx.ingress.kubernetes.io/canary-by-header-value："xxx"
  labels:
    app: demo
  name: demo-ingress
  namespace: demo-canary
spec:
  rules:
  - host: canary.example.com
    http:
      paths:
      - backend:
          serviceName: demo-canary
          servicePort: 80
        path: /

使用如下请求可以正常访问demo-canary

curl -s -H “new: xxx” http://canary.example.com

基于Cookie的流量切分

nginx.ingress.kubernetes.io/canary-by-cookie：用于通知Ingress将请求路由到Canary Ingress中指定的服务的cookie。当cookie值设置always为时，它将被路由到Canary。当cookie被设置never为时，它将永远不会被路由到Canary。对于任何其他值，cookie将被加入，并且请求与其他Canary规则的优先级进行比较。

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-cookie: "use_under_30_feature"
  labels:
    app: demo
  name: demo-ingress
  namespace: demo-canary
spec:
  rules:
  - host: canary.example.com
    http:
      paths:
      - backend:
          serviceName: demo-canary
          servicePort: 80
        path: /

上面的这个示例，当cookie设置为use_under_30_feature为always的时候，则会访问demo-canary

基于服务权重的流量切分

nginx.ingress.kubernetes.io/canary-weight：将路由到金丝雀Ingress中指定的服务的随机请求的整数（0 - 100）百分比。权重为0意味着该Canary规则不会向Canary入口中的服务发送任何请求。权重为100意味着所有请求都将被发送到Ingress中指定的替代服务。示例如下（使用20%的权重）：

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "20"
  labels:
    app: demo
  name: demo-ingress
  namespace: demo-canary
spec:
  rules:
  - host: canary.example.com
    http:
      paths:
      - backend:
          serviceName: demo-canary
          servicePort: 80
        path: /

注意这里权重不是一个精确的百分比，使用过程当中，只是会看到一个近似分布。

实际案例

以下测试基于服务权重的流量切分，也可以将nginx.ingress.kubernetes.io/canary-weight: "30"改为基于 header 的流量切分。

准备老版本程序
老版本程序 app-old

app-old.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: app-old
spec:
  replicas: 2
  selector:
    matchLabels:
      run: app-old
  template:
    metadata:
      labels:
        run: app-old
    spec:
      containers:
      - image: zouhl/app:v2.1
        imagePullPolicy: Always
        name: app-old
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: app-old
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: app-old
  sessionAffinity: None
  type: NodePor

老版本的 ingress

app-v1.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-app
  labels:
    app: my-app
  annotations:
    kubernetes.io/ingress.class: nginx
  namespace: default
spec:
  rules:
    - host: test.192.168.2.20.xip.io
      http:
        paths:
          - backend:
              serviceName: app-old
              servicePort: 80
            path: /

装备新版本程序

新版本app-new.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: app-new
spec:
  replicas: 2
  selector:
    matchLabels:
      run: app-new
  template:
    metadata:
      labels:
        run: app-new
    spec:
      containers:
      - image: zouhl/app:v2.2
        imagePullPolicy: Always
        name: app-new
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: app-new
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: app-new
  sessionAffinity: None
  type: NodePort

新版本 canary-ingress

app-v2-canary.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-app-canary
  labels:
    app: my-app
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "30"
  namespace: default
spec:
  rules:
    - host: test.192.168.2.20.xip.io
      http:
        paths:
          - backend:
              serviceName: app-new
              servicePort: 80
            path: /

新版本 ingress yaml

app-v2.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-app
  labels:
    app: my-app
  annotations:
    kubernetes.io/ingress.class: nginx
  namespace: default
spec:
  rules:
    - host: test.192.168.2.20.xip.io
      http:
        paths:
          - backend:
              serviceName: app-new
              servicePort: 80
            path: /

流程

发布流程

$ tree                                              
.
├── app-new.yaml
├── app-old.yaml
├── app-v1.yaml
├── app-v2-canary.yaml
└── app-v2.yaml

# 创建old pod 
kubectl create -f app-old.yaml
# 创建old ingress
kubectl create -f app-v1.yaml
# 创建new pod
kubectl create -f app-new.yaml
# 创建canary ingress
kubectl create -f app-v2-canary.yaml

检查

$ kubectl get ingresses.extensions   
NAME            HOSTS                       ADDRESS   PORTS   AGE
app-ingress     www.example.com                       80      109m
my-app          test.192.168.2.20.xip.io              80      25m
my-app-canary   test.192.168.2.20.xip.io              80      1s
nginx-test      nginx.192.168.2.20.xip.io             80      3h12m

在后台观察，70% to v1，30% to v2

如果一切正常则可以正式发布

# delete the canary ingress
kubectl delete -f app-v2-canary.yaml
# set 100% traffic to v2
kubectl apply -f app-v2.yaml
# delete the canary ingress kubectl delete -f app-v2-canary.yaml # set 100% traffic to v2 kubectl apply -f app-v2.yaml

检查 ingress

$ kubectl get ingresses.extensions    
NAME          HOSTS                       ADDRESS   PORTS   AGE
app-ingress   www.example.com                       80      109m
my-app        test.192.168.2.20.xip.io              80      25m
nginx-test    nginx.192.168.2.20.xip.io             80      3h13m