《kubernetes-1.8.0》13-addon-traefik

《kubernetes 1.8.0 测试环境安装部署》

时间：2017-11-30

一、traefix介绍

trarfik是一个类似ingress-nginx的http反向代理和负载均衡服务。支持Docker, Swarm mode, Kubernetes, Marathon, Consul, Etcd, Rancher, Amazon ECS等。

Imagine that you have deployed a bunch of microservices on your infrastructure. You probably used a service registry (like etcd or consul) and/or an orchestrator (swarm, Mesos/Marathon) to manage all these services. If you want your users to access some of your microservices from the Internet, you will have to use a reverse proxy and configure it using virtual hosts or prefix paths:

假设你在架构中部署了一些微服务，可能用到了一些服务注册组件（类似etcd或者consul）或者一个服务编排工具（swarm, Mesos/Marathon）用来管理所有的服务。如果你希望用户通过互联网访问你的一些微服务，你需要配置反向代理利用虚拟主机或者prefix paths。

• > domain api.domain.com will point the microservice api in your private network
• > path domain.com/web will point the microservice web in your private network
• > domain backoffice.domain.com will point the microservices backoffice in your private network, load-balancing between your multiple instances

But a microservices architecture is dynamic… Services are added, removed, killed or upgraded often, eventually several times a day.

但是一个微服务架构是动态的，服务经常添加、删除、killed或者更新，甚至一天多次。

Traditional reverse-proxies are not natively dynamic. You can’t change their configuration and hot-reload easily.

传统的反向代理无法实现动态性，你无法轻松的更改配置并动态加载。

Here enters Træfik.

接下来看一下traefix

Træfik can listen to your service registry/orchestrator API, and knows each time a microservice is added, removed, killed or upgraded, and can generate its configuration automatically. Routes to your services will be created instantly.

traefik能够监听你所部署的服务 注册/编排组件 的API，能够知道任何时候微服务的增加，删除，killed或者更新，能够自动生成相应的配置。立即闯将并将流量路由到你的服务。

目前看起来真的和ingress-nginx很像

Features：

• > It’s fast
• > No dependency hell, single binary made with go
• > Tiny official docker image
• > Rest API
• > Hot-reloading of configuration. No need to restart the process
• > Circuit breakers, retry
• > Round Robin, rebalancer load-balancers
• > Metrics (Rest, Prometheus, Datadog, Statd)
• > Clean AngularJS Web UI
• 有一个AngularJS框架的web UI
• > Websocket, HTTP/2, GRPC ready
• > Access Logs (JSON, CLF)
• 日志文件支持json格式
• > Let’s Encrypt support (Automatic HTTPS with renewal)
• 支持 Let’s Encrypt 的免费ssl证书
• > High Availability with cluster mode

二、traefix部署（kubernetes）

由于之前做了ingress-nginx，功能上与traefik重复，因此删除ingress后再部署：

##清空ingress-nginx的部署内容：
[[e-mail]131 traefik]# kubectl get all -n ingress-nginx
NAME                          DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR                          AGE
ds/nginx-ingress-controller   3         3         3         3            3           custom/ingress-controller-ready=true   1d

NAME                          DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/default-http-backend   1         1         1            1           1d

NAME                                 DESIRED   CURRENT   READY     AGE
rs/default-http-backend-6c59748b9b   1         1         1         1d

NAME                          DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/default-http-backend   1         1         1            1           1d

NAME                          DESIRED   CURRENT   READY     UP-TO-DATE   AVAILABLE   NODE SELECTOR                          AGE
ds/nginx-ingress-controller   3         3         3         3            3           custom/ingress-controller-ready=true   1d

NAME                                 DESIRED   CURRENT   READY     AGE
rs/default-http-backend-6c59748b9b   1         1         1         1d

NAME                                       READY     STATUS    RESTARTS   AGE
po/default-http-backend-6c59748b9b-9t7nm   1/1       Running   0          1d
po/nginx-ingress-controller-5jxp5          1/1       Running   0          1d
po/nginx-ingress-controller-845z5          1/1       Running   0          1d
po/nginx-ingress-controller-wprgh          1/1       Running   0          1d

NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
svc/default-http-backend   ClusterIP   10.254.204.231   <none>        80/TCP    1d

###

kubectl delete ds nginx-ingress-controller -n ingress-nginx
kubectl delete deploy default-http-backend   -n ingress-nginx
kubectl delete svc default-http-backend  -n ingress-nginx

traefik-rbac.yaml

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

Træfik部署有两种方式 Deployment or DaemonSet，可扩展性上Deployment应该会更好些，本例使用Deployment方式，如果想看DaemonSet方式–> Træfik DaemonSet

traefik-deployment.yaml

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 2
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      containers:
      - image: traefik
        name: traefik-ingress-lb
        args:
        - --web
        - --kubernetes
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - nodePort: 30080
    port: 80
    protocol: TCP
    targetPort: 80
    name: web
  - nodePort: 38080
    port: 80
    protocol: TCP
    targetPort: 8080
    name: admin
  type: NodePort

• type: NodePort: 对应的service采用nodeport方式，将在所有节点监听对应端口
• 由于与本地环境端口冲突，web端对应端口更换为30080（缺省80）
• 由于与本地环境端口冲突，wadmin端对应端口更换为38080（缺省8080）
• replicas: 更改为2（缺省为1）

部署yaml文件：

kubectl apply -f traefik-rbac.yaml 
kubectl apply -f traefik-deployment.yaml 

检查pod是否running：

[[e-mail]131 traefik]# kubectl get pod -n kube-system -o wide
NAME                                          READY     STATUS    RESTARTS   AGE       IP                NODE
...
traefik-ingress-controller-7bcb868b8b-24z9d   1/1       Running   0          12m       192.168.188.26    node.133
traefik-ingress-controller-7bcb868b8b-b9t5d   1/1       Running   0          2m        192.168.33.232    node.131

三、测试ingress

先实现traefik自己ui界面的ingress：

ui.yaml:

---
apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  rules:
  - host: traefik-ui.bs.com
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web

host: 该字段更改为自己使用的域名，本例中使用traefik-ui.bs.com

修改hosts文件加上制定域名的解析，并测试访问：

出图就可以了：

四、测试Basic Authentication

用htpasswd创建 base64-encoded 的验证文件：

##创建验证文件
[root@node-131 traefik]# htpasswd -c ./auth myusername
New password: 
Re-type new password: 
Adding password for user myusername

##查看验证文件
[root@node-131 traefik]# cat auth 
myusername:$apr1$rebFRzKS$jMmTxbQE.zRQjuB1V.5v90

##用auth文件创建secret
[root@node-131 traefik]# kubectl create secret generic mysecret --from-file auth --namespace=monitoring
secret "mysecret" created

Create the ingress using the following annotations to specify basic auth and that the username and password is stored in mysecret.

创建ingress的时候用上annotations 制定basic验证方式以及使用之前创建的mysecret进行验证

ingress.kubernetes.io/auth-type: “basic”
ingress.kubernetes.io/auth-secret: “mysecret”

创建ingress：

prometheus-ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: prometheus-dashboard
 namespace: monitoring
 annotations:
   kubernetes.io/ingress.class: traefik
   ingress.kubernetes.io/auth-type: "basic"
   ingress.kubernetes.io/auth-secret: "mysecret"
spec:
 rules:
 - host: prometheus.bs.com
   http:
     paths:
     - backend:
         serviceName: prometheus-k8s
         servicePort: 9090
 - host: alert.bs.com
   http:
     paths:
     - backend:
         serviceName: alertmanager-main
         servicePort: 9093
 - host: grafana.bs.com
   http:
     paths:
     - backend:
         serviceName: grafana
         servicePort: 3000

在hosts文件中添加 prometheus.bs.com 、 alert.bs.com 、 grafana.bs.com 这三个域名的解析记录，并测试

弹出 basic auth验证对话框：

输入用户名口令后成功访问：

五、测试Path based routing

利用现有default namespace里的两个svc：

[[e-mail]131 traefik]# kubectl get svc
NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
...
demo-service           NodePort    10.254.23.220    <none>        80:31001/TCP   7d
...
nginx-service          NodePort    10.254.197.49    <none>        80:31000/TCP   7d

demo-path-ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: demo-path
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.frontend.rule.type: PathPrefixStrip
spec:
  rules:
  - host: demo.bs.com
    http:
      paths:
      - path: /demo
        backend:
          serviceName: demo-service
          servicePort: 80
      - path: /nginx
        backend:
          serviceName: nginx-service
          servicePort: 80

部署yaml文件：

kubectl create -f  demo-path-ingress.yaml

查看ingress：

[[e-mail]131 traefik]# kubectl get ing
NAME           HOSTS                              ADDRESS   PORTS     AGE
...
demo-path      demo.bs.com  

修改hosts文件加上域名记录，并测试访问：

本系列其他内容：

• 01-环境准备

• 02-etcd群集搭建

• 03-kubectl管理工具

• 04-master搭建

• 05-node节点搭建

• 06-addon-calico

• 07-addon-kubedns

• 08-addon-dashboard

• 09-addon-kube-prometheus

• 10-addon-EFK

• 11-addon-Harbor

• 12-addon-ingress-nginx

• 13-addon-traefik

参考资料

https://docs.traefik.io/user-guide/kubernetes/#path-based-routing