k8s-控制器

概述

控制器是什么

• 控制器是 k8s 内置的管理工具。可以帮助用户实现 Pod 的自动部署、自维护、扩容、滚动更新等功能的自动化程序

为什么要使用控制器

•  有大量 Pod 需要维护管理
• 需要维护 Pod 的健康状态
• 控制器可以像机器人一样可以替用户完成维护管理的工作

Deployment 控制器

• 最常用的无状态服务控制器，由 Deployment、ReplicaSet、Pod 组成、支持集群扩容缩容、滚动、更新、自动维护 Pod 可用性及副本数量等功能
• ReplicaSet 和 Pod 由 Deployment 自动管理，用户无需干预 

创建 Deployment

# 资源对象模板
[root@master ~]# kubectl create deployment (name) --image=(名称:标签) --dry-run=client -o yaml
[root@master ~]# vim mydeploy.yaml
kind: Deployment        # 资源对象类型            
apiVersion: apps/v1     # 版本
metadata:               # 元数据
  name: myweb           # 名称
spec:                   # 详细定义
  replicas: 2           # 副本数量
  selector:             # 定义标签选择器
    matchLabels:        # 支持 matchExpressions 表达式语法
      app: httpd        # 通过标签来确定那个 Pod 由它来管理
  template:             # 定义用来创建 Pod 的模板,以下为 Pod 定义
    metadata:           # Pod 元数据
      labels:           # 名称由控制器生成
        app: httpd      # 这里自能定义标签
    spec:               # Pod 的详细定义
      restartPolicy: Always    # 重启策略
      containers:              # 容器定义
      - name: apache           # 容器名称
        image: myos:httpd      # 创建容器使用的镜像
[root@master ~]# kubectl apply -f mydeploy.yaml 
deployment.apps/myweb created
[root@master ~]# kubectl get deployments.apps 
NAME    READY   UP-TO-DATE   AVAILABLE   AGE
myweb   2/2     2            2           11s
[root@master ~]# kubectl get replicasets.apps 
NAME               DESIRED   CURRENT   READY   AGE
myweb-5c75f7c579   2         2         2       24s
[root@master ~]# kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
myweb-5c75f7c579-fcwdw   1/1     Running   0          40s
myweb-5c75f7c579-mv62j   1/1     Running   0          40s

如何访问 Deployment 资源

• Deployment 管理多个 Pod，可以使用服务对其资源进行访问

ClusterIP 服务

# 创建服务访问集群
[root@master ~]# vim websvc.yaml 
---
kind: Service
apiVersion: v1
metadata:
  name: websvc
spec:
  type: ClusterIP
  clusterIP: 10.245.1.80
  selector:
    app: httpd
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
[root@master ~]# kubectl apply -f websvc.yaml
[root@master ~]# curl http://10.245.1.80
Welcome to The Apache.

Pod维护管理

# 自维护自治理
[root@master ~]# kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE        NOMINATED NODE   READINESS GATES
myweb-5c75f7c579-fcwdw   1/1     Running   0          36m   10.244.240.166   node-0004   <none>           <none>
myweb-5c75f7c579-mv62j   1/1     Running   0          36m   10.244.153.160   node-0005   <none>           <none>
# Pod 被删除后，Deploy 会自动创建新的 Pod 来维护集群的完整性
[root@master ~]# kubectl delete pods --all
pod "myweb-5c75f7c579-fcwdw" deleted
pod "myweb-5c75f7c579-mv62j" deleted
[root@master ~]# kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP               NODE        NOMINATED NODE   READINESS GATES
myweb-5c75f7c579-645km   1/1     Running   0          5s    10.244.153.161   node-0005   <none>           <none>
myweb-5c75f7c579-tfxw2   1/1     Running   0          4s    10.244.243.222   node-0003   <none>           <none>

集群扩容缩容

[root@master ~]# kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
myweb-5c75f7c579-645km   1/1     Running   0          6m19s
myweb-5c75f7c579-tfxw2   1/1     Running   0          6m18s
# 扩容
[root@master ~]# kubectl scale deployment myweb --replicas 10
deployment.apps/myweb scaled
[root@master ~]# kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
myweb-5c75f7c579-645km   1/1     Running   0          7m16s
myweb-5c75f7c579-bwxhf   1/1     Running   0          10s
myweb-5c75f7c579-gbbr9   1/1     Running   0          10s
myweb-5c75f7c579-hc2qv   1/1     Running   0          10s
myweb-5c75f7c579-nh778   1/1     Running   0          10s
myweb-5c75f7c579-pjqhc   1/1     Running   0          10s
myweb-5c75f7c579-tfxw2   1/1     Running   0          7m15s
myweb-5c75f7c579-tj7ct   1/1     Running   0          10s
myweb-5c75f7c579-wzlnt   1/1     Running   0          10s
myweb-5c75f7c579-zknbd   1/1     Running   0          10s
# 缩容
[root@master ~]# kubectl scale deployment myweb --replicas 2
deployment.apps/myweb scaled
[root@master ~]# kubectl get pods
NAME                     READY   STATUS    RESTARTS   AGE
myweb-5c75f7c579-gbbr9   1/1     Running   0          80s
myweb-5c75f7c579-tfxw2   1/1     Running   0          8m25s

历史版本信息

# 查看历史版本
[root@master ~]# kubectl rollout history deployment myweb 
deployment.apps/myweb 
REVISION  CHANGE-CAUSE
1         <none>

# 添加注释信息
[root@master ~]# kubectl annotate deployments.apps myweb kubernetes.io/change-cause="httpd.v1"
deployment.apps/myweb annotated
[root@master ~]# kubectl rollout history deployment myweb 
deployment.apps/myweb 
REVISION  CHANGE-CAUSE
1         httpd.v1

滚动更新

# 修改镜像，滚动更新集群（apache是Deployment资源文件中）
[root@master ~]# kubectl set image deployment myweb apache=myos:nginx
deployment.apps/myweb image updated
# 访问验证服务
[root@master ~]# curl http://10.245.1.80
Nginx is running !
# 查看历史版本信息
[root@master ~]# kubectl rollout history deployment myweb 
deployment.apps/myweb 
REVISION  CHANGE-CAUSE
1         httpd.v1
2         httpd.v1

# 给新版本添加注释信息
[root@master ~]# kubectl annotate deployments myweb kubernetes.io/change-cause="nginx.v1"
deployment.apps/myweb annotated
# 查看历史版本信息
[root@master ~]# kubectl rollout history deployment myweb 
deployment.apps/myweb 
REVISION  CHANGE-CAUSE
1         httpd.v1
2         nginx.v1

版本回滚

[root@master ~]# curl http://10.245.1.80
Nginx is running !
# 查看历史版本
[root@master ~]# kubectl rollout history deployment myweb 
deployment.apps/myweb 
REVISION  CHANGE-CAUSE
1         httpd.v1
2         nginx.v1

# 历史版本与回滚
[root@master ~]# kubectl rollout undo deployment myweb --to-revision 1
deployment.apps/myweb rolled back
# 验证版本回滚
[root@master ~]# curl http://10.245.1.80
Welcome to The Apache.
# 查看历史版本
[root@master ~]# kubectl rollout history deployment myweb 
deployment.apps/myweb 
REVISION  CHANGE-CAUSE
2         nginx.v1
3         httpd.v1

# 删除控制器方法1
[root@master ~]# kubectl delete deployments myweb 
deployment.apps "myweb" deleted
# 删除控制器方法2
[root@master ~]# kubectl delete -f mydeploy.yaml
deployment.apps "myweb" deleted

DaemonSet控制器

特性

• 无法自定义副本数量
• 所创建的 Pod 与 node 节点绑定
• 每个 node 上都会运行一个 Pod
• 当有新 Node 加入集群时，会为它新增 Pod 副本，当 Node 从集群移除时，这些 Pod 也会被回收，典型应用：kube-proxy

资源对象案例

[root@master ~]# cp -a mydeploy.yaml myds.yaml
[root@master ~]# vim myds.yaml
---
kind: DaemonSet  # 资源对象类型
apiVersion: apps/v1
metadata:
  name: myds     # 控制器名称
spec:
  # replicas: 2  # 删除副本参数
  selector:
    matchLabels:
      app: httpd
  template:
    metadata:
      labels:
        app: httpd
    spec:
      restartPolicy: Always
      containers:
      - name: webserver
        image: myos:httpd
        imagePullPolicy: Always
# 与 node 绑定，每节点都创建容器
[root@master ~]# kubectl apply -f myds.yaml 
[root@master ~]# kubectl get pods -o wide
NAME         READY   STATUS    RESTARTS   IP            NODE
myds-msrcx   1/1     Running   0          10.244.1.11   node-0001
myds-lwq8l   1/1     Running   0          10.244.2.17   node-0002
myds-4wt72   1/1     Running   0          10.244.3.14   node-0003
myds-6k82t   1/1     Running   0          10.244.4.15   node-0004
myds-9c6wc   1/1     Running   0          10.244.5.19   node-0005

污点干扰

# 设置污点，重建 daemonset
[root@master ~]# kubectl taint node node-0001 k=v:NoSchedule
node/node-0001 tainted
[root@master ~]# kubectl delete -f myds.yaml 
daemonset.apps "myds" deleted
[root@master ~]# kubectl apply -f myds.yaml 
daemonset.apps/myds created
# 有污点不会部署，特殊需求可以设置容忍策略
[root@master ~]# kubectl get pods
NAME         READY   STATUS    RESTARTS   AGE
myds-pkdn9   1/1     Running   0          2s
myds-pkp6b   1/1     Running   0          2s
myds-j84cw   1/1     Running   0          2s
myds-5c69p   1/1     Running   0          2s
# 删除污点后会立即部署
[root@master ~]# kubectl taint node node-0001 k=v:NoSchedule-
node/node-0001 untainted
[root@master ~]# kubectl get pods
NAME         READY   STATUS    RESTARTS   AGE
myds-2c86p   1/1     Running   0          1s
myds-pkdn9   1/1     Running   0          9s
myds-pkp6b   1/1     Running   0          9s
myds-j84cw   1/1     Running   0          9s
myds-5c69p   1/1     Running   0          9s

# 删除控制器
[root@master ~]# kubectl delete -f myds.yaml 
daemonset.apps "myds" deleted

Job/CronJob控制器

• Job 是一个单任务控制器，负责执行一次任务，保证任务在一个或多个Pod上执行成功
• CronJob 像是Job的升级版，它是基于时间管理的Job控制器

Job资源对象

restartPolicy 会判断 exit 状态码

状态码为 0 表示正常，其它都表示失败

# 资源对象模板
[root@master ~]# kubectl create job myjob --image=myos:8.5 --dry-run=client -o yaml -- sleep 3
[root@master ~]# vim myjob.yaml 
---
kind: Job
apiVersion: batch/v1
metadata:
  name: myjob                    # 资源对象名称
spec:                            # Job的详细定义
  template:                      # 以下定义 Pod 模板
    spec:                        # Pod的详细定义
      restartPolicy: OnFailure   # 只支持[OnFailure，Nerver]
      containers:                # 容器定义
      - name: myjob              # 容器名称
        image: myos:8.5          # 创建容器使用的镜像
        command: ["/bin/bash"]   # 自定义任务，可使用脚本
        args:
        - -c
        - |
          sleep 3
          exit $((RANDOM%2))
[root@master ~]# kubectl apply -f myjob.yaml 

# 失败了会重启
[root@master ~]# kubectl get pods -l job-name=myjob -w
NAME             READY   STATUS      RESTARTS     AGE
myjob--1-lrtbk   1/1     Running     0            2s
myjob--1-lrtbk   0/1     Error       0            4s
myjob--1-lrtbk   1/1     Running     1 (1s ago)   5s
myjob--1-lrtbk   0/1     Completed   1            9s

# 完成后不会重复执行
[root@master ~]# kubectl get jobs.batch 
NAME    COMPLETIONS   DURATION   AGE
myjob   1/1           8s         12s

# 删除Job控制器
[root@master ~]# kubectl delete -f myjob.yaml 
job.batch "myjob" deleted

CronJob资源对象

时间定义 schedule：分、时、日、月、周

# 资源对象模板
[root@master ~]# kubectl create cronjob mycj --image=myos:8.5 --schedule='* * * * *' --dry-run=client -o yaml -- sleep 3
[root@master ~]# vim mycj.yaml
---
kind: CronJob
apiVersion: batch/v1
metadata:
  name: mycj
spec:
  schedule: "* * * * 1-5"
  jobTemplate:  # 以下定义 Job 模板
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
          - name: myjob
            image: myos:8.5
            command: ["/bin/bash"]
            args:
            - -c
            - |
              sleep 3
              exit $((RANDOM%2))

[root@master ~]# kubectl apply -f mycj.yaml 
cronjob.batch/mycj created
[root@master ~]# kubectl get cronjobs 
NAME   SCHEDULE        SUSPEND   ACTIVE   LAST SCHEDULE   AGE
mycj   */1 * * * 1-5   False     0        <none>          4s

# 按照时间周期，每分钟触发一个任务
[root@master ~]# kubectl get jobs -w
NAME                     READY   STATUS              RESTARTS
mycj-27808172--1-w6sbx   0/1     Pending             0
mycj-27808172--1-w6sbx   0/1     ContainerCreating   0
mycj-27808172--1-w6sbx   1/1     Running             0
mycj-27808172--1-w6sbx   0/1     Completed           1

# 保留三次结果，多余的会被删除
[root@master ~]# kubectl get jobs 
NAME            COMPLETIONS   DURATION   AGE
mycj-27605367   1/1           31s        3m30s
mycj-27605368   1/1           31s        2m30s
mycj-27605369   1/1           31s        90s
mycj-27605370   0/1           30s        30s
[root@master ~]# kubectl get jobs 
NAME            COMPLETIONS   DURATION   AGE
mycj-27605368   1/1           31s        2m33s
mycj-27605369   1/1           31s        93s
mycj-27605370   1/1           31s        33s

# 删除CJ控制器
[root@master ~]# kubectl delete -f mycj.yaml 
cronjob.batch "mycj" deleted

StatefulSet控制器

StatefulSet 旨在与有状态的应用及分布式系统一起使用，涉及了 Headless 服务、存储卷、DNS 等相关知识点，是一个宽泛而复杂的话题

图例：

Headless 服务

• 在配置 StatefulSet 的时候，首先要定义一个 Headless 的服务
• 在创建 Pod 的时候会自动把 <Pod名称> 注册为域名

# 配置 headless 服务
[root@master ~]# vim mysvc2.yaml 
---
kind: Service
apiVersion: v1
metadata:
  name: mysvc2       
spec:
  type: ClusterIP
  clusterIP: None    # 设置 IP 为 None
  selector:
    app: httpd
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

[root@master ~]# kubectl apply -f mysvc2.yaml 
service/mysvc2 created
[root@master ~]# kubectl get service mysvc2
NAME          TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
mysvc2        ClusterIP   None          <none>        80/TCP    61s

StatefulSet资源对象

资源对象文件

[root@master ~]# vim mysts.yaml
---
kind: StatefulSet      # 资源对象类型
apiVersion: apps/v1
metadata:
  name: mysts          # 控制器名称
spec:
  serviceName: mysvc2  # 新增 headless 服务名称
  replicas: 3
  selector:
    matchLabels:
      app: httpd
  template:
    metadata:
      labels:
        app: httpd
    spec:
      restartPolicy: Always
      containers:
      - name: webserver
        image: myos:httpd
        imagePullPolicy: Always
        

[root@master ~]# kubectl apply -f mysts.yaml 
statefulset.apps/mysts created

[root@master ~]# kubectl get pods
NAME      READY   STATUS    RESTARTS   AGE
mysts-0   1/1     Running   0          3s
mysts-1   1/1     Running   0          2s
mysts-2   1/1     Running   0          1s

访问 StatefulSet 资源

# Pod 名称自动注册 DNS 服务
[root@master ~]# host mysts-0.mysvc2.default.svc.cluster.local 10.245.0.10
Using domain server:
Name: 10.245.0.10
Address: 10.245.0.10#53
Aliases: 

mysts-0.mysvc2.default.svc.cluster.local has address 10.244.1.81

[root@master ~]# host mysvc2.default.svc.cluster.local 10.245.0.10
Using domain server:
Name: 10.245.0.10
Address: 10.245.0.10#53
Aliases: 

mysvc2.default.svc.cluster.local has address 10.244.3.82
mysvc2.default.svc.cluster.local has address 10.244.2.83
mysvc2.default.svc.cluster.local has address 10.244.1.81

# 删除sts控制器
[root@master ~]# kubectl delete -f mysts.yaml -f mysvc2.yaml
statefulset.apps "mysts" deleted
service "mysvc2" deleted

HPA控制器

• HorizontalPodAutoscaling 简称 HPA，可以在 kubernetes 集群中基于CPU利用率或其它应用程序提供的度量指标实现水平自动伸缩的功能，自动缩放POD的数量
• 控制器会周期性的获取平均利用率
• 与目标值相比较后来调整副本数量

图例：

创建 Deployment

# 为 Deploy 控制器添加资源配额
[root@master ~]# vim mydeploy.yaml 
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: myweb
spec:
  replicas: 1                 # 修改副本数量
  selector:
    matchLabels:
      app: httpd
  template:
    metadata:
      labels:
        app: httpd
    spec:
      restartPolicy: Always
      containers:
      - name: webserver
        image: myos:httpd
        imagePullPolicy: Always
        resources:           # 为该资源设置配额
          requests:          # HPA 控制器会根据配额使用情况伸缩集群
            cpu: 200m        # CPU 配额

创建 ClusterIP 服务

[root@master ~]# vim websvc.yaml
---
kind: Service
apiVersion: v1
metadata:
  name: websvc
spec:
  type: ClusterIP
  clusterIP: 10.245.1.80
  selector:
    app: httpd
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80

创建后端集群

[root@master ~]# kubectl apply -f mydeploy.yaml -f websvc.yaml
deployment.apps/myweb created
service/websvc created

# 验证服务
[root@master ~]# kubectl top pods
NAME                    CPU(cores)   MEMORY(bytes)   
myweb-c8c66f894-2cfjv   1m           17Mi  
[root@master ~]# curl -s http://10.245.1.80
Welcome to The Apache.

创建 HPA 控制器

[root@master ~]# vim myhpa.yaml 
---
kind: HorizontalPodAutoscaler
apiVersion: autoscaling/v1
metadata:
  name: myweb
spec:
  minReplicas: 1    # 最少保留的副本数量
  maxReplicas: 5    # 最大创建的副本数量
  targetCPUUtilizationPercentage: 50    # 警戒值，以百分比计算
  scaleTargetRef:                       # 监控的资源对象
    kind: Deployment                    # 资源对象类型
    apiVersion: apps/v1
    name: myweb


[root@master ~]# kubectl apply -f myhpa.yaml 
horizontalpodautoscaler.autoscaling/myweb created

# 刚刚创建 unknown 是正常现象，最多等待 60s 就可以正常获取数据
[root@master ~]# kubectl get horizontalpodautoscalers.autoscaling 
NAME    REFERENCE          TARGETS         MINPODS   MAXPODS   REPLICAS
myweb   Deployment/myweb   <unknown>/50%   1         5         0

[root@master ~]# kubectl get horizontalpodautoscalers.autoscaling 
NAME    REFERENCE          TARGETS         MINPODS   MAXPODS   REPLICAS
myweb   Deployment/myweb   0%/50%          1         5         1

弹性集群验证测试

持续访问 Web 服务，增加CPU负载，在超过警戒值之后，会触发扩容操作，等待几秒之后发现副本数量增加了

停止访问，让CPU空闲，为了防止集群性能抖动，副本不会立即释放，必须300秒内的平均负载小于警戒值，才开始释放副本，当副本达到最小值时停止释放

测试时可以通过访问php脚本增加CPU的负载

# 终端 1 访问提高负载
[root@master ~]# while sleep 1;do
    curl -s "http://10.245.1.80/info.php?id=100000" -o /dev/null
done
# 终端 2 监控 HPA 变化
[root@master ~]# kubectl get hpa -w
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
myweb   Deployment/myweb   0%/50%    1         3         1          8m21s
myweb   Deployment/myweb   49%/50%   1         3         1          9m
myweb   Deployment/myweb   56%/50%   1         3         1          9m15s
myweb   Deployment/myweb   56%/50%   1         3         2          9m30s
myweb   Deployment/myweb   37%/50%   1         3         2          9m45s
myweb   Deployment/myweb   32%/50%   1         3         2          10m
myweb   Deployment/myweb   41%/50%   1         3         2          11m
myweb   Deployment/myweb   48%/50%   1         3         2          11m
myweb   Deployment/myweb   51%/50%   1         3         2          11m
myweb   Deployment/myweb   59%/50%   1         3         2          11m
myweb   Deployment/myweb   58%/50%   1         3         3          12m
myweb   Deployment/myweb   42%/50%   1         3         3          12m
myweb   Deployment/myweb   34%/50%   1         3         3          12m

# 如果 300s 内平均负载小于标准值，就会自动缩减集群规模
[root@master ~]# kubectl get hpa -w
NAME    REFERENCE          TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
myweb   Deployment/myweb   38%/50%   1         3         3          19m
myweb   Deployment/myweb   21%/50%   1         3         3          20m
myweb   Deployment/myweb   17%/50%   1         3         3          21m
myweb   Deployment/myweb    7%/50%   1         3         3          22m
myweb   Deployment/myweb    0%/50%   1         3         3          23m
myweb   Deployment/myweb    0%/50%   1         3         2          25m
myweb   Deployment/myweb    0%/50%   1         3         1          28m