Ansible通过kubernetes.core.k8s_info和kubernetes.core.k8s访问OCP

环境

OCP

api.kai1212.cp.fyre.ibm.com

Red Hat Enterprise Linux release 8.8 (Ootpa)
OpenShift 4.14.1
- Red Hat Enterprise Linux CoreOS release 4.14
Python 3.6.8
pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)

[root@api.kai1212.cp.fyre.ibm.com ~]# oc version
Client Version: 4.14.1
Kustomize Version: v5.0.1
Server Version: 4.14.1
Kubernetes Version: v1.27.6+f67aeb3

kubectl

[root@api.kai1212.cp.fyre.ibm.com ~]# kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{
       Major:"1", Minor:"27", GitVersion:"v1.27.4", GitCommit:"0c63f9da2694c080257111616c60005f32a5bf47", GitTreeState:"clean", BuildDate:"2023-10-20T23:16:49Z", GoVersion:"go1.20.10 X:strictfipsruntime", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{
       Major:"1", Minor:"27", GitVersion:"v1.27.6+f67aeb3", GitCommit:"f3ec0ed759cde48849b6e3117c091b7db90c95fa", GitTreeState:"clean", BuildDate:"2023-10-20T22:20:44Z", GoVersion:"go1.20.10 X:strictfipsruntime", Compiler:"gc", Platform:"linux/amd64"}

Client（Ansible控制节点）

kairedhat91.fyre.ibm.com
- Red Hat Enterprise Linux release 9.2 (Plow)
- Ansible 2.14.9

步骤

准备工作

在client端配置ssh免密登录OCP端

用 ssh-keygen -t rsa 命令生成密钥对，然后把公钥（ /root/.ssh/id_rsa.pub ）的内容添加到OCP端（ api.kai1212.cp.fyre.ibm.com ）的 /root/.ssh/authorized_keys 文件里。

测试一下ssh登录，确保能够免密登录。

在client端安装Ansible

在client端安装Ansible：

yum install ansible

如果报错：

No match for argument: ansible
Error: Unable to find a match: ansible

则需要配置repo源。修改 /etc/yum.repos.d/redhat.repo ，添加如下内容：

[epel]
name=epel
baseurl=https://mirrors.aliyun.com/epel/9/Everything/x86_64/
enabled=1
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/epel/RPM-GPG-KEY-EPEL-9

然后再安装，就OK了。

......
......
Installed:
  ansible-1:7.7.0-1.el9.noarch           ansible-core-1:2.14.9-1.el9.x86_64      python3-cffi-1.14.5-5.el9.x86_64         python3-cryptography-36.0.1-4.el9.x86_64    python3-packaging-20.9-5.el9.noarch    python3-ply-3.11-14.el9.noarch
  python3-pycparser-2.20-6.el9.noarch    python3-pyparsing-2.4.7-9.el9.noarch    python3-resolvelib-0.5.4-5.el9.noarch    sshpass-1.09-4.el9.x86_64

Complete!

检查Ansible版本：

[root@kairedhat91 ~]# ansible --version
ansible [core 2.14.9]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.9/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.9.18 (main, Sep  7 2023, 00:00:00) [GCC 11.4.1 20230605 (Red Hat 11.4.1-2)] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

编辑 /etc/ansible/hosts ，添加如下内容：

[myvm]
api.kai1212.cp.fyre.ibm.com

测试一下Ansible连通性：

[root@kairedhat91 ~]# ansible all -m ping
api.kai1212.cp.fyre.ibm.com | SUCCESS => {
   
    "ansible_facts": {
   
        "discovered_interpreter_python": "/usr/libexec/platform-python"
    },
    "changed": false,
    "ping": "pong"
}

kubernetes.core.k8s_info

在client端创建 test1.yml 如下：

---
- hosts: all
  tasks:
    - name: task1
      kubernetes.core.k8s_info:
        api_version: v1
        kind: Namespace
        name: myns1
      register: var1

    - name: task2
      debug:
        msg: "{
   { var1 }}"

该脚本会尝试读取名为 myns1 namespace信息，当然，该namespace目前并不存在。

第1次尝试

运行 ansible-playbook test1.yml ，报错如下：

TASK [task1] **************************************************************************************************************************************************************************************************************************************
fatal: [api.kai1212.cp.fyre.ibm.com]: FAILED! => {
   "changed": false, "msg": "Failed to import the required Python library (kubernetes) on api.kai1212.cp.fyre.ibm.com's Python /usr/libexec/platform-python. Please read the module documentation and install it in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"}

这是因为，按Ansible官网文档所言，需要有以下先决条件：

python >= 3.6
kubernetes >= 12.0.0
PyYAML >= 3.11

在OCP端安装python和pip3

在OCP端安装python和pip3：

yum install python3

如果报错说找不到，则需要配置repo源，参考上面的做法。

安装成功，如下：

......
......
Installed:
  python3-pip-9.0.3-22.el8.noarch                                       python3-setuptools-39.2.0-7.el8.noarch                                       python36-3.6.8-38.module+el8.5.0+12207+5c5719bc.x86_64

Complete!

安装好以后，查看python3和pip3的版本：

[root@api.kai1211.cp.fyre.ibm.com ~]# python3 --version
Python 3.6.8

[root@api.kai1211.cp.fyre.ibm.com ~]# pip3 -V
pip 9.0.3 from /usr/lib/python3.6/site-packages (python 3.6)

注：如果已有python3，则可以下载安装pip3。

下载 get-pip.py 文件：

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

安装pip3：

python3 get-pip.py

在OCP端安装kubernetes

在OCP端安装kubernetes：

pip3 install kubernetes

注：如果报错 AttributeError: module 'tarfile' has no attribute 'data_filter' ，则需升级pip3：

python3 -m pip install --upgrade pip

在OCP端安装PyYAML

我没有单独做这一步，应该是在安装kubernetes的同时，也安装了满足条件的PyYAML。

如果不满足条件，则可能需要重新安装kubernetes，详见下面的“其它问题”。

第2次尝试

再次运行 ansible-playbook test1.yml ，报错如下：

TASK [task1] **************************************************************************************************************************************************************************************************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ansible_collections.kubernetes.core.plugins.module_utils.k8s.exceptions.CoreException: Could not create API client: Invalid kube-config file. No configuration found.
fatal: [api.kai1212.cp.fyre.ibm.com]: FAILED! => {
   "changed": false, "msg": "Could not create API client: Invalid kube-config file. No configuration found."}

这是因为我们没有指定config文件。默认的config文件是 ~/.kube/config 。

在OCP端配置config文件

在OCP端，找到kube config文件，将其复制到 ~/.kube/ 目录下：

cp /root/auth/kubeconfig ~/.kube/config

第3次尝试

再次运行 ansible-playbook test1.yml ，这次终于成功了：

TASK [task1] **************************************************************************************************************************************************************************************************************************************
ok: [api.kai1212.cp.fyre.ibm.com]

TASK [task2] **************************************************************************************************************************************************************************************************************************************
ok: [api.kai1212.cp.fyre.ibm.com] => {
   
    "msg": {
   
        "api_found": true,
        "changed": false,
        "failed": false,
        "resources": []
    }
}

运行成功，只不过没找到 myns1 namespace，获取到的信息为空。

其它问题

PyYAML

如果遇到如下报错：

ERROR: Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

解决方法为：

pip3 install --ignore-installed PyYAML

pip3 install kubernetes

安装/更新kubectl（可选）

下载 kubectl ：

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

添加 x 属性：

chmod +x kubectl

将其放到 /usr/bin/ 目录下：

mv kubectl /usr/bin/

查看版本，比如：

[root@kai12101 ~]# kubectl version
Client Version: v1.28.4
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
The connection to the server localhost:8080 was refused - did you specify the right host or port?

kubernetes.core.k8s

前面通过 kubernetes.core.k8s_info 来读取信息，接下来我们通过 kubernetes.core.k8s 来操作k8s，比如创建一个namespace。

创建文件 test2.yml 如下：

---
- hosts: all
  tasks:
    - name: task1
      kubernetes.core.k8s:
        name: myns1
        api_version: v1
        kind: Namespace
        state: present

运行结果如下：

[root@kairedhat91 ~]# ansible-playbook test2.yml
......
......
TASK [task1] **************************************************************************************************************************************************************************************************************************************
changed: [api.kai1212.cp.fyre.ibm.com]

最后，我们再次运行 test1.yml ，获取namespace myns1 的信息，结果如下：

TASK [task1] **************************************************************************************************************************************************************************************************************************************
ok: [api.kai1212.cp.fyre.ibm.com]

TASK [task2] **************************************************************************************************************************************************************************************************************************************
ok: [api.kai1212.cp.fyre.ibm.com] => {
   
    "msg": {
   
        "api_found": true,
        "changed": false,
        "failed": false,
        "resources": [
            {
   
                "apiVersion": "v1",
                "kind": "Namespace",
                "metadata": {
   
                    "annotations": {
   
                        "openshift.io/sa.scc.mcs": "s0:c26,c15",
                        "openshift.io/sa.scc.supplemental-groups": "1000680000/10000",
                        "openshift.io/sa.scc.uid-range": "1000680000/10000"
                    },
                    "creationTimestamp": "2023-12-12T01:07:08Z",
                    "labels": {
   
                        "kubernetes.io/metadata.name": "myns1",
                        "pod-security.kubernetes.io/audit": "restricted",
                        "pod-security.kubernetes.io/audit-version": "v1.24",
                        "pod-security.kubernetes.io/warn": "restricted",
                        "pod-security.kubernetes.io/warn-version": "v1.24"
                    },
                    "managedFields": [
                        {
   
                            "apiVersion": "v1",
                            "fieldsType": "FieldsV1",
                            "fieldsV1": {
   
                                "f:metadata": {
   
                                    "f:labels": {
   
                                        "f:pod-security.kubernetes.io/audit": {
   },
                                        "f:pod-security.kubernetes.io/audit-version": {
   },
                                        "f:pod-security.kubernetes.io/warn": {
   },
                                        "f:pod-security.kubernetes.io/warn-version": {
   }
                                    }
                                }
                            },
                            "manager": "pod-security-admission-label-synchronization-controller",
                            "operation": "Apply",
                            "time": "2023-12-12T01:07:08Z"
                        },
                        {
   
                            "apiVersion": "v1",
                            "fieldsType": "FieldsV1",
                            "fieldsV1": {
   
                                "f:metadata": {
   
                                    "f:labels": {
   
                                        ".": {
   },
                                        "f:kubernetes.io/metadata.name": {
   }
                                    }
                                }
                            },
                            "manager": "OpenAPI-Generator",
                            "operation": "Update",
                            "time": "2023-12-12T01:07:08Z"
                        },
                        {
   
                            "apiVersion": "v1",
                            "fieldsType": "FieldsV1",
                            "fieldsV1": {
   
                                "f:metadata": {
   
                                    "f:annotations": {
   
                                        ".": {
   },
                                        "f:openshift.io/sa.scc.mcs": {
   },
                                        "f:openshift.io/sa.scc.supplemental-groups": {
   },
                                        "f:openshift.io/sa.scc.uid-range": {
   }
                                    }
                                }
                            },
                            "manager": "cluster-policy-controller",
                            "operation": "Update",
                            "time": "2023-12-12T01:07:08Z"
                        }
                    ],
                    "name": "myns1",
                    "resourceVersion": "37555",
                    "uid": "7e4dcd8b-eae2-4f4a-8153-b229e279b0c4"
                },
                "spec": {
   
                    "finalizers": [
                        "kubernetes"
                    ]
                },
                "status": {
   
                    "phase": "Active"
                }
            }
        ]
    }
}

可见，成功获取了namespace myns1 的信息。

参考

https://blog.csdn.net/qq_55977540/article/details/120235601
https://blog.csdn.net/weixin_41010198/article/details/103852838
https://blog.51cto.com/99cloud/2336420
https://docs.ansible.com/ansible/latest/collections/kubernetes/core/k8s_info_module.html
https://docs.ansible.com/ansible/latest/collections/kubernetes/core/k8s_module.html
https://www.runoob.com/w3cnote/python-pip-install-usage.html
https://mirrors.aliyun.com/epel/9/Everything/x86_64/