[kubespray] offline 설치 환경 구성

목적 : kubespray를 오프라인 환경에서 설치하기 위한 환경 구성

 

*nexus3 docker registry는 구성되어 있음.

 

 

 

1. kubespray 다운로드

[root@test-deploy opt]# git clone https://github.com/kubernetes-sigs/kubespray

*버전확인 : grep -r ^kube_version

*원하는 버전으로 checkout

 

 

2. python3 venv 환경 구성

[root@test-deploy kubespray]# yum install python3-pip
[root@test-deploy kubespray]# pip3 install virtualenv
[root@test-deploy kubespray]# virtualenv kubespray-venv
[root@test-deploy kubespray]# . kubespray-venv/bin/activate

(kubespray-venv) [root@test-deploy kubespray]# 

 

 

3. requirement 설치

(kubespray-venv) [root@test-deploy kubespray]# pip install -r requirements.txt

(kubespray-venv) [root@test-deploy kubespray]# deactivate

[root@test-deploy bin]#

 

 

4. kubespray 설치에 필요한 이미지 리스트 확보

[root@test-deploy kubespray]# cd contrib/offline

[root@test-deploy offline]# mkdir temp

[root@test-deploy offline]# bash generate_list.sh

[root@test-deploy offline]# ls temp/
files.list  generate.sh  images.list

 

5. docker image

5-1. docker images 다운로드

(이미지 옮기기)

5-2. docker images tag 변경

5-3. nexus3로 docker images push

5-4. docker image 리스트 확인

# curl 20.21.0.10:5000/v2/_catalog

*버전까지는 어떻게 보지?

 

 

6. kubernetes files 다운로드

/opt/kubespray/k8s_offline_file.sh

#!/bin/bash DOWNLOAD_PATH=/tmp/kubespray.$(date +%Y%m%d) HTTP_ROOT=/var/www/repos/kube_files   #DocumentRoot KUBE_VERSION=v1.0.0 CALICO_VERSION=v1.0.0 mkdir -p ${DOWNLOAD_PATH} for ofile in $(grep -v "^#" contrib/offline/temp/files.list) do   wget ${ofile} -P ${DOWNLOAD_PATH} done # copy http service path cp ${DOWNLOAD_PATH}/calicoctl-linux-amd64                    ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/cni-plugins-linux-amd64-v0.9.1.tgz       ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/crictl-v1.21.0-linux-amd64.tar.gz        ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/crun-0.19-linux-amd64                    ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/etcd-v3.4.13-linux-amd64.tar.gz          ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/helm-v3.5.4-linux-amd64.tar.gz           ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/kata-static-1.12.1-x86_64.tar.xz         ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/krew.tar.gz                              ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/kubeadm                                  ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/kubectl                                  ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/kubelet                                  ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/nerdctl-0.8.1-linux-amd64.tar.gz         ${HTTP_ROOT}/ cp ${DOWNLOAD_PATH}/v3.18.4.tar.gz                           ${HTTP_ROOT}/

 

7. offline.yml에 repo 설정

/opt/kubespray/inventory/k8s/group_vars/all/offline.yml | grep -v ^#

--- registry_host: "20.21.0.10:5000" files_repo: "http://20.21.0.10/kube_files" yum_repo: "http://20.21.0.10" kube_image_repo: "{{ registry_host }}" gcr_image_repo: "{{ registry_host }}" docker_image_repo: "{{ registry_host }}" quay_image_repo: "{{ registry_host }}" kubeadm_download_url: "{{ files_repo }}/kubeadm" kubectl_download_url: "{{ files_repo }}/kubectl" kubelet_download_url: "{{ files_repo }}/kubelet" cni_download_url: "{{ files_repo }}/cni-plugins-linux-{{ image_arch }}-{{ cni_version }}.tgz" crictl_download_url: "{{ files_repo }}/crictl-{{ crictl_version }}-{{ ansible_system | lower }}-{{ image_arch }}.tar.gz" etcd_download_url: "{{ files_repo }}/etcd-{{ etcd_version }}-linux-amd64.tar.gz" calicoctl_download_url: "{{ files_repo }}/calicoctl-linux-{{ image_arch }}" calico_crds_download_url: "{{ files_repo }}/{{ calico_version }}.tar.gz" helm_download_url: "{{ files_repo }}/helm-{{ helm_version }}-linux-{{ image_arch }}.tar.gz" crun_download_url: "{{ files_repo }}/crun-{{ crun_version }}-linux-{{ image_arch }}" docker_rh_repo_base_url: "{{ yum_repo }}/docker-ce-stable"

 

8. local repo이기 때문에 gpgcheck=0 설정

# vi /opt/kubespray/roles/container-engine/docker/templates/rh_docker.repo.j2

 

 

9. ~/inventory/k8s/group_vars/에 추가 필요 설정

- metrics-server

- network 설정

- ceph 설정

- ingress 설정

- inventory.ini

...

 

 

10. 배포