Devops整合Kubernetes
Kubernetes部署
快速安装Kubernetes
**官网：**https://kuboard.cn/
选择默认支持docker的版本1.19
前置环境部署
所有节点均需执行同操作
# 配置主机名解析
[root@Kubernetes-master ~]# echo "127.0.0.1 $(hostname)" >> /etc/hosts# 防火墙关闭，禁用selinux
[root@master ~]# systemctl disable firewalld.service --now
[root@master ~]# setenforce 0# 配置固定IP
[root@master ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
IPV6INIT="yes"
IPV6_AUTOCONF="yes"
IPV6_DEFROUTE="yes"
IPV6_FAILURE_FATAL="no"
IPV6_ADDR_GEN_MODE="stable-privacy"
NAME="ens33"
UUID="f0967cdb-cfb4-4ed8-973b-03a171214395"
DEVICE="ens33"
ONBOOT="yes"
IPADDR="10.1.8.103"
NETMASK="255.255.255.0"
GATEWAY="10.1.8.2"
DNS1="223.5.5.5"
DNS1="223.6.6.6"
[root@Kubernetes-master ~]# systemctl restart network
安装Docker和Kubelet
# 最后一个参数 1.19.5 用于指定 kubenetes 版本，支持所有 1.19.x 版本的安装
# 腾讯云 docker hub 镜像
# export REGISTRY_MIRROR="https://mirror.ccs.tencentyun.com"
# DaoCloud 镜像
# export REGISTRY_MIRROR="http://f1361db2.m.daocloud.io"
# 华为云镜像
# export REGISTRY_MIRROR="https://05f073ad3c0010ea0f4bc00b7105ec20.mirror.swr.myhuaweicloud.com"
# 阿里云 docker hub 镜像
# export REGISTRY_MIRROR=https://registry.cn-hangzhou.aliyuncs.com
curl -sSL https://kuboard.cn/install-script/v1.19.x/install_kubelet.sh | sh -s 1.19.5
所有节点安装
[root@Kubernetes-master ~]# export REGISTRY_MIRROR=https://registry.cn-hangzhou.aliyuncs.com
[root@Kubernetes-master ~]# curl -sSL https://kuboard.cn/install-script/v1.19.x/install_kubelet.sh | sh -s 1.19.5
初始化master节点
在master节点上进行初始化
修改：export MASTER_IP=10.1.8.103
export APISERVER_NAME=abner.com
其他内容保持不变进行复制粘贴
# 只在 master 节点执行
# 替换 x.x.x.x 为 master 节点实际 IP（请使用内网 IP）
# export 命令只在当前 shell 会话中有效，开启新的 shell 窗口后，如果要继续安装过程，请重新执行此处的 export 命令
export MASTER_IP=10.1.8.103
# 替换 apiserver.demo 为 您想要的 dnsName
export APISERVER_NAME=abner.com
# Kubernetes 容器组所在的网段，该网段安装完成后，由 kubernetes 创建，事先并不存在于您的物理网络中
export POD_SUBNET=10.100.0.1/16
echo "${MASTER_IP} ${APISERVER_NAME}" >> /etc/hosts
curl -sSL https://kuboard.cn/install-script/v1.19.x/init_master.sh | sh -s 1.19.5
[root@Kubernetes-master ~]# export MASTER_IP=10.1.8.103
[root@Kubernetes-master ~]# export APISERVER_NAME=abner.com
[root@Kubernetes-master ~]# export POD_SUBNET=10.100.0.1/16
[root@Kubernetes-master ~]# echo "${MASTER_IP} ${APISERVER_NAME}" >> /etc/hosts
[root@Kubernetes-master ~]# curl -sSL https://kuboard.cn/install-script/v1.19.x/init_master.sh | sh -s 1.19.5查看node状态
# 查看结果等待全部running
[root@Kubernetes-master ~]# kubectl get pod -n kube-system -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
calico-kube-controllers-6c89d944d5-4tbwx 1/1 Running 1 60m 10.100.237.5 kubernetes-master <none> <none>
calico-node-cjnvr 1/1 Running 1 60m 10.1.8.103 kubernetes-master <none> <none>
coredns-59c898cd69-99sqk 1/1 Running 1 60m 10.100.237.6 kubernetes-master <none> <none>
coredns-59c898cd69-mrpjx 1/1 Running 1 60m 10.100.237.4 kubernetes-master <none> <none>
etcd-kubernetes-master 1/1 Running 2 61m 10.1.8.103 kubernetes-master <none> <none>
kube-apiserver-kubernetes-master 1/1 Running 2 61m 10.1.8.103 kubernetes-master <none> <none>
kube-controller-manager-kubernetes-master 1/1 Running 1 61m 10.1.8.103 kubernetes-master <none> <none>
kube-proxy-22tb9 1/1 Running 2 60m 10.1.8.103 kubernetes-master <none> <none>
kube-scheduler-kubernetes-master 1/1 Running 1 61m 10.1.8.103 kubernetes-master <none> <none>
# 查看node状态
[root@Kubernetes-master ~]# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
kubernetes-master Ready master 60m v1.19.5 10.1.8.103 <none> CentOS Linux 7 (Core) 3.10.0-693.el7.x86_64 docker://19.3.11初始化node节点
获取join命令参数
在master节点执行，生成的token有效时间为2小时
[root@Kubernetes-master ~]# kubeadm token create --print-join-command
W0814 10:55:55.430938 81817 configset.go:348] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
kubeadm join abner.com:6443 --token vbxsvs.ijri7kwh4fdajzra --discovery-token-ca-cert-hash sha256:467eedcc799fd0cd8fbe333c41fde8524373866df460d87b2e455134b5c50054
所有node节点执行
# 只在 node 节点执行
# 替换 x.x.x.x 为 master 节点的内网 IP
export MASTER_IP=10.1.8.103
# 替换 apiserver.demo 为初始化 master 节点时所使用的 APISERVER_NAME
export APISERVER_NAME=abner.com
echo "${MASTER_IP} ${APISERVER_NAME}" >> /etc/hosts
kubeadm join abner.com:6443 --token vbxsvs.ijri7kwh4fdajzra --discovery-token-ca-cert-hash sha256:467eedcc799fd0cd8fbe333c41fde8524373866df460d87b2e455134b5c50054
[root@Kubernetes-node ~]# export MASTER_IP=10.1.8.103
[root@Kubernetes-node ~]# export APISERVER_NAME=abner.com
[root@Kubernetes-node ~]# echo "${MASTER_IP} ${APISERVER_NAME}" >> /etc/hosts
[root@Kubernetes-node ~]# kubeadm join abner.com:6443 --token vbxsvs.ijri7kwh4fdajzra --discovery-token-ca-cert-hash sha256:467eedcc799fd0cd8fbe333c41fde8524373866df460d87b2e455134b5c50054
检查初始化结果，等待一会，node节点为Ready状态即可
[root@Kubernetes-master ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
kubernetes-master Ready master 65m v1.19.5
kubernetes-node Ready <none> 55s v1.19.5
整合Yaml文件
因为yaml资源文件可以在k8s上创建应用pod，所以需要提前创建yaml，拉取到k8s服务器上在gitlab上新建YAML文件
gitlab页面-项目中
apiVersion: apps/v1
kind: Deployment
metadata:name: pipelinelabels:app: pipeline
spec:replicas: 2selector:matchLabels:app: pipelinetemplate:metadata:labels:app: pipelinespec:containers:- name: pipelineimage: 10.1.8.102:80/repo/pipeline:v3.0imagePullPolicy: Always ports:- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:labels:app: pipelinename: pipeline
spec:selector:app: pipelineports:- port: 8081targetPort: 8080type: NodePort
对接harbor仓库
在所有节点添加harbor仓库地址配置
[root@Kubernetes-master ~]# cat /etc/docker/daemon.json
{"insecure-registries": ["10.1.8.102:80"], # 添加harbor仓库"registry-mirrors": ["https://09def58152000fc00ff0c00057bad7e0.mirror.swr.myhuaweicloud.com"],"exec-opts": ["native.cgroupdriver=systemd"],"log-driver": "json-file","log-opts": {"max-size": "100m"},"storage-driver": "overlay2","storage-opts": ["overlay2.override_kernel_check=true"]
}
[root@Kubernetes-master ~]# systemctl restart docker# 测试 harbor节点登录
[root@Kubernetes-master ~]# docker login -u admin -p Gzq20000308. 10.1.8.102:80
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-storeLogin SucceededYAML推送K8S服务器设置
主节点创建推送目标目录
# 创建推送目标目录
[root@Kubernetes-master ~]# mkdir /usr/local/k8s
[root@Kubernetes-master ~]# chmod 777 /usr/local/k8s
[root@Kubernetes-master ~]# ll /usr/local/ | grep k8s
drwxrwxrwx 2 root root 6 8月 14 11:32 k8s
jenkins-系统管理-系统配置
jenkins页面-pipline项目-流水线语法
将流水线脚本复制到gitlab的Jenkinsfile中
sshPublisher(publishers: [sshPublisherDesc(configName: 'K8S', transfers: [sshTransfer(cleanRemote: false, excludes: '', execCommand: '', execTimeout: 120000, flatten: false, makeEmptyDirs: false, noDefaultExcludes: false, patternSeparator: '[, ]+', remoteDirectory: '', remoteDirectorySDF: false, removePrefix: '', sourceFiles: 'pipeline.yaml')], usePromotionTimestamp: false, useWorkspaceInPromotion: false, verbose: false)])
新增tag的标签
构建测试yaml文件的推送
master节点上查看是否传递成功
[root@Kubernetes-master ~]# ls /usr/local/k8s/
pipeline.yaml
Jenkins使用无密码登录K8S的master节点
因为希望使用ssh 用户名@k8s地址 kubectl apply -f /usr/local/k8s/pipeline.yml创建资源，会进行ssh免交互
把Jenkins中的公钥内容传递给k8s的master实现免密
[root@Jenkins ~]# docker exec -it jenkins bash
jenkins@392ae884a4ea:/$ cd ~
jenkins@392ae884a4ea:~$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/var/jenkins_home/.ssh/id_rsa):
Created directory '/var/jenkins_home/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /var/jenkins_home/.ssh/id_rsa
Your public key has been saved in /var/jenkins_home/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:Xz49X2FABJOHBr9xElJvnMnHRJvIcpGriYsZaBSvSzg jenkins@392ae884a4ea
The key's randomart image is:
+---[RSA 3072]----+
| oo==+oo |
| . o+OoB o|
| o .=.&.= |
| . . O.o |
| o o S ..+ o |
| E = . o = .. .|
| + . + o o o .|
| . o . . o.|
| .|
+----[SHA256]-----+
jenkins@392ae884a4ea:~$ cd .ssh/
# 复制公钥内容传递给master
jenkins@392ae884a4ea:~/.ssh$ cat id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyMzwHnfEvSdbgwFqlPPTqLZGCsDHoYkbwywAKsniM2nKfRXl3WCSgGvNF8RL9YJvcsmJSDuPtuXBX/8LlwT7Wf28pSApHuRq4ud9JwuAYd90vKUcopXAnWaxRHEWLMvXOg3H5w5pEgzk/KpJA39awA0TKaohc4nt2o+xvgXaCgHS3rMR75KV0ypS+2ABHEugK0hdHXx1lXNJzAn2L/wKB78DJXpmNv4pebW1x50uBZY8YZuShBo/0nzFajQzOW3G6zYYVma7wzUH3UU7aGeyN/goRVVk0EzVYEsRD0CLrr3I3WBlpFnD3TbVN6/70uuA2jjiHXCwpCCssVWsOQyXlw2GYABpmhgy0C5ekJrwdbsqI230JYO9VD1BrxAnEBlAUJtQfBIiQGrxYrSFEWHXtek86MVgRYetQHM/8VvuV+2MoGPYzZZgFNoARUMYocSJJU8YIsHjEF4xD7A6wKzlnMcr5eS/a56lx1MLdgrY0eAX7m1m2pD4IwDxJZrbzmMU= jenkins@392ae884a4eamaster节点
[root@Kubernetes-master ~]# mkdir .ssh
[root@Kubernetes-master ~]# cd .ssh/
[root@Kubernetes-master .ssh]# cat authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyMzwHnfEvSdbgwFqlPPTqLZGCsDHoYkbwywAKsniM2nKfRXl3WCSgGvNF8RL9YJvcsmJSDuPtuXBX/8LlwT7Wf28pSApHuRq4ud9JwuAYd90vKUcopXAnWaxRHEWLMvXOg3H5w5pEgzk/KpJA39awA0TKaohc4nt2o+xvgXaCgHS3rMR75KV0ypS+2ABHEugK0hdHXx1lXNJzAn2L/wKB78DJXpmNv4pebW1x50uBZY8YZuShBo/0nzFajQzOW3G6zYYVma7wzUH3UU7aGeyN/goRVVk0EzVYEsRD0CLrr3I3WBlpFnD3TbVN6/70uuA2jjiHXCwpCCssVWsOQyXlw2GYABpmhgy0C5ekJrwdbsqI230JYO9VD1BrxAnEBlAUJtQfBIiQGrxYrSFEWHXtek86MVgRYetQHM/8VvuV+2MoGPYzZZgFNoARUMYocSJJU8YIsHjEF4xD7A6wKzlnMcr5eS/a56lx1MLdgrY0eAX7m1m2pD4IwDxJZrbzmMU= jenkins@392ae884a4ea
# 重启sshd服务
[root@Kubernetes-master ~]# systemctl restart sshd
Jenkins容器中验证
jenkins@392ae884a4ea:~/.ssh$ ssh root@10.1.8.103 ls /
bin
boot
dev
dvd
etc
home
lib
lib64
media
mnt
opt
proc
root
run
sbin
srv
sys
tmp
usr
var
Jenkins中设置YAML部署
Jenkins页面-pipline-流水线-流水线语法
sh 'ssh root@10.1.8.103 kubectl apply -f /usr/local/k8s/pipeline.yaml'
gitlab的jenkinsfile中增加步骤
修改pipeline.yaml拉取4.0镜像
修改首页内容
路径：src/main/java/com/chenyan/demo/controller/TestController.java
gitlab中把原来的v4.0标签删除，重新创建，命名依然写v4.0
Jenkins页面-pipline-进行构建
# master节点上查看资源
[root@Kubernetes-master ~]# kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
pipeline-54f899865d-7gqgk 1/1 Running 0 12m 10.100.7.130 kubernetes-node <none> <none>
pipeline-54f899865d-8z6d4 1/1 Running 0 12m 10.100.7.129 kubernetes-node <none> <none>[root@Kubernetes-master ~]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4h53m
pipeline NodePort 10.96.166.168 <none> 8081:30831/TCP 10m# node上查看pipeline镜像
[root@Kubernetes-node ~]# docker images 10.1.8.102:80/repo/pipeline
REPOSITORY TAG IMAGE ID CREATED SIZE
10.1.8.102:80/repo/pipeline v4.0 99a6a3f548d5 13 minutes ago 543MB查看网页
查看钉钉通知
-------svm支持向量机)
