文章目錄

• k8s多節點
• • 配置部署環境
  • 負載均衡部署
  • 部署keepalived服務
  • k8s拓展知識

主機名ip地址安裝包

Master01	192.168.136.88	kube-apiserver kube-controller-manager kube-scheduler etcd
Master02	192.168.136.60	kube-apiserver kube-controller-manager kube-scheduler
Ndoe01	192.168.136.40	kubelet kube-proxy docker flannel etcd
Ndoe02	192.168.136.30	kubelet kube-proxy docker flannel etcd
nginx01	192.168.136.10	nginx、keepalived
nginx02	168.168.136.20	nginx、keepalived

k8s多節點

配置部署環境

優先關閉防火墻和selinux服務

在master01上操作

復制kubernetes目錄到master02

[root@localhost ~]# scp -r /opt/kubernetes/ root@192.168.136.60:/opt

復制master中的三個組件啟動腳本到master02上

scp /usr/lib/systemd/system/{kube-apiserver,kube-controller-manager,kube-scheduler}.service root@192.168.136.60:/usr/lib/systemd/system/

需要拷貝master01上已有的etcd證書給master02使用

特別注意：master02一定要有etcd證書

[root@localhost ~]# scp -r /opt/etcd/ root@192.168.136.60:/opt/

master02上操作

修改配置文件

cd /opt/kubernetes/cfg/ [root@master2 cfg]# vim kube-apiserver5 --bind-address=192.168.136.60 \6 --secure-port=6443 \7 --advertise-address=192.168.136.60 \

啟動master02中的三個組件服務

[root@localhost cfg]# systemctl start kube-apiserver.service [root@localhost cfg]# systemctl start kube-controller-manager.service [root@localhost cfg]# systemctl start kube-scheduler.service

增加環境變量

[root@localhost cfg]# vim /etc/profile #末尾添加 export PATH=$PATH:/opt/kubernetes/bin/ [root@matser02 cfg]# source /etc/profile

檢查是否開啟

[root@matser02 cfg]# kubectl get node NAME STATUS ROLES AGE VERSION 192.168.136.30 Ready <none> 24h v1.12.3 192.168.136.40 Ready <none> 25h v1.12.3

負載均衡部署

在01和b02上操作

負載均衡
192.168.136.10",
192.168.136.20",

[root@localhost ~]# yum clean all [root@localhost ~]# yum list systemctl stop NetworkManager systemctl disable NetworkManager

b01 和b02操作，建立nginx源

[root@localhost ~]# vim /etc/yum.repos.d/nginx.repo [nginx] name=nginx repo baseurl=http://nginx.org/packages/centos/7/$basearch/ gpgcheck=0

安裝nginx

[root@localhost ~]# yum install nginx -y

添加apr四層轉發

所謂四層就是基于IP+端口的負載均衡，通過虛擬IP+端口接收請求，然后再分配到真實的服務器

[root@localhost ~]# vim /etc/nginx/nginx.conf events {worker_connections 1024; } -----------------------------------添加 stream {log_format main '$remote_addr $upstream_addr - [$time_local] $status $upstream_bytes_sent';access_log /var/log/nginx/k8s-access.log main;upstream k8s-apiserver {server 192.168.136.80:6443;server 192.168.136.60:6443;}server {listen 6443;proxy_pass k8s-apiserver;}} ----------------------------------------- http {

開啟服務查看端口

[root@localhost ~]# systemctl start nginx [root@localhost ~]# netstat -ntap | grep nginx tcp 0 0 0.0.0.0:6443 0.0.0.0:* LISTEN 67128/nginx: master tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 67128/nginx: master

部署keepalived服務

配置01和02節點

[root@localhost ~]# yum install keepalived -y

修改配置文件nide1節點

刪除原有的配置文件，復制下面的代碼

[root@localhost ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { # 接收郵件地址 notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } # 郵件發送地址 notification_email_from Alexandre.Cassen@firewall.loc smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id NGINX_MASTER } vrrp_script check_nginx {script "/etc/nginx/check_nginx.sh" }vrrp_instance VI_1 { state MASTER interface ens33virtual_router_id 51 # VRRP 路由 ID實例，每個實例是唯一的 priority 100 # 優先級，備服務器設置 90 advert_int 1 # 指定VRRP 心跳包通告間隔時間，默認1秒 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.136.100/24 } track_script {check_nginx} }

修改配置文件nide2節點

刪除原有的配置文件，復制下面的代碼

[root@localhost ~]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { # 接收郵件地址 notification_email { acassen@firewall.loc failover@firewall.loc sysadmin@firewall.loc } # 郵件發送地址 notification_email_from Alexandre.Cassen@firewall.loc smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id NGINX_MASTER } vrrp_script check_nginx {script "/etc/nginx/check_nginx.sh" }vrrp_instance VI_1 { state BACKUP interface ens33virtual_router_id 51 # VRRP 路由 ID實例，每個實例是唯一的 priority 90 # 優先級，備服務器設置 90 advert_int 1 # 指定VRRP 心跳包通告間隔時間，默認1秒 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.136.100/24 } track_script {check_nginx} }

在fznode1和fznode2上建立nginx腳本（目的nginx關閉keepalived也隨之關閉）

[root@localhost ~]# vim /etc/nginx/check_nginx.sh count=$(ps -ef |grep nginx |egrep -cv "grep|$$")if [ "$count" -eq 0 ];thensystemctl stop keepalived fi

加權限

[root@localhost ~]# chmod +x /etc/nginx/check_nginx.sh

開啟服務

[root@localhost ~]# systemctl start keepalived

//查看lb01地址信息

驗證地址漂移

（fz01中使用pkill nginx，再在fz02中使用ip a 查看）

//恢復操作（在lb01中先啟動nginx服務，再啟動keepalived服務）

開始修改node節點配置文件統一VIP

注意不是負載均衡節點是下面👇，兩個都要改

node節點
192.168.136.40
192.168.136.30

[root@localhost ~]# vim /opt/kubernetes/cfg/bootstrap.kubeconfig [root@localhost ~]# vim /opt/kubernetes/cfg/kubelet.kubeconfig [root@localhost ~]# vim /opt/kubernetes/cfg/kube-proxy.kubeconfig //統統修改為VIP server: https://192.168.195.100:6443 [root@localhost cfg]# systemctl restart kubelet.service [root@localhost cfg]# systemctl restart kube-proxy.service

開啟服務

[root@localhost cfg]# systemctl restart kubelet.service [root@localhost cfg]# systemctl restart kube-proxy.service

替換完成直接自檢

[root@localhost cfg]# grep 100 * bootstrap.kubeconfig: server: https://192.168.136.100:6443 kubelet.kubeconfig: server: https://192.168.136.100:6443 kube-proxy.kubeconfig: server: https://192.168.136.100:6443

在lb01上查看nginx的k8s日志

[root@localhost ~]# tail /var/log/nginx/k8s-access.log 192.168.136.40 192.168.136.60:6443 - [05/Oct/2020:23:23:59 +0800] 200 1121 192.168.136.30 192.168.136.80:6443, 192.168.136.60:6443 - [05/Oct/2020:23:24:00 +0800] 200 0, 1119 192.168.136.40 192.168.136.80:6443, 192.168.136.60:6443 - [05/Oct/2020:23:24:00 +0800] 200 0, 1119 192.168.136.30 192.168.136.80:6443, 192.168.136.60:6443 - [05/Oct/2020:23:24:00 +0800] 200 0, 1121

在master01上操作

測試創建pod

kubectl run nginx --image=nginx

查看狀態

[root@localhost ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-dbddb74b8-nf9sk 0/1 ContainerCreating 0 33s //正在創建中 [root@localhost ~]# kubectl get pods NAME READY STATUS RESTARTS AGE nginx-dbddb74b8-nf9sk 1/1 Running 0 80s //創建完成，運行中

k8s拓展知識

k8s注意日志問題

看不了日志因為我們是匿名

[root@localhost ~]# kubectl logs nginx-dbddb74b8-c8gjh Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-dbddb74b8-c8gjh)

解決：

賦予管理員權限

kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous

再次查看就可以看到

[root@localhost ~]# kubectl logs nginx-dbddb74b8-c8gjh /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh 10-listen-on-ipv6-by-default.sh: Getting the checksum of /etc/nginx/conf.d/default.conf 10-listen-on-ipv6-by-default.sh: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh /docker-entrypoint.sh: Configuration complete; ready for start up

查看日志

查看pod網絡（mster端口操作）

[root@localhost ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE nginx-dbddb74b8-c8gjh 1/1 Running 0 8m57s 172.17.54.2 192.168.136.40 <none>

在對應網段的node節點上操作可以直接訪問

[root@localhost cfg]# curl 172.17.54.2 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style>body {width: 35em;margin: 0 auto;font-family: Tahoma, Verdana, Arial, sans-serif;} </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p><p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p><p><em>Thank you for using nginx.</em></p> </body> </html>

訪問就會產生日志

[root@localhost ~]# kubectl get pods -o wide 172.17.54.1 - - [05/Oct/2020:15:39:24 +0000] "GET / HTTP/1.1" 200 612 "-" "curl/7.29.0" "-"

總結

以上是生活随笔為你收集整理的部署k8s多节点的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。