一 Nginx代理實現kube-apiserver高可用
1.1 Nginx實現高可用
基於 nginx 代理的 kube-apiserver 高可用方案。
控制節點的 kube-controller-manager、kube-scheduler 是多實例部署，所以只要有一個實例正常，就可以保證高可用；
集群內的 Pod 使用 K8S 服務域名 kubernetes 訪問 kube-apiserver， kube-dns 會自動解析出多個 kube-apiserver 節點的 IP，所以也是高可用的；
在每個節點起一個 nginx 進程，後端對接多個 apiserver 實例，nginx 對它們做健康檢查和負載均衡；
kubelet、kube-proxy、controller-manager、scheduler 通過本地的 nginx（監聽 127.0.0.1）訪問 kube-apiserver，從而實現 kube-apiserver 的高可用；
從而基於 nginx 4 層透明代理功能實現 K8S 節點( master 節點和 worker 節點)高可用訪問 kube-apiserver 。
1.2 下載編譯Nginx
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# wget http://nginx.org/download/nginx-1.15.3.tar.gz 3 [root@k8smaster01 work]# tar -xzvf nginx-1.15.3.tar.gz 4 [root@k8smaster01 ~]# cd /opt/k8s/work/nginx-1.15.3/ 5 [root@k8smaster01 nginx-1.15.3]# mkdir nginx-prefix 6 [root@k8smaster01 nginx-1.15.3]# ./configure --with-stream --without-http --prefix=$(pwd)/nginx-prefix --without-http_uwsgi_module --without-http_scgi_module --without-http_fastcgi_module 7 [root@k8smaster01 ~]# cd /opt/k8s/work/nginx-1.15.3/ 8 [root@k8smaster01 nginx-1.15.3]# make && make install
解釋：
- –with-stream：開啟 4 層透明轉發(TCP Proxy)功能；
- –without-xxx：關閉所有其他功能，這樣生成的動態鏈接二進製程序依賴最小。
- [root@k8smaster01 ~]# cd /opt/k8s/work/nginx-1.15.3/
- [root@k8smaster01 nginx-1.15.3]# ./nginx-prefix/sbin/nginx -v
1.3 驗證編譯后的Nginx
1 [root@k8smaster01 ~]# cd /opt/k8s/work/nginx-1.15.3 2 [root@k8smaster01 nginx-1.15.3]# ./nginx-prefix/sbin/nginx -v 3 nginx version: nginx/1.15.3 4 [root@k8smaster01 nginx-1.15.3]# ldd ./nginx-prefix/sbin/nginx #查看 nginx 動態鏈接的庫 5 linux-vdso.so.1 => (0x00007ffdda980000) 6 libdl.so.2 => /lib64/libdl.so.2 (0x00007feb37300000) 7 libpthread.so.0 => /lib64/libpthread.so.0 (0x00007feb370e4000) 8 libc.so.6 => /lib64/libc.so.6 (0x00007feb36d17000) 9 /lib64/ld-linux-x86-64.so.2 (0x00007feb37504000)
提示：由於只開啟了 4 層透明轉發功能，所以除了依賴 libc 等操作系統核心 lib 庫外，沒有對其它 lib 的依賴(如 libz、libssl 等)，以便達到精簡編譯的目的。
1.4 安裝和部署Nginx
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]} 4 do 5 echo ">>> ${master_ip}" 6 mkdir -p /opt/k8s/kube-nginx/{conf,logs,sbin} 7 done #創建Nginx目錄 8 [root@k8smaster01 ~]# cd /opt/k8s/work 9 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 10 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]} 11 do 12 echo ">>> ${master_ip}" 13 scp /opt/k8s/work/nginx-1.15.3/nginx-prefix/sbin/nginx root@${master_ip}:/opt/k8s/kube-nginx/sbin/kube-nginx 14 ssh root@${master_ip} "chmod a+x /opt/k8s/kube-nginx/sbin/*" 15 ssh root@${master_ip} "mkdir -p /opt/k8s/kube-nginx/{conf,logs,sbin}" 16 done #分發Nginx二進制
1.5 配置Nginx 四層透明轉發
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# cat > kube-nginx.conf <<EOF 3 worker_processes 1; 4 5 events { 6 worker_connections 1024; 7 } 8 9 stream { 10 upstream backend { 11 hash $remote_addr consistent; 12 server 172.24.8.71:6443 max_fails=3 fail_timeout=30s; 13 server 172.24.8.72:6443 max_fails=3 fail_timeout=30s; 14 server 172.24.8.73:6443 max_fails=3 fail_timeout=30s; 15 } 16 17 server { 18 listen 127.0.0.1:8443; 19 proxy_connect_timeout 1s; 20 proxy_pass backend; 21 } 22 } 23 EOF 24 [root@k8smaster01 ~]# cd /opt/k8s/work 25 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 26 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]} 27 do 28 echo ">>> ${master_ip}" 29 scp kube-nginx.conf root@${master_ip}:/opt/k8s/kube-nginx/conf/kube-nginx.conf 30 done #分發Nginx四層透明代理配置文件
1.6 配置Nginx system
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# cat > kube-nginx.service <<EOF 3 [Unit] 4 Description=kube-apiserver nginx proxy 5 After=network.target 6 After=network-online.target 7 Wants=network-online.target 8 9 [Service] 10 Type=forking 11 ExecStartPre=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx -t 12 ExecStart=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx 13 ExecReload=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx -s reload 14 PrivateTmp=true 15 Restart=always 16 RestartSec=5 17 StartLimitInterval=0 18 LimitNOFILE=65536 19 20 [Install] 21 WantedBy=multi-user.target 22 EOF
1.7 分發Nginx systemd
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]} 4 do 5 echo ">>> ${master_ip}" 6 scp kube-nginx.service root@${master_ip}:/etc/systemd/system/ 7 done
二 啟動並驗證
2.1 啟動Nginx
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]} 4 do 5 echo ">>> ${master_ip}" 6 ssh root@${master_ip} "systemctl daemon-reload && systemctl enable kube-nginx && systemctl restart kube-nginx" 7 done
2.2 檢查Nginx服務
1 [root@k8smaster01 ~]# cd /opt/k8s/work 2 [root@k8smaster01 work]# source /opt/k8s/bin/environment.sh 3 [root@k8smaster01 work]# for master_ip in ${MASTER_IPS[@]} 4 do 5 echo ">>> ${master_ip}" 6 ssh root@${master_ip} "systemctl status kube-nginx |grep 'Active:'" 7 done
本站聲明:網站內容來源於博客園,如有侵權,請聯繫我們,我們將及時處理
【其他文章推薦】
※USB CONNECTOR掌控什麼技術要點? 帶您認識其相關發展及效能
※評比前十大台北網頁設計、台北網站設計公司知名案例作品心得分享
※智慧手機時代的來臨，RWD網頁設計已成為網頁設計推薦首選
※評比南投搬家公司費用收費行情懶人包大公開