群晖、NAS、PVE、飞牛、二手品牌机、垃圾佬

每一位热爱生活的垃圾佬都值得敬畏

勤奋的凯尔森同学
云原生

CentOS Stream 10安装Kubernetes(k8s v1.35.0)高可用集群

内容目录 CentOS Stream 10安装Kubernetes(k8s v1.35.0)高可用集群 本着学 […]
jason0
内容目录

CentOS Stream 10安装Kubernetes(k8s v1.35.0)高可用集群

本着学习新技术和使用新版本的原则，本K8S集群选用刚刚发布的k8s v1.35.0，系统版本选用了CentOS Stream 10也是最新版本的LST系统。

3台master主节点(2C4G)、4台(2C4G)worker node，如果CPU低于2核心是没法初始化K8S的，根据个人硬件配置建议尽可能多核心大内存，对应的 IP如下：

一、 集群概览

本次部署采用 3 Master + 4 Worker 方案，通过 kube-vip 实现控制平面高可用。

Hostname IP Function OS
VIP 192.168.31.220 lb.k8s.hep.com —
hep-k8s-master-01 192.168.31.221 Control plane CentOS Stream 10
hep-k8s-master-02 192.168.31.222 Control plane CentOS Stream 10
hep-k8s-master-03 192.168.31.223 Control plane CentOS Stream 10
hep-k8s-worker-01 192.168.31.224 Worker node CentOS Stream 10
hep-k8s-worker-02 192.168.31.225 Worker node CentOS Stream 10
hep-k8s-worker-03 192.168.31.226 Worker node CentOS Stream 10
hep-k8s-worker-04 192.168.31.227 Worker node CentOS Stream 10

二、 前置工作

我的K8S集群节点都在PVE上，为了更方便，那些重复性的工作我就放在一台机器hep-k8s-master-worker-temp上执行，它是一台centOS Server机器，然后直接复制虚拟机，大大提高效率。如果你是单独的机器，可以在机器上重复执行这些命令即可，从而达到机器配置的一致性。

Screenshot 2025-12-24 113007

2.1 基础环境配置

# 切换至 root
sudo -i

# 更新系统
dnf update -y

#设置hostname
hostnamectl set-hostname hep-k8s-master-01

#固定IP
nmcli connection modify ens18 ipv4.addresses 192.168.31.221/24

# 配置 hosts
cat >> /etc/hosts << EOF
192.168.31.220  lb.k8s.hep.com
192.168.31.221  hep-k8s-master-01
192.168.31.222  hep-k8s-master-02
192.168.31.223  hep-k8s-master-03
192.168.31.224  hep-k8s-worker-01
192.168.31.225  hep-k8s-worker-02
192.168.31.226  hep-k8s-worker-03
192.168.31.227  hep-k8s-worker-04
EOF

# 关闭 SELinux
setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

# 时间同步 (CentOS 10 使用 chrony)
dnf install chrony -y
systemctl enable --now chronyd

# 禁用 Swap
swapoff -a
sed -i '/swap/s/^/#/' /etc/fstab

# 加载内核模块
cat << EOF | tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
ip_vs
ip_vs_rr
ip_vs_wrr
ip_vs_sh
nf_conntrack
EOF

modprobe overlay
modprobe br_netfilter
modprobe ip_vs
modprobe ip_vs_rr
modprobe ip_vs_wrr
modprobe ip_vs_sh
modprobe nf_conntrack

# 内核参数调优
cat << EOF | tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF
sysctl --system

2.2 安装容器运行时 (Docker & cri-dockerd)

# 安装 Docker 仓库
dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
dnf install docker-ce docker-ce-cli containerd.io -y
systemctl enable --now docker

# 安装 cri-dockerd
# 注意：CentOS 10 建议下载适用于 CentOS 的 rpm 或 二进制
wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.21/cri-dockerd-0.3.21.amd64.tgz
tar xf cri-dockerd-0.3.21.amd64.tgz
install -o root -g root -m 0755 cri-dockerd/cri-dockerd /usr/local/bin/cri-dockerd

# 获取服务文件
cd cri-dockerd
wget https://raw.githubusercontent.com/Mirantis/cri-dockerd/master/packaging/systemd/cri-docker.service
wget https://raw.githubusercontent.com/Mirantis/cri-dockerd/master/packaging/systemd/cri-docker.socket
cp cri-docker.service cri-docker.socket /etc/systemd/system/
sed -i -e 's,/usr/bin/cri-dockerd,/usr/local/bin/cri-dockerd,' /etc/systemd/system/cri-docker.service

# 指定 pause 镜像 (k8s 1.35 推荐 3.10)
sed -i 's|ExecStart=.*|ExecStart=/usr/local/bin/cri-dockerd --container-runtime-endpoint fd:// --pod-infra-container-image=registry.k8s.io/pause:3.10|' /etc/systemd/system/cri-docker.service

systemctl daemon-reload
systemctl enable --now cri-docker.socket cri-docker

2.3 配置 Kubernetes 仓库

cat << EOF | tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.35/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.35/rpm/repodata/repomd.xml.key
EOF

三、准备Master&Worker节点

复制hep-k8s-master-worker-temp虚拟机，右键clone即可，然后修改主机名、IP地址。一共复制出三台Master四台Worker即可，这些节点都有上面配置好的内容。如果你是独立的Linux，可以在每台机器上都执行一下步骤二的所用内容。

image-20251225153733816

3.1 设置主机名&固定IP地址

# hep-k8s-master-01节点
# 设置主机名
hostnamectl set-hostname hep-k8s-master-01
#固定IP
nmcli connection modify ens18 ipv4.addresses 192.168.31.221/24
# 重启生效
reboot

# hep-k8s-master-02节点
# 设置主机名
hostnamectl set-hostname hep-k8s-master-02
#固定IP
nmcli connection modify ens18 ipv4.addresses 192.168.31.222/24
# 重启生效
reboot

# hep-k8s-master-03节点
# 设置主机名
hostnamectl set-hostname hep-k8s-master-03
#固定IP
nmcli connection modify ens18 ipv4.addresses 192.168.31.223/24
# 重启生效
reboot

# hep-k8s-worker-01节点
# 设置主机名
hostnamectl set-hostname hep-k8s-worker-01
#固定IP
nmcli connection modify ens18 ipv4.addresses 192.168.31.224/24
# 重启生效
reboot

# hep-k8s-worker-02节点
# 设置主机名
hostnamectl set-hostname hep-k8s-worker-02
#固定IP
nmcli connection modify ens18 ipv4.addresses 192.168.31.225/24
# 重启生效
reboot

# hep-k8s-worker-03节点
# 设置主机名
hostnamectl set-hostname hep-k8s-worker-03
#固定IP
nmcli connection modify ens18 ipv4.addresses 192.168.31.226/24
# 重启生效
reboot

# hep-k8s-worker-04节点
# 设置主机名
hostnamectl set-hostname hep-k8s-worker-04
#固定IP
nmcli connection modify ens18 ipv4.addresses 192.168.31.227/24
# 重启生效
reboot

3.2 开放端口号

为了集群的安全性考虑，我这里并没有完全关闭防火墙，而是采用需要哪个端口就打开哪个端口，这样也更符合企业使用习惯，也会具有更高的可靠性安全性。

3.2.1 Master节点

# 核心转发与网段信任 (必须先执行) 
firewall-cmd --permanent --direct --add-rule ipv4 filter FORWARD 0 -j ACCEPT
firewall-cmd --permanent --add-masquerade
firewall-cmd --permanent --zone=public --add-source=192.168.0.0/12
firewall-cmd --permanent --zone=public --add-source=10.96.0.0/12
firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.0.0/12" accept'
firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="10.96.0.0/12" accept'

#  Kubernetes 核心组件端口 
firewall-cmd --permanent --add-port=6443/tcp      # API Server
firewall-cmd --permanent --add-port=2379-2380/tcp # Etcd
firewall-cmd --permanent --add-port=10250/tcp     # Kubelet API
firewall-cmd --permanent --add-port=10257/tcp     # Kube-controller-manager
firewall-cmd --permanent --add-port=10259/tcp     # Kube-scheduler
firewall-cmd --permanent --add-port=9100/tcp      # Node Exporter (监控常用)

# 网络插件 (Calico & kube-vip) 
firewall-cmd --permanent --add-port=179/tcp       # BGP
firewall-cmd --permanent --add-port=5473/tcp      # Typha
firewall-cmd --permanent --add-port=4789/udp      # VXLAN
firewall-cmd --permanent --add-port=8472/udp      # Flannel/Other VXLAN (备用)

# 服务发现 (DNS) 
firewall-cmd --permanent --add-port=53/tcp
firewall-cmd --permanent --add-port=53/udp
firewall-cmd --permanent --add-port=9153/tcp     # CoreDNS Metrics

#  业务端口 
firewall-cmd --permanent --add-port=30000-32767/tcp # NodePort

# 补全 Controller 和 Scheduler 端口（虽然 10250-10259 包含了，但建议显式确认）
firewall-cmd --permanent --add-port=10257/tcp
firewall-cmd --permanent --add-port=10259/tcp
firewall-cmd --permanent --add-port=10256/tcp

# 立即生效
firewall-cmd --reload

3.2.2 worker节点

# 核心转发与网段信任
firewall-cmd --permanent --direct --add-rule ipv4 filter FORWARD 0 -j ACCEPT
firewall-cmd --permanent --add-masquerade
firewall-cmd --permanent --zone=public --add-source=192.168.0.0/12
firewall-cmd --permanent --zone=public --add-source=10.96.0.0/12
firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="192.168.0.0/12" accept'
firewall-cmd --permanent --add-rich-rule='rule family="ipv4" source address="10.96.0.0/12" accept'

# Kubernetes 组件与监控
firewall-cmd --permanent --add-port=10250/tcp     # Kubelet API
firewall-cmd --permanent --add-port=10256/tcp     # Kube-Proxy (Health check)
firewall-cmd --permanent --add-port=9100/tcp      # Node Exporter

# 网络插件 (Calico) 
firewall-cmd --permanent --add-port=179/tcp       # BGP
firewall-cmd --permanent --add-port=5473/tcp      # Typha
firewall-cmd --permanent --add-port=4789/udp      # VXLAN

#  服务发现与业务 
firewall-cmd --permanent --add-port=53/tcp
firewall-cmd --permanent --add-port=53/udp
firewall-cmd --permanent --add-port=30000-32767/tcp # NodePort

# 补全 Node Exporter 监控
firewall-cmd --permanent --add-port=9100/tcp
# 补全 Kube-Proxy 健康检查
firewall-cmd --permanent --add-port=10256/tcp

# 立即生效
firewall-cmd --reload

四、 集群初始化准备

4.1 K8S集群软件&容器镜像

# 安装指定版本的 K8s 核心组件
# Master节点安装kubelet、kubeadm、kubectl，Worker节点安装kubelet、kubeadm
# dnf install -y kubelet kubeadm --disableexcludes=kubernetes
dnf install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

# 设置 kubelet 开机自启
systemctl enable --now kubelet

# 查看 K8s 1.35.0 所需的镜像列表
kubeadm config images list
kubeadm config images list --kubernetes-version=v1.35.0

# 拉取 K8s 1.35.0 镜像(指定 cri-dockerd 容器运行时）
# 这个时候，你没有科学上网应该是拉取不了的，想知道怎么配置可以转到文章后面部分，了解一下。
kubeadm config images pull --cri-socket unix:///var/run/cri-dockerd.sock

4.2 云原生负载均衡器kube-vip准备

# 在Master01上执行
# 定义kube-vip所需环境变量
export VIP=192.168.31.220
export INTERFACE=ens18 # 注意查看你的网卡名，CentOS可能是 ens18 或 eth0
export KVVERSION=v1.0.3

docker run -it --rm --net=host ghcr.io/kube-vip/kube-vip:$KVVERSION manifest pod \
--interface $INTERFACE \
--address $VIP \
--controlplane \
--services \
--arp \
--enableLoadBalancer \
--leaderElection | tee /etc/kubernetes/manifests/kube-vip.yaml

# 同样将此文件分发到 master-02, master-03 的对应目录
# 将kube-vip.yaml文件复制到hep-k8s-master-02节点的对应目录
scp /etc/kubernetes/manifests/kube-vip.yaml hep-k8s-master-02:/etc/kubernetes/manifests/

# 将kube-vip.yaml文件复制到hep-k8s-master-03节点的对应目录
scp /etc/kubernetes/manifests/kube-vip.yaml hep-k8s-master-03:/etc/kubernetes/manifests/

五、K8S集群初始化

5.1 kubeadm-config.yaml配置

kubeadm-config.yaml文件的修改是重点，这个搞好了就成功了一半了。

# 生成配置文件样例 kubeadm-config.yaml
kubeadm config print init-defaults --component-configs KubeProxyConfiguration > kubeadm-config.yaml

# 修改这个配置文件以下内容
# advertiseAddress: 192.168.31.221，改成自己的主机地址
# criSocket: unix:///var/run/cri-dockerd.sock 使用cri-dockerd
# name: hep-k8s-master-01，自己的主机名
# 增加 certSANs: 认证证书配置，Master节点的主机名和IP都写上啦
  #- lb.k8s.hep.com
  #- hep-k8s-master-01
  #- hep-k8s-master-02
  #- hep-k8s-master-03
  #- 192.168.31.221
  #- 192.168.31.222
  #- 192.168.31.223
# 增加 controlPlaneEndpoint: "lb.k8s.hep.com:6443"，VIP地址和端口
# 增加 podSubnet: 192.168.0.0/12，和Calico 默认 Pod 子网匹配，当然也可以默认。我这里就没修改，采用默认的
# strictARP: true
# mode: "ipvs"
apiVersion: kubeadm.k8s.io/v1beta4
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: 192.168.31.221
nodeRegistration:
  criSocket: unix:///var/run/cri-dockerd.sock
  name: hep-k8s-master-01
---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
kubernetesVersion: v1.35.0
controlPlaneEndpoint: "lb.k8s.hep.com:6443"
apiServer:
  certSANs:
  - lb.k8s.hep.com
  - 192.168.31.220
  - 192.168.31.221
  - 192.168.31.222
  - 192.168.31.223
networking:
  podSubnet: 192.168.0.0/16
---
apiVersion: kubeproxy.config.k8s.io/v1alpha1
kind: KubeProxyConfiguration
mode: "ipvs"

5.2 master节点配置

# kubeadm 初始化前修改 kube-vip.yaml
sed -i 's#path: /etc/kubernetes/admin.conf#path: /etc/kubernetes/super-admin.conf#' /etc/kubernetes/manifests/kube-vip.yaml

# 预拉取镜像
kubeadm config images pull --cri-socket unix:///var/run/cri-dockerd.sock

# 执行初始化
kubeadm init --config kubeadm-config.yaml --upload-certs

[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes running the following command on each as root:

  kubeadm join lb.k8s.hep.com:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:4e83465atcbd1eb05aa8e9f7244a760565b0fa27c9db8cf5a41ea283856d715 \
        --control-plane --certificate-key 056c3140d0a4c1d06501bb040bb6dc959569fdfa49888ef0cd3efc6dd7edc60f

Please note that the certificate-key gives access to cluster sensitive data, keep it secret!
As a safeguard, uploaded-certs will be deleted in two hours; If necessary, you can use
"kubeadm init phase upload-certs --upload-certs" to reload certs afterward.

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join lb.k8s.hep.com:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:4e83465atcbd1eb05aa8e9f7244a760565b0fa27c9db8cf5a41ea283856d715
[root@hep-k8s-master-01 kelsen]#

# 完成后配置 kubectl
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

# hep-k8s-master-02、hep-k8s-master-03加入控制节点，一定带上--cri-socket unix:///var/run/cri-dockerd.sock参数
kubeadm join lb.k8s.hep.com:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:4e83465atcbd1eb05aa8e9f7244a760565b0fa27c9db8cf5a41ea283856d715 \
        --control-plane --certificate-key 056c3140d0a4c1d06501bb040bb6dc959569fdfa49888ef0cd3efc6dd7edc60f  --cri-socket unix:///var/run/cri-dockerd.sock

# hep-k8s-master-02、hep-k8s-master-03成功加入控制节点后，配置kubectl环境
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

5.3 worker节点配置

# hep-k8s-worker-01、hep-k8s-worker-02、hep-k8s-worker-03、hep-k8s-worker-04加入集群，一定带上--cri-socket unix:///var/run/cri-dockerd.sock参数
kubeadm join lb.k8s.hep.com:6443 --token abcdef.0123456789abcdef \
        --discovery-token-ca-cert-hash sha256:4e83465atcbd1eb05aa8e9f7244a760565b0fa27c9db8cf5a41ea283856d715  --cri-socket unix:///var/run/cri-dockerd.sock

5.4 安装网络插件 (Calico)

# 应用Calico Operator资源清单(部署Calico控制器）
kubectl create -f https://raw.githubusercontent.com/projectcalico/calico/v3.29.1/manifests/tigera-operator.yaml
# 查看tigera-operator是否为Running
kubectl get ns
kubectl get pods -n tigera-operator

# 下载Calico自定义资源配置文件
wget https://raw.githubusercontent.com/projectcalico/calico/v3.29.1/manifests/custom-resources.yaml

# 修改自定义资源文件(匹配kubeadm初始化的Pod网络CIDR），我这里没修改，用的默认192.168.0.0
vim custom-resources.yaml
# (修改第13行的cidr为kubeadm init --pod-network-cidr指定的地址，默认为192.168.0.0/16）

# 应用Calico自定义资源配置(完成Calico部署），大概过个五分钟，就都Running状态了
kubectl create -f custom-resources.yaml

kubectl get ns
kubectl get pods -n calico-system
kubectl get nodes

# 修改 Worker 节点 ROLES为worker
kubectl label node hep-k8s-worker-01 node-role.kubernetes.io/worker=worker
kubectl label node hep-k8s-worker-02 node-role.kubernetes.io/worker=worker
kubectl label node hep-k8s-worker-03 node-role.kubernetes.io/worker=worker
kubectl label node hep-k8s-worker-04 node-role.kubernetes.io/worker=worker
# worker的ROLES被打上worker的label了
kubectl get nodes

六、部署Nginx验证集群可用性

[root@hep-k8s-master-01 kelsen]# kubectl get service -n kube-system
NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
kube-dns   ClusterIP   10.96.0.10           53/UDP,53/TCP,9153/TCP   16h
[root@hep-k8s-master-01 kelsen]# dig -t a www.baidu.com @10.96.0.10

; <<>> DiG 9.18.33 <<>> -t a www.baidu.com @10.96.0.10
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 6108
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 8d225c06f081a7e7 (echoed)
;; QUESTION SECTION:
;www.baidu.com.                 IN      A

;; ANSWER SECTION:
www.baidu.com.          5       IN      CNAME   www.a.shifen.com.
www.a.shifen.com.       5       IN      CNAME   www.wshifen.com.
www.wshifen.com.        5       IN      A       103.235.46.102
www.wshifen.com.        5       IN      A       103.235.46.115

;; Query time: 250 msec
;; SERVER: 10.96.0.10#53(10.96.0.10) (UDP)
;; WHEN: Thu Dec 25 15:53:05 CST 2025
;; MSG SIZE  rcvd: 204

[root@hep-k8s-master-01 kelsen]#

6.2 利用K8S部署Nginx

# 创建一个nginx.yaml文件，其内容如下
vim nginx.yaml

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginxweb
spec:
  selector:
    matchLabels:
      app: nginxweb1
  replicas: 2
  template:
    metadata:
      labels:
        app: nginxweb1
    spec:
      containers:
      - name: nginxwebc
        image: nginx:latest
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginxweb-service
spec:
  externalTrafficPolicy: Cluster
  selector:
    app: nginxweb1
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
    nodePort: 30080
  type: NodePort

创建Nginx应用

kubectl apply -f nginx.yaml

kubectl get pods

kubectl get service

6.3 验证Nginx

# 在Master和worker所有机器上开通30080端口
firewall-cmd --permanent --add-port=30080/tcp
# 在局域网浏览器中访问http://192.168.31.224:30080/即可看到Nginx主页
# 三台Master+30080以及四台worker+30080都可以访问Nginx

image-20251225155810077

七、CentOS Stream 10 配置镜像加速

# 进入存放目录
cd /home/kelsen

# 下载兼容版核心
wget https://github.com/MetaCubeX/mihomo/releases/download/v1.18.9/mihomo-linux-amd64-compatible-v1.18.9.gz

# 解压并移动
gunzip -f mihomo-linux-amd64-compatible-v1.18.9.gz
chmod +x mihomo-linux-amd64-compatible-v1.18.9
mv -f mihomo-linux-amd64-compatible-v1.18.9 /usr/local/bin/mihomo

# 创建配置文件夹并下载资源
mkdir -p /root/.config/mihomo
# 请将下方链接替换为你真实的订阅地址
curl -L -o /root/.config/mihomo/config.yaml "你的Mihomo订阅链接"
# 下载地理位置库
curl -L -o /root/.config/mihomo/Country.mmdb https://testingcf.jsdelivr.net/gh/MetaCubeX/meta-rules-dat@release/geoip.metadb

# 再次尝试手动运行
/usr/local/bin/mihomo -d /root/.config/mihomo
# 清理旧进程并创建 Systemd 服务
sudo pkill -9 mihomo || true

cat <> ~/.bashrc << 'EOF'

# Mihomo Proxy Settings
export http_proxy="http://127.0.0.1:9981"
export https_proxy="http://127.0.0.1:9981"
# K8S 重要排除项
export no_proxy="localhost,127.0.0.1,192.168.31.0/24,10.96.0.0/12,192.168.0.0/16,lb.k8s.hep.com,.svc,.cluster.local"
EOF

# .bashrc生效
source ~/.bashrc

# 配置 Docker 代理
mkdir -p /etc/systemd/system/docker.service.d
cat << EOF | sudo tee /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://127.0.0.1:9981"
Environment="HTTPS_PROXY=http://127.0.0.1:9981"
Environment="NO_PROXY=localhost,127.0.0.1,192.168.31.0/24,lb.k8s.hep.com,.cluster.local"
EOF

systemctl daemon-reload
systemctl restart docker || echo "Docker 未安装，跳过重启"

#  验证
echo "正在测试代理连通性..."
curl -I https://www.google.com

# 预拉取 K8S 镜像
echo "正在预拉取 Kubernetes v1.35.0 镜像..."
# 注意：确保 cri-dockerd 已安装并运行
kubeadm config images pull --cri-socket unix:///var/run/cri-dockerd.sock

八、集群优雅开关机

8.1 K8S集群关机

# 如果是为了长期停机或维护，建议先清空节点。如果只是临时重启，可跳过此步。
# 在 master01 执行，循环处理 worker 节点
kubectl drain hep-k8s-worker-01 --ignore-daemonsets --delete-emptydir-data
# 对其他 worker02-04 重复此操作
kubectl drain hep-k8s-worker-02 --ignore-daemonsets --delete-emptydir-data
kubectl drain hep-k8s-worker-03 --ignore-daemonsets --delete-emptydir-data
kubectl drain hep-k8s-worker-04 --ignore-daemonsets --delete-emptydir-data
# 关闭所有 Worker Nodes
# 依次登录到四台 Worker 节点（01-04），执行关机
# 停止 kubelet，防止它在关机过程中尝试拉起容器
sudo systemctl stop kubelet
sudo systemctl stop containerd
sudo shutdown -h now
# 逐个关闭 Master 节点 (关键)
# 先关 Master 02 和 Master 03
sudo systemctl stop kubelet
sudo systemctl stop containerd
sudo shutdown -h now
# 最后关 Master 01 (VIP 承载者)： 最后关闭持有 VIP 的节点，确保控制平面在关机最后一刻依然可用。

8.2 K8S集群开机

# 同时开启 Master 01, 02, 03
# 检查 kube-vip： 由于使用了 kube-vip，它通常作为静态 Pod 运行。Master 节点启动后，检查 VIP 是否能够 Ping 通
ping 192.168.31.200
# 检查控制平面状态： 登录到 Master 01，观察核心组件和 etcd 状态
kubectl get nodes
kubectl get pods -n kube-system
# 启动 Worker 节点
# 一旦 kubectl get nodes 显示 Master 节点为 Ready 状态，即可启动所有 Worker 节点
kubectl uncordon hep-k8s-worker-01
kubectl uncordon hep-k8s-worker-02
kubectl uncordon hep-k8s-worker-03
kubectl uncordon hep-k8s-worker-04

九、Helm

Reference:

官方文档k8s1.30安装部署高可用集群，kubeadm安装Kubernetes1.30最新版本:https://blog.csdn.net/weixin_45652150/article/details/138492600

ubuntu22.04安装Kubernetes1.25.0(k8s1.25.0)高可用集群：http://www.huerpu.cc:7000/?p=432

60分钟极速部署企业级kubernetes k8s 1.35集群:https://www.bilibili.com/video/BV1oNqkBzEuy/

发表回复