Rancher series of articles - Install a highly available Rancher cluster on Tencent Cloud's K3S

This article was last updated on: February 7, 2024 pm


📜 introduction

  • A threesome will have my master
  • Knowledge sharing, the world for the public


Install Rancher on Tencent Cloud’s K3S

Programme objectives

  1. High availability
    1. 3 master k3s clusters
    2. Rancher in high-availability mode
  2. Data backup
    1. Rancher data is backed up to Tencent Cloud Object Storage COS
  3. Secure encryption
    1. There can be no http, all https
  4. Customer-facing
    1. Public network accessibility;
    2. Domain name accessible;
    3. Official certificate
  5. Reuse the capabilities of the public cloud as much as possible
    1. ~~Tencent Cloud Controller Manager~~ (❌ Because Tencent Cloud has given up maintaining the relevant source code, it cannot be reused)
    2. ~~SVC LoadBalancer calls CLB~~ (❌ Because Tencent Cloud has given up maintaining the relevant source code, it cannot be reused)
    3. CLB - Uses a 4-layer CLB
    4. Backup - Use Tencent Cloud COS


  1. If you have a Tencent Cloud account, the account has at least the following permissions:auto k3s installation - Set up CAM and these permissions:

    1. QcloudTAGFullAccess
  2. The Tencent Cloud account has the corresponding API key and address:Access Keys - Console (tencent.com) , or have the relevant permissions:cam:QueryCollApiKeyand cam:CreateCollApiKey

  3. An object storage pass cos for backup

  4. Rancher’s domain name

  5. Rancher’s domain name certificate (if not, try to automatically generate a free certificate via cert-manager and let’s encrypt)


Rancher installation considerations

  1. Highly available installation via Helm Chart

  2. Adjustments are required before installation:

    1. Security groups
  3. After installation, you need to configure:

    1. LB
    2. Backup
  4. ⚠️ Paid model, COS adjusts the payment model as needed.

Installation steps


🚩 Important:

Install via Helm Chart

Rancher port requirements

📚️ Quote:

The port of the Rancher Server node on K3s

Inbound rules for Rancher Server nodes

Agreement Port Source Description
TCP 80 Load balancer/proxy, which is an external SSL termination Rancher UI/API when terminated using external SSL
TCP 443 Server Node Agent Node hosted/registered Kubernetes for any source that needs to be able to use the Rancher UI or API Rancher agent, Rancher UI/API, kubectl
TCP 6443 K3s server node Kubernetes API

Finally, the specific security group configuration is as follows: (It should be possible to tighten further)

Rancher is a highly available installation

Install the helm chart first and create ns:

helm repo add rancher-stable http://rancher-mirror.oss-cn-beijing.aliyuncs.com/server-charts/stable

kubectl create namespace cattle-system

The SSL options are:Existing certificates, install Rancher via Helm:

📚️ Quote:

Depending on the SSL option you choose, install Rancher through Helm

Add the certificate to the k8s secret first:

kubectl -n cattle-system create secret tls tls-rancher-ingress \
--cert=tls.crt \
helm install rancher rancher-stable/rancher \
--namespace cattle-system \
--set hostname=<your-rancher-domain> \
--set replicas=3 \
--set ingress.tls.source=secret \
--set systemDefaultRegistry=registry.cn-hangzhou.aliyuncs.com \
--set auditLog.level=1 \

The output after running is as follows:

NAME: rancher
LAST DEPLOYED: Sat Feb 12 20:10:14 2022
NAMESPACE: cattle-system
STATUS: deployed
Rancher Server has been installed.

NOTE: Rancher may take several minutes to fully initialize. Please standby while Certificates are being issued, Containers are started and the Ingress rule comes up.

Check out our docs at https://rancher.com/docs/

If you provided your own bootstrap password during installation, browse to https://<your-rancher-domain> to get started.

If this is the first time you installed Rancher, get started by running this command and clicking the URL it generates:

echo https://<your-rancher-domain>/dashboard/?setup=$(kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{.data.bootstrapPassword|base64decode}}')

To get just the bootstrap password on its own, run:

kubectl get secret --namespace cattle-system bootstrap-secret -o go-template='{{.data.bootstrapPassword|base64decode}}{{"\n"}}'

Happy Containering!

🔥 Notice:

Note that the 443 permission of the Rancher domain name must be enabled.

ℹ️ Info:

  • To install a specific version of Rancher, use--version Flags, such as:--version 2.3.6

After that, access the UI for initial password setting and other work.

🎉 At this point, the Rancher high-availability cluster installation is complete.

Rancher China optimizes configuration

Refer here:

Finishing touches

Adjust the security group

Inbound rules:

  1. TCP:22 (SSH) port permissions are tightened
  2. TCP:6443 (K8S API) Port permissions are tightened
  3. UDP: 8472 (K3s vxlan) is only open to intranets
  4. TCP:10250 (kube api-server) is only available to intranets

The end effect is as follows: (should be able to tighten further)

Configure LB

📚️ Quote:

External TLS Termination:

We recommend configuring the load balancer as a Layer 4 equalizer that forwards normal 80/tcp and 443/tcp to the Rancher management cluster nodes. The Ingress Controller on the cluster redirects http traffic on port 80 to https on port 443.

如上面所述,所以通过 4 层 LB, 将 443/tcp 转到后端。如下图:

配置 Rancher Backup

📚️ Quote:

Rancher v2.5 中的 Backup 和恢复 | Rancher 文档

Backup Rancher | Rancher 文档

Rancher Backup Examples

通过 UI 安装:

先创建 cos 存储的认证信息 Secret:

apiVersion: v1
accessKey: <your-ak>
secretKey: <your-sk>
kind: Secret
name: cos-creds
namespace: cattle-resources-system
type: Opaque

然后在 应用市场 选择 Rancher Backup 安装:


配置 对象存储:


helm upgrade --install=true --namespace=cattle-resources-system --timeout=10m0s --values=/home/shell/helm/values-rancher-backup-crd-2.1.0.yaml --version=2.1.0 --wait=true rancher-backup-crd /home/shell/helm/rancher-backup-crd-2.1.0.tgz
SUCCESS: helm upgrade --install=true --namespace=cattle-resources-system --timeout=10m0s --values=/home/shell/helm/values-rancher-backup-crd-2.1.0.yaml --version=2.1.0 --wait=true rancher-backup-crd /home/shell/helm/rancher-backup-crd-2.1.0.tgz
helm upgrade --install=true --namespace=cattle-resources-system --timeout=10m0s --values=/home/shell/helm/values-rancher-backup-2.1.0.yaml --version=2.1.0 --wait=true rancher-backup /home/shell/helm/rancher-backup-2.1.0.tgz
SUCCESS: helm upgrade --install=true --namespace=cattle-resources-system --timeout=10m0s --values=/home/shell/helm/values-rancher-backup-2.1.0.yaml --version=2.1.0 --wait=true rancher-backup /home/shell/helm/rancher-backup-2.1.0.tgz

配置 *Backup*, 如下:


🎉 登录 COS 发现已经成功 Backup。

## 总结

🎉🎉🎉 至此,完成腾讯云上 K3S 高可用集群 及 Rancher 高可用集群的搭建,并配置 Backup。


### K3s

1.  3 个 Master 和 Server 地址

2.  K3S API Server 地址:`https://<3 个 master IP 地址任一个 >:6443` (6443 端口目前没有配置 CLB)

3.  K3S kubeconfig 配置:位于 k3s 的 `/etc/rancher/k3s/k3s.yaml` 以及操作机的 `/root/.autok3s/.kube/config`

### Rancher

1.  地址:
    1.  公网访问:`https://<your-rancher-domain>:<port>/`
    2.  内网访问:`https://<your-rancher-domain>:443` (需要编辑自己电脑的 `hosts` , 将 3 个 master 任一内网 IP 映射到该域名)
2.  账号:`Admin`
3.  密码

### 安全组



### CLB

使用的 CLB

监听器为:`rancher(TCP:<port>)` 转到 3 台 master 的 443 端口。

### Backup COS

K3S 和 Rancher 都配置了 Backup,Backup 到对象存储 cos 中。具体的地址为:

1.  桶:rancher-backup-<cos-id>
2.  域名:`https://rancher-backup-<cos-id>.cos.ap-shanghai.myqcloud.com`
3.  S3 Endpoint: `cos.ap-shanghai.myqcloud.com`
4.  文件夹为:
    1.  k3s 为:`/rancher-1/rancher/rancher`(Backup 策略:每天 0 点 Backup,保留 5 份)
    2.  rancher 为:`/rancher-1/rancher/k3s` (Backup 策略,每天 0 点 Backup)
5.  COS 生命周期为:自动清理 1 个月前的文件。(配置 [自动清理规则](https://console.cloud.tencent.com/cos/bucket?bucket=rancher-backup-1258988025\&region=ap-shanghai\&type=basicconfig\&anchorType=lifeCycle))

Rancher series of articles - Install a highly available Rancher cluster on Tencent Cloud's K3S
Posted on
February 20, 2022
Licensed under