The Hotel Hero

Notes by a Sysadmin


Cluster | Philosophy | Stack

Rancher server HA on k3s

March 29, 2022 | Cluster

The following will assume that a K3s cluster is already up and running. We want to make it a starting point of our cluster management with the Rancher server distribution.

Note: If the installation fail or you somehow decide to make a reinstall of rancher. You will need to make a complete cleanup of both Cert-manager and Rancher server.

Start by adding the Rancher server to the helm repo:

helm repo add rancher-stable https://releases.rancher.com/server-charts/stable

and adding a namespace for Rancher:

kubectl create namespace cattle-system

Cert-manager

Cert-manager is used to keep track of our certificate, it will be selfsigned by our Rancer server (it can be setup to use Letsencrypt instead). Start by adding jetpack to the repo:

helm repo add jetstack https://charts.jetstack.io

update every repo before continue:

helm repo update

Install the latest CRD from Cert-manager:

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.11.0/cert-manager.crds.yaml

Install with helm:

helm install \
  cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --version v1.11.0

Rancher install continued..

Now, install Rancher:

helm install rancher rancher-stable/rancher \
  --namespace cattle-system \
  --set hostname=rancher.my.org \
  --set bootstrapPassword=somepassword

IngressRoute

Now, as Traefik as our default ingress/reverse proxy we will stick to that. To access our new Cluster Manager we need to create a route (ran_ingress.yml):

apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: rancher-ingr
  namespace: cattle-system
spec:
  entryPoints:
    - web
  routes:
    - match: Host(`rancher.your_domain.com`) # or a bit less obvious subdomain
      kind: Rule
      services:
        - name: rancher
          port: 443

And create the ingressroute:

kubectl create -f ran_ingress.yml

If everything went fine, you should now be able to access the cluster remotely (with a selfsigned cert.)

Troubleshooting

Password error, somehow reinstall of Rancher can cause issues with the bootstrap password. I did a lot of search, uninstall and deletion of everything related to cattle-system and cert-manager, but somehow etcd keeps the old password and do not update on reinstall. After a lot of research I found a solution on StackOverflow:

kubectl -n cattle-system exec $(kubectl -n cattle-system get pods -l app=rancher | grep '1/1' | head -1 | awk '{ print $1 }') -- reset-password

This line resets the bootstrap password.


About

I'm a Sysadmin, network manager and cyber security entusiast. The main purpose of this public "notebook" is for referencing repetitive tasks, but it might as well come in handy to others. Windows can not be supported! But all other OS compliant with the POSIX-standard can (with minor adjustments) apply the configs on the site. It is Mac OSX, RHEL and all the Fedora based distros and Debian based (several 100's of OS's), all the BSD distros, Solaris, AIX and HP-UX.

Links