Notes by a Sysadmin
The following is a HA setup of stacked etcd/cp on vanilla k8s, on rpi4 with 3 cp and 3 workers.
Storage is NFS nvme m2 hosted by one of the workers thru USB3.
OS: Ubuntu 22.04 lts server (arm).
Deploy tool: Kubeadm
Use the Image burner "Raspberry Pi Image burner",... Read more
Currently I had a problem with Longhorn not compatible with k3s version v1.25.4+k3s1. So, after a lot of debugging I came to the conclusion that I had to do a downgrade of the cluster. I guess almost the same procedure would apply for an upgrade.
As you probably have some more fine grained... Read more
By default Traefik is deployed in K3s. So, in production we would like automating valid wildcard certificate creation. In this case there are two main approaches to generate and store certificates; cert-manager and traefik acme. On it's own Traefik acme can be used to create and store the certificate, but to get HA this way is... Read more
The following will assume that a K3s cluster is already up and running. We want to make it a starting point of our cluster management with the Rancher server distribution.
Note: If the installation fail or you somehow decide to make a reinstall of rancher. You will need to make a complete cleanup of... Read more
K3s is Ranchers (SUSE) small kubernetes solution. In the following I'll try to create a small HA kubernetes cluster, made up of cheap VPS.
BTW: K3s can actually be run... Read more
Basically you would usually don't have to concern your self to much about this subject, when using the established cloud providers, as they usually have their own Load Blancer on the 2/3 OSI level. And networking in a K8s cluster is allready difficult for most people to get compfortably with. But, not to get confused with the... Read more
As private user and a Open Source advocate. There is only a couple of SIEM systems to choose from, one of them is an offspring of the probably most known of them all OSSEC, this system is called Wazuh. One of the great things about Wazuh is the abillity to deploy a scalable container setup. In the following i will try to set... Read more
Often times it comes in handy to encrypt data, that are used for automation. Sensitive data like passwords and other credidentials for infrastructure should be handled with a certain amount of precaution.
Ansible vault is an encrypted storage to Ansible, even though it is also possible to... Read more
Make a traefik.yml in you project directory:
entryPoints:
# http redirect to https
web:
address: ":80"
http:
redirections:
entryPoint:
to: websecure
scheme: https
# https
websecure:
address: ":443"
http:
tls:
... Read more
Start by setting up an overlay network:
docker network create --driver=overlay traefik-publicThen we will export some variables befor deploying out docker-compose.yml:
export EMAIL=admin@example.com
export... Read more
Using jwilders configuration. Go to you project directory and create one directory (atleast), "conf.d" and maybe "html". Create a docker-compose.yml in the directory to (remember to change to the right path). Of cause you also need to figure out if you are running nginx on one or several nodes, in regard to storage and... Read more
Create an image on Docker Hub, there are other ways of creating images. But, the following is based on a "Dockerfile".
"cd" into the directory of your Dockerfile, which is the formula for your image, and execute the following line:
docker build -t... Read more
Portainer is a GUI admin tool for Docker (and K8s). And it is almost a "must have", not that you can't do without. But, this tool is really help full in getting and understanding of your whole cluster, it is stable and great for debugging.
The following "Stack" deployment will setup two services that run... Read more
Now, I came to Swarm after have been in the K8s universe for a year or so. And I just fell in love with Docker Swarm from first try. So, I would encurage anyone working with Pi's to choose Docker Swarm over any K8s solution out there, any day under any circumstances!
The first think and most complicated is to install... Read more
Setting a MQTT broker up on Swarm.
I have setup a nfs share that resides on each node and mounting is done on this share. In preparation I have have made the directories in the nfs-share that the bind is going to use.
The "stack" deployment file would look like this (usually you would probably... Read more
Follow instructions on Docker for setting up Swarm and Portainer (easy)...
In Docker Swarm containers run in "services". Create a service:
docker service create --name redis --replicas=5 redis:3.0.6Docker "ls" to list services:
In my experience Swarm is the ideal orchestration tool for ARM based clusters. Wait! Isn't Kubernetes the future for orchestration, and will Docker not drop support for Swarm in 2022 (you might say). And yes, very sadly! Most datacenters and companies working with container orchestration is moving towards the more complex... Read more
There is a lot of different types of volume mounting, mainly because of the fact that Kubernetes (K8s) is often used for the purpose of high availabillity. So, the whole concept of storage on one node, is kind of opposed to that therminology. But, to try and make things simple here, we will use... Read more
Installation for MicroK8s can be found here: https://ubuntu.com/tutorials/how-to-kubernetes-cluster-on-raspberry-pi#1-overview
Enable the dashboard:
microk8s enable... Read more
There is a lot of tutorials out there about how to turn your Raspberry Pi's into a cluster. Before you do this yourself here are a couple of advices, depending of your goal with the project.
By default Raspbian currently only take advantage of 32 bit in the core and user land event thou it has an 64 bit CPU in the latest PPi 4. Depending of the purpose of your Raspberry Pi, you may want to take advantage of the 64 bit ARM architecture ( you may be developing some applications or maybe want to use it for container... Read more
Generel configuration and setup of different kinds of UNIX-like Read more