🪴 Sudipto

Search

SearchSearch

Setting up a Bare Metal Kubernetes Cluster with k0s

Nov 17, 20243 min read

Recently, I started setting up a Kubernetes Cluster on my 3 Raspberry Pi 4B’s and an Intel N100 NUC, specifically, SkullSaints Agni. I tinkered with microk8s for a while, but found a bunch of problems with it and Cert Manager (namely microk8s bundled Cert manager not supporting Step-CA based ACME configuration), and finally landed on k0s.

For storage, while microk8s’s hostpath storage would have worked for me, if I tried setting that up in my k0s cluster, but I didn’t take that route. I fiddled with OpenEBS, but turns out, it don’t support nodes based on ARM64 architecture, so 3 of my 4 nodes would have been useless. That’s when I decided to go with LongHorn.

First things first

To avoid the pitfalls, and countless hours of stressing that I had to face, ensure you follow the networking setup steps for Kubernetes Cluster Setup.

Install k0s

For this part, all I did was follow the instructions in the k0s Quickstart Guide.

Setting up the k0s cluster

Setting this up was easy peasy, thanks to their documentation about installing k0s with k0sctl. I’m demonstrating what I did on my RasPis, but you can get your architecture’s release from the k0sctl Releases page. Here’s what I did:

SSH Connectivity

I made private keys in each of my nodes, and copied them to all the other nodes.

1. `ssh-keygen`
2. `ssh-copy-id xtreme@k0r0pserver0 && ssh-copy-id xtreme@k0r0pserver1 ssh-copy-id xtreme@k0r0pserver2`
3. The next order of business was to copy the ssh keys for root, so that we can ssh into the root user (this is needed because `k0sctl` needs to ssh as root into all the nodes from each other).
	1. `ssh xtreme@k0r0ptserver0`
	2. `sudo su`
	3. `cp /home/xtreme/.ssh/authorized_keys ~/.ssh/`

Installing k0sctl

Next order of business, install k0sctl

  1. wget https://github.com/k0sproject/k0sctl/releases/download/v0.19.4/k0sctl-linux-arm64
  2. chmod +x k0sctl-linux-arm64
  3. sudo mv k0sctl-linux-arm64 /usr/bin/k0sctl
  4. Then I wrote my Cluster config yaml - vim k0s-cluster.yml, and put this config in there. You’d need to change the addresses in there
apiVersion: k0sctl.k0sproject.io/v1beta1
kind: Cluster
metadata:
  name: k0s-cluster
spec:
  hosts:
  - role: controller
    ssh:
      address: 192.168.1.230
      user: root
      keyPath: ~/.ssh/id_rsa
  - role: worker
    ssh:
      address: 192.168.1.231
      user: root
      keyPath: ~/.ssh/id_rsa
  - role: worker
    ssh:
      address: 192.168.1.232
      user: root
      keyPath: ~/.ssh/id_rsa
  - role: worker
    ssh:
      address: 192.168.1.233
      user: root
      keyPath: ~/.ssh/id_rsa
  1. Then I had to just k0sctl apply --config k0s-cluster.yml

Setting up kubectl

  1. Install kubectl with snap install kubectl --classic.
  2. Install our cluster config with this - k0sctl kubeconfig --config k0s-cluster.yml > .kube/config

Verify the Nodes are all there

Now that the cluster has been setup, a simple kubectl get no should show that the nodes are up and running.

Next Steps

After setting up the k0s cluster, I did a bunch of other things to make stuff work on the cluster:

Graph View

Backlinks