🪴 Sudipto

Search

SearchSearch

Networking setup for Kubernetes Cluster Setup

Nov 17, 20242 min read

While I was setting up my homelab K8s cluster, I faced a few networking, DNS and connectivity issues, while setting up a lot of stuff in my cluster. To ensure you don’t have to go through the same misery as I did, let’s get these out of the way:

  1. Allow iptables to accept forward requests on each of the nodes, otherwise networking won’t work - sudo iptables -P FORWARD ACCEPT. Once that’s done, we’ll need to ensure that this is persisted and isn’t wiped out on reboots. I had to do these in my Intel installation, but not on the cloud-init based installs on the Raspberry PIs. If you know why, lemme know.
    1. sudo apt install iptables-persistent
    2. Answer yes to saving the configs.
    3. sudo service netfilter-persistent restart
  2. If you have ufw running, and you can verify if it is with sudo ufw status, you’ll need it to allow routed packets on the calico interfaces (assuming we’ll be running a cluster which uses calico for CNI):
    1. sudo ufw default allow routed
    2. sudo ufw allow in on vxlan.calico
    3. sudo ufw allow out on vxlan.calico
  3. I had a Bind9 DNS server installed for resolving my LAN-only Domain. If you have the same.
    1. use 127.0.0.1 (bind9) for DNS resolution in the PKI CA Node, so that it can resolve itself - sudo resolvectl dns eth0 127.0.0.1 1.1.1.1 8.8.8.8
    2. Verify the DNS Servers - resolvectl dns eth0 (change the interface to the one you’re going to use for your cluster communications)
    3. For me step 1 here didn’t work. It was temporary and would get replaced in a while, leading to certificate renewal failures. So to address that, I had to:
      1. sudo vim /etc/systemd/resolved.conf
      2. Add this line in there: DNS=127.0.0.1 after the line that says [Resolve].
      3. Restart the systemd-resolve service - sudo service systemd-resolved restart
  4. Update /etc/hosts on each of your nodes, to have entries for each of your other nodes. Mine looks like this: 
    ...
127.0.1.1 k0r0ptserver0
127.0.0.1 localhost
192.168.1.230 k0r0ptnas
192.168.1.232 k0r0ptserver1
...
  5. If you’re using a cloud-init image (I was, with Ubuntu), you need to ensure that /etc/hosts is not regenerated upon restart, as that will wipe out your changes.
    1. Open the cloud-init config for editing vim /etc/cloud/cloud.cfg
    2. Comment out the line which says update_etc_hosts to - # - update_etc_hosts
    3. Save and Quit

Graph View

Backlinks