In this post, I’ll be detailing how I setup Cert Manager on my k0s Kubernetes cluster. I’ll be detailing the problems I faced, and how I got to the solutions.
First things first
To avoid the pitfalls, and countless hours of stressing that I had to face, ensure you follow the networking setup steps for Kubernetes Cluster Setup.
Install k0s
In case you haven’t you can go through How I setup my bare metal Kubernetes Cluster with k0s.
Install Cert Manager
While I struggled a lot with this setup in microk8s, mostly because it’s bundled with an older version, which doesn’t support private acme for some reason, it was a breeze with k0s. I just had to ensure that the feature gate AdditionalCertificateOutputFormats is enabled.
I used Helm to install it.
Add the helm repo
helm repo add jetstack https://charts.jetstack.io --force-update
Update Helm repos
helm repo update
Do the helm install
helm upgrade --install \
cert-manager jetstack/cert-manager \
--namespace cert-manager \
--create-namespace \
--version v1.18.2 \
--set crds.enabled=true \
--set config.featureGates.AdditionalCertificateOutputFormats=true \
--set webhook.config.featureGates.AdditionalCertificateOutputFormats=true \
--set global.leaderElection.namespace=cert-manager
Note that with version 1.18+, running with nginx version 1.8.0+, the nginx setup will need --set controller.config.strict-validate-path-type=false otherwise cert-manager will fail because nginx won’t allow a dot in the value when PathType is Exact.
Setup CoreDNS name resolution with private DNS server
To be able to resolve our private subdomains inside Kubernetes, I had to edit the coredns configuration, to include the name server for my subdomains, I previously had put that entry in its /etc/hosts file, by adding a hosts section in the config, which would resolve our private CA (there’s another way of doing this by editing resolv.conf on each of the nodes in the cluster, but this is how I did it). Also adding forward DNS resolver with fallthrough, for subdomains of my domain - kubectl -n kube-system edit configmap coredns:
...
k0r0pt.int {
forward . 192.168.1.230
}
.:53 {
...
hosts {
192.168.1.230 ca.k0r0pt.int
fallthrough
}
prometheus :9153
forward . /etc/resolv.conf
...
}
...After editing it, tail the coredns logs to ensure that the new config was picked up and is reloaded: kubectl -n kube-system logs -f deployment/coredns.