My Microk8s Based Kubernetes Home Lab

Table of Contents

I have a couple old laptops laying around and figured they would make for a decent kubernetes cluster. Initially I had planned to use kubeadm and set up the cluster somewhat manually - but I had ran into some issues. Not wanting to get bogged down I turned to microk8s in order to bootstrap a “production ready” cluster.

The cluster consists of three devices; an old ‘13 macbook air, an old fujitsu laptop, and my desktop. As these devices are dedicated for use in the cluster there are no worker nodes, each is a tainted control plane.

Installing Snapd & Microk8s

The ease of installing and setting up a kubernetes cluster with microk8s is astonishing.

  1. install snapd - sudo apt install -y snapd
  2. install the microk8s snap - sudo snap install microk8s --classic --channel=1.25/stable
  3. the snap channel flag corresponds with the version kubeadm
  4. change permissions - sudo usermod -a -G microk8s $USER & sudo chown -f -R $USER ~/.kube
  5. reopen your terminal or su <user> -
  6. check if microk8s is running - microk8s status --wait-ready
  7. add nodes - microk8s add-node. The join command and token displayed are only good for adding one host. You’ll have to run the add-node command again to add a second.

Initial Cluster Deployments

Another beneficial part of using microk8s is the core and community addons that can be easily installed via single enable command.

Metrics & Dashboard

Metrics I installed via the microk8s addon. However, I ran into an issue with the kubernetes dashboard addon where the workload graphs weren’t being displayed. In fact I was getting a 404 for the graphs themselves.

I deployed the kubernetes dashboard manually which is a simple one line command. kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.6.1/aio/deploy/recommended.yaml

If you deploy the dashboard manually keep in mind you’ll have to create a user to obtain a token - if that’s your desired setup. Or, you can be lazy like me and simply use your config file 😉

Mayastor / OpenEBS

One hiccup that had me avoiding microk8s until now was storage. Out of the box microk8s comes with an storage addon that only applies to each host - meaning pods will scale horizontally but the data wont.

Microk8s has an addon that bridges that wide AF gap via Mayastor / OpenEBS. From what I gather it creates .img files akin to what I would see used as an HDD in a QEMU virtual machine keeping them synchronised across nodes. However the docs do mention for production it’s recommended to use dedicated partitions / drives over .img files.

📝 NOTE - whilst I set up the addon I haven’t had a chance to deploy anything using the storage classes.

💡 READ ME - follow the microk8s mayastor tutorial on setting up the addon. I had an issue with one of my hosts not playing nicely and had to scale back the daemonset to two hosts.

Portainer

While I’m still learning kubernetes deployments AND helm templating, Portainer UI helps simply things allows me to more easily grow accustomed to standards. To deploy Portainer I opted to use the microk8s community addon which deploys Portainer to the cluster using NodePort for access via randomly assigned port.

Posts in this series