2: Setting Up a Kubernetes (K8s) Cluster from Scratch

Kubernetes (K8s) is a powerful container orchestration tool that simplifies the management of applications in a clustered environment. Setting up a K8s cluster involves multiple steps, including creating virtual machines (VMs), configuring networking, and installing necessary components. In this blog, we’ll walk through the process of setting up a basic Kubernetes cluster using VMs.

Step 1: Create 3 Virtual Machines (VMs)

To create a Kubernetes cluster, you need one master node and two worker nodes. Here’s what you need for each VM:

• Master Node (1 VM): At least 2GB RAM and a static IP.

• Worker Nodes (2 VMs): Each with at least 2GB RAM and their own static IPs.

Steps to set up the VMs:

1. Set up 3 virtual machines, ensuring each VM has:

   • 2GB of RAM.

   • An additional network adapter to allow the VMs to communicate over a static network.

2. During the installation of Ubuntu Server on each VM, configure the second network adapter (eth1) to use static IP addresses.

Step 2: Configure Static IP Addresses

You’ll use the second network adapter to assign static IP addresses in the 10.0.0.0/8 subnet. Follow these IP configurations:

• Master Node: IP 10.0.0.100

• Worker Node 1: IP 10.0.0.1

• Worker Node 2: IP 10.0.0.2

Steps to set static IPs:

1. Open the network configuration file:

    sudo nano /etc/netplan/00-installer-config.yaml
   

2. Modify the file to look like the following for the Master Node:

     codenetwork:
      ethernets:
        eth1:
          dhcp4: no
          addresses:
            - 10.0.0.100/8
          gateway4: 10.0.0.1
          nameservers:
            addresses: [8.8.8.8, 8.8.4.4]
      version: 2
   

   For the worker nodes, replace 10.0.0.100 with the IPs 10.0.0.1 and 10.0.0.2, respectively.

3. Apply the network configuration:

    sudo netplan apply
   

Step 3: Verify Network Connectivity

After configuring static IPs, ensure that the VMs can communicate with each other. You can use the ping command to test this.

On the Master Node, try pinging the worker nodes:

ping 10.0.0.1  # Ping Worker Node 1
ping 10.0.0.2  # Ping Worker Node 2

Similarly, test the reverse from the worker nodes to ensure the master node is reachable:

ping 10.0.0.100  # Ping the Master Node from Worker Nodes

Once the VMs can successfully ping each other, you’ve confirmed that the network is set up properly. This static IP setup is crucial for a smooth Kubernetes installation, as the master node must be able to control the worker nodes over the network.

Step 4: Disable Swap

Kubernetes requires swap to be disabled to function correctly. Disabling swap prevents the system from using disk space as memory, which can cause performance issues in a Kubernetes cluster.

To temporarily disable swap, use the following command:

sudo swapoff -a

However, this is not persistent and will be reset after a reboot. To make sure swap is disabled permanently, you need to edit your system’s configuration files, like /etc/fstab or systemd.swap, depending on your setup.

1. Open the /etc/fstab file:

    sudo nano /etc/fstab
   

2. Find the line that starts with "swap" and comment it out by adding a # at the beginning of the line.

Once this is done, swap will remain disabled even after rebooting your VMs.

Step 5: Enable IPv4 Packet Forwarding

Next, we need to ensure that IPv4 packet forwarding is enabled. This allows packets to flow between network interfaces, which is necessary for Kubernetes networking.

To enable IPv4 packet forwarding, follow these steps:

1. Create a new sysctl configuration file for Kubernetes:

    cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
    net.ipv4.ip_forward = 1
    EOF
   

2. Apply the new settings without restarting the machine:

    sudo sysctl --system
   

3. Verify that IPv4 forwarding is enabled:

    sysctl net.ipv4.ip_forward
   

If the output is net.ipv4.ip_forward = 1, you're good to go!

Step 6: Install containord as the Container Runtime

Kubernetes requires a container runtime to manage and run containers. One of the most popular runtimes is Docker. Let’s go through the process of installing Docker on your VMs.

Method: Install Docker via the Apt Repository

Step 1: Set up the Docker Repository

1. Update the package index and install prerequisites:

    sudo apt-get update
    sudo apt-get install ca-certificates curl
   

2. Create a directory for Docker’s GPG key:

    sudo install -m 0755 -d /etc/apt/keyrings
   

3. Download Docker’s official GPG key:

    sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
   

4. Set the correct permissions for the key:

    sudo chmod a+r /etc/apt/keyrings/docker.asc
   

5. Add Docker’s repository to your apt sources:

    echo \
      "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu \
      $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
      sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
   

6. Update the apt package index again:

    sudo apt-get update
   

Step 2: Install Docker

To install Docker and related components (Docker CE, CLI, and containerd), run the following command:

sudo apt-get install containerd.io

This will install Docker Engine and the necessary tools for container management. Docker is now ready to be used as a runtime for Kubernetes.

Step 7: Configure containerd for Kubernetes

Kubernetes uses containerd as the container runtime underneath Docker, so it's essential to configure it properly. After installing Docker, containerd comes pre-installed, but we need to ensure its configuration is aligned with Kubernetes requirements.

Step 1: Reset the containerd Configuration

To reset the configuration of containerd, run the following command, which outputs the default configuration to the config.toml file:

sudo containerd config default > /etc/containerd/config.toml #maybe it give you error just give chatgpt and you can solve this

This will overwrite any existing configuration with the default settings.

Step 2: Configure cgroups

To enable Kubernetes to manage containers properly, we need to ensure that the systemd cgroup driver is used. The cgroup driver allows Kubernetes to manage resources such as CPU and memory efficiently.

1. Open the containerd configuration file in your favorite text editor:

    sudo nano /etc/containerd/config.toml
   

2. Find the section labeled [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]. Underneath it, look for the SystemdCgroup parameter.

3. Set SystemdCgroup to true:

    [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
       SystemdCgroup = true
   

This change ensures that Kubernetes will use systemd as the cgroup manager instead of the default cgroupfs.

Step 3: Restart containerd

Once you've made these changes, restart the containerd service to apply the new configuration:

sudo systemctl restart containerd

Verification

To verify that containerd is running correctly and the configuration changes are in effect, check the status of the service:

sudo systemctl status containerd

This should show that the service is running smoothly. At this point, you have successfully configured containerd for Kubernetes.

Step 8: Install Kubernetes Components

Kubernetes consists of three key components:

• kubelet: An agent that runs on each node in the cluster.

• kubeadm: A tool to bootstrap the Kubernetes cluster.

• kubectl: The command-line interface to interact with the cluster.

Step 1: Prepare the System

Before installing Kubernetes components, we need to install the necessary dependencies and configure the Kubernetes apt repository.

1. Update the apt package index and install packages required to use the Kubernetes apt repository:

    sudo apt-get update
    sudo apt-get install -y apt-transport-https ca-certificates curl gpg
   

2. Download the Kubernetes public signing key to verify the Kubernetes packages:

    sudo mkdir -p -m 755 /etc/apt/keyrings
    curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.31/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
   

   Note: On older distributions (e.g., Debian 12, Ubuntu 22.04), the /etc/apt/keyrings directory may not exist by default. In this case, ensure the directory is created before running the curl command.

Step 2: Add the Kubernetes Repository

We need to add the Kubernetes v1.31 repository to the apt sources list to install the correct packages.

1. Add the Kubernetes repository:

    echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.31/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
   

Step 3: Install kubelet, kubeadm, and kubectl

Now that the repository is set up, we can install the Kubernetes components.

1. Update the apt package index to reflect the new repository:

    sudo apt-get update
   

2. Install the Kubernetes components:

    sudo apt-get install -y kubelet kubeadm kubectl
   

3. Hold the versions of these packages to prevent unintended upgrades:

    sudo apt-mark hold kubelet kubeadm kubectl
   

   This ensures that your Kubernetes setup remains stable, even if the system is updated.

Step 4: Enable the kubelet Service

Kubelet is responsible for managing the lifecycle of pods and containers on each node. To ensure that it runs at startup, we enable the kubelet service.

1. Enable and start kubelet:

    sudo systemctl enable --now kubelet
   

At this point, Kubernetes components are installed and running. You now have kubelet, kubeadm, and kubectl ready to manage the Kubernetes cluster.

Step 9: Initialize the Kubernetes Control Plane

On the master node, we need to initialize the control plane using kubeadm. This will configure the master node to run the Kubernetes API server, etcd (the distributed database), and the controller-manager.

1. Run kubeadm init with the appropriate options:

    sudo kubeadm init --apiserver-advertise-address=10.0.0.100 --pod-network-cidr=172.16.0.0/16
   

   • --apiserver-advertise-address: This is the static IP address we assigned to the master node (10.0.0.100).

   • --pod-network-cidr: This specifies the CIDR for the pod network. In this example, we use 172.16.0.0/16.

Once the initialization is complete, kubeadm will output a set of instructions, including a kubeadm join command. Save this command, as you’ll need it later to add worker nodes to the cluster.

Step 10: Configure Kubectl on the Master Node

After the control plane is initialized, you need to set up the Kubernetes configuration for kubectl so that you can interact with your cluster.

1. Create a directory for the kube config:

    mkdir -p $HOME/.kube
   

2. Copy the Kubernetes admin configuration:

    sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
   

3. Change ownership of the configuration file to your user so that you can access it without sudo:

    sudo chown $(id -u):$(id -g) $HOME/.kube/config
   

Step 11: Verify the Master Node

To check the status of the master node, use the following kubectl commands:

1. Verify the master node's status:

    kubectl get no
   

2. Monitor the node status with a live update:

    watch kubectl get no
   

   The output should display the master node in a Ready state once it's fully initialized.

Step 12: Install a Pod Network (Calico)

Kubernetes requires a network plugin to manage the communication between pods. One popular choice is Calico, which provides networking and network policy features.

1. Download the Calico manifest:

    curl https://raw.githubusercontent.com/projectcalico/calico/v3.28.0/manifests/calico.yaml -O
   

2. Apply the Calico manifest to the cluster:

    kubectl apply -f calico.yaml
   

3. Monitor the node status again:

    watch kubectl get no
   

   Once the Calico plugin is applied, the master node will transition to the Ready state.

Step 13: Verify Pod Network and System Pods

After applying Calico, verify that the system pods are running correctly:

1. Check the status of the Kubernetes system pods:

    kubectl -n kube-system get pods
   

2. Monitor the system pods in real-time:

    watch kubectl -n kube-system get pods
   

   All system pods should be in a Running state.

Final Check

Now that everything is set up, verify the master node one final time:

kubectl get no

You should see the master node marked as Ready, and the cluster is now ready for the worker nodes to join.

Step 14: Check the Internal IP Address

First, verify the internal IP addresses using the following command:

kubectl get no -o wide

If the INTERNAL-IP addresses do not match your static IP assignments, you'll need to make the necessary updates in the kubelet configuration file.

Step 15: Manually Update the Node IP

We'll specify the correct static IP for each node in the kubelet configuration file.

On the Master Node

1. Open the kubelet configuration file using a text editor:

    sudo vim /etc/default/kubelet
   

2. Add the following line to specify the node's static IP:

    KUBELET_EXTRA_ARGS="--node-ip=10.0.0.100"
   

   This line sets the static IP for the master node.

3. Save the file and exit the editor.

4. Reload the systemd daemon to recognize the changes:

    sudo systemctl daemon-reload
   

5. Restart the kubelet service to apply the changes:

    sudo systemctl restart kubelet
   

On the Worker Nodes

Repeat the same steps for each worker node, using the appropriate static IP addresses.

1. Open the kubelet configuration:

    sudo vim /etc/default/kubelet
   

2. Add the correct static IP for the worker node:

    KUBELET_EXTRA_ARGS="--node-ip=10.0.0.1"  # For the first worker node
   

   Or:

    KUBELET_EXTRA_ARGS="--node-ip=10.0.0.2"  # For the second worker node
   

3. Save the changes and exit the editor.

4. Reload the systemd daemon:

    sudo systemctl daemon-reload
   

5. Restart the kubelet service:

    sudo systemctl restart kubelet
   

Step 16: Verify the Internal IP Update

After updating the IP configuration for all nodes, run the following command again to verify that the INTERNAL-IP now reflects the correct static IP addresses:

kubectl get no -o wide

The INTERNAL-IP column should now display the static IPs you set for each node (master and workers).