Kubernetes – Setting up CentOS 7 Clusters (Without manual interventions)

In this guide, we shall discuss the configuration steps to set up Kubernetes in a CentOS 7 minimal cluster.

VM Setup

As aforementioned we are using a cluster running CentOS 7 minimal version. In this cluster, we expect to set up a master and 2 minions.

First of all, it is essential to configure the network of VMs as follows.

Setting up network

We will first set up 2 interfaces on each VM, one as a NAT interface with DHCP and the other as a host-only interface with a static IP. The NAT interface would allow the VM to access the internet from within the VM while the host-only interface will allow the host to connect with the VM.

  1. Check the network interfaces on your device using the command
nmcli d
After running nmcli d command

2. Go to the network manager using the command

nmtui

Hit Edit a connection.

Network Manager

There you will see a list of interfaces.

3. Choose the interfaces that you would keep as the host-only interfaces and hit <Edit…>

Edit the network interface

4. Edit the configuration to resemble the figure below.

Setting up IP configurations

Do this for the other interface as well.

5. Configure the host-only interface with a static IP by editing the file /etc/sysconfig/network-scripts/ifcfg-enp0s8.

In my case, enp0s8 is the interface I chose as host-only interface.

vi /etc/sysconfig/network-scripts/ifcfg-enp0s8

TYPE=Ethernet
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=enp0s8
UUID=3ae28325–9259–4d2d-8d49-e64363403f94
DEVICE=enp0s8
ONBOOT=yes
PEERDNS=yes
PEERROUTES=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes
IPADDR=192.168.56.120
NETMASK=255.255.255.0
vi /etc/sysconfig/network-scripts/ifcfg-enp0s3
TYPE=Ethernet
BOOTPROTO=dhcp
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
NAME=enp0s3
UUID=812dbaa4–8e34–4afa-93d9–9bc396fa51a0
DEVICE=enp0s3
ONBOOT=yes
PEERDNS=yes
PEERROUTES=yes
IPV6_PEERDNS=yes
IPV6_PEERROUTES=yes

Make sure that you have changed ONBOOT line in both interfaces to yes.

6. Add the hostname into the “/etc/hosts” file as follows,

vi /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.56.120 centosMaster

7. Restart your network using the following command,

/etc/init.d/network restart

To check whether the internet connection works, just try to ping to google.com as follows

ping google.com

Repeat this for all 3 nodes and remember the static IPs used for each of the minions and the master.

After setting up the network, you will need to disable the firewall of the VMs by disabling the firewalld service. For this, run the following commands in each of the VMs:

systemctl stop firewalld
systemctl disable firewalld

SSH with the VMs

After creating the 3 VMs, it is essential to check whether the host can access them via ssh. Run following commands (assuming you have an SSH key pair already; if not, follow this guide to create a key pair and you will be able to ssh to your VMs from the host successfully, without having to enter the password each time. (This is helpful since the Kubernetes installation process will connect to each instance via SSH multiple times.)

ssh-copy-id root@192.168.56.112
ssh root@192.168.56.112

Setting up Kubernetes

We are using a Git clone of the Kubernetes repository from master branch (tag 1.7.0-alpha3) on an Ubuntu 16.04 host.

We first need to build this project in order to set up the clusters.

Building Kubernetes

  1. Edit clusters/centos/build.sh
  • Comment out the following lines (as we are relying on locally built Kubernetes client/server artifacts):
# echo "Download kubernetes release v${K8S_VERSION} ..."
# curl -L ${K8S_CLIENT_DOWNLOAD_URL} -o ${RELEASES_DIR}/kubernetes-client-linux-amd64.tar.gz
# curl -L ${K8S_SERVER_DOWNLOAD_URL} -o ${RELEASES_DIR}/kubernetes-server-linux-amd64.tar.gz
  • Comment out the following lines and edit the reference paths for k8s:
# k8s
 # if [[ -f ${RELEASES_DIR}/kubernetes-client-linux-amd64.tar.gz ]] ; then
 #   tar xzf ${RELEASES_DIR}/kubernetes-client-linux-amd64.tar.gz -C ${RELEASES_DIR}
    cp /opt/kubernetes/_output/dockerized/bin/linux/amd64/kubectl ${BINARY_DIR}
 # fi

 # if [[ -f ${RELEASES_DIR}/kubernetes-server-linux-amd64.tar.gz ]] ; then
 #   tar xzf ${RELEASES_DIR}/kubernetes-server-linux-amd64.tar.gz -C ${RELEASES_DIR}
    cp /opt/kubernetes/_output/dockerized/bin/linux/amd64/kube-apiserver \
       /opt/kubernetes/_output/dockerized/bin/linux/amd64/kube-controller-manager \
       /opt/kubernetes/_output/dockerized/bin/linux/amd64/kube-scheduler ${BINARY_DIR}/master/bin
    cp /opt/kubernetes/_output/dockerized/bin/linux/amd64/kubelet \
       /opt/kubernetes/_output/dockerized/bin/linux/amd64/kube-proxy ${BINARY_DIR}/node/bin
#  fi

2. Run

cluster/centos/build.sh all

Following are common invocations associated with the build.sh script

clean — Clean up downloaded releases and unpacked binaries.

download — Download releases to releases directory

unpack — Unpack releases downloaded in releases directory, and copy binaries to binaries directory

all — Download releases and unpack them.

If you are running this command for the first time, you may use all invocation.

Installing Kubernetes

  1. Edit clusters/centos/config-default.sh
  • Add master node’s IP
export MASTER=${MASTER:-"root@192.168.56.120"}
  • Add minion nodes’ IPs
export NODES="${NODES:-"root@192.168.56.112 root@192.168.56.113"}"

2. Run

KUBE_FASTBUILD=true KUBE_RELEASE_RUN_TESTS=n build/release.sh

The KUBE_RELEASE_RUN_TESTS=n parameter avoids running tests.

Since we are using CentOS 7 for the cluster, we do not need to build binaries for other platforms. We achieve this by passing KUBE_FASTBUILD=true parameter. However, if you are in need to build all the binaries, you may use

build/release.sh

If you encounter an “insufficient memory” error, try the following steps:

  1. Remove any prior build residue (dangling Docker images) with,
docker rmi -f `docker images -f “dangling=true” -q`

2. Stop docker service

sudo service docker stop

3. Move /var/lib/docker to a partition with enough space and create a symbolic link from there back to /var/lib/docker.

4. Move kubernetes git source folder to the same partition as well

5. Start docker service

sudo service docker start

6. Re-run the command

build/release.sh

You will see the following tar balls in the _output/release_tars folder after a successful release.

kubernetes-client-darwin-386.tar.gz kubernetes-client-windows-amd64.tar.gz
kubernetes-client-darwin-amd64.tar.gz kubernetes-manifests.tar.gz
kubernetes-client-linux-386.tar.gz kubernetes-node-linux-amd64.tar.gz
kubernetes-client-linux-amd64.tar.gz kubernetes-salt.tar.gz
kubernetes-client-linux-arm64.tar.gz kubernetes-server-linux-amd64.tar.gz
kubernetes-client-linux-arm.tar.gz kubernetes-src.tar.gz
kubernetes-client-linux-ppc64le.tar.gz kubernetes.tar.gz
kubernetes-client-linux-s390x.tar.gz kubernetes-test.tar.gz
kubernetes-client-windows-386.tar.gz

3. Run the following command after a successful release.

KUBERNETES_PROVIDER=centos CERT_GROUP=user ./kube-up.sh

If you encounter an error, cannot create regular file ‘/<<folder path>>/kubernetes/cluster/centos/ca-cert/server.cert’: Permission denied

This means a permission issue. You need to give permission to the indicated folder as follows (note that yourusername and yourusergroup would generally be the same):

sudo chown -R yourusername:yourusergroup /<<folder path>>/kubernetes/cluster/centos/

If you encounter an error, chgrp: invalid group: ‘kube-cert’

This error means the specified CERT_GROUP is not valid. You need to provide a valid user group with root permissions.

4. To check whether the installation is successful, you can go to _output/dockerized/bin/linux/amd64 and run the following command:

./kubectl get nodes

You will see the nodes (minions) listed.

Nodes list

Now you are ready to work with the brand new Kubernetes cluster on your CentOS 7 VMs!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s