Getting Started with Multi-Node Kubernetes Cluster using LinuxKit

Here’s a BIG news for the entire Container Community – “Kubernetes Support is coming to Docker Platform“. What does this mean? This means that developers and operators can now build apps with Docker and seamlessly test and deploy them using both Docker Swarm and Kubernetes. This announcement was made on the first day of Dockercon EU 2017 Copenhagen where Solomon Hykes, Founder of Docker invited Kubernetes Co-Founder  Tim Hockin on the stage for the first time. Tim welcomed the Docker family to the vibrant Kubernetes world. 

In case you missed out the news, here are the important takeaways from Docker-Kubernetes announcement –

  • Kubernetes support will be added to Docker Enterprise Edition for the first time.
  • Kubernetes support will be added optionally to Docker Community Edition for Mac and Windows.
  • Kubernetes and Docker Swarm both orchestrators will be available under Docker Platform.
  • Kubernetes integration will be made available under Moby projects too.

Docker Inc firmly believes that bringing Kubernetes to Docker will simplify and advance the management of Kubernetes for developers, enterprise IT and hackers to deliver the advanced capabilities of Docker to a broader set of applications.

 

 

In case you want to be first to test drive, click here to sign up and be notified when the beta is ready.

Please be aware that the beta program is expected to be ready at the end of 2017.

Can’t wait for beta program? Here’s a quick guide on how to get Multi-Node Kubernetes cluster up and running using LinuxKit.

Under this blog post, we will see how to build 5-Node Kubernetes Cluster using LinuxKit. Here’s the bonus – We will try to deploy WordPress application on top of Kubernetes Cluster. I will be building it on my macOS Sierra 10.12.6 system running the latest Docker for Mac 17.09.0 Community Edition up and running.

Pre-requisite:

  1. Ensure that the latest Docker release is up and running.

       2. Clone the LinuxKit repository:

 [simterm]

$sudo git clone https://github.com/ajeetraina/linuxkit

$cd projects/kubernetes/

[/simterm]

3. Execute the below script to build Kubernetes OS images using LinuxKit & Moby:

[simterm]

$sudo chmod +x script4k8s.sh

[/simterm]

All I have done is put all the manual steps under a script as shown below:

 

https://gist.github.com/ajeetraina/9e083e26273acea39ec170c439b0946e

 

This should build up Kubernetes OS image.

Run the below command to know the IP address of the kubelet container:

[simterm]

$ip adds show dev eth0

[/simterm]

 

You can check the list of tasks running as shown below:

Once this kubelet is ready, it’s time to login into it directly from a new terminal:

[simterm]

$sudo ./ssh_into_kubelet.sh 192.168.65.3

[/simterm]

So you have already SSH into the kubelet container rightly named “LinuxKit Kubernetes Project”.

We are going to mark it as “Master Node”.

Initializing the Kubernetes Master Node

Now it’s time to manually initialize master node with Kubeadm command as shown below:

[simterm]

$sudo kubeadm-init.sh

[/simterm]

Wait for few minutes till kubeadm command completes. Once done, you can use kubeadm join argument as shown below:

 

Your Kubernetes master node gets initialized successfully as shown below:

 

 

It’s time to run the below list of commands command to join the list of worker nodes:

[simterm]

$pwd

/Users/ajeetraina/linuxkit/projects/kubernetes

$sudo ./boot.sh 1 –token 4cec03.1a7ccb44115f427a 192.168.65.3:6443

$sudo ./boot.sh 2 –token 4cec03.1a7ccb44115f427a 192.168.65.3:6443

$sudo ./boot.sh 3 –token 4cec03.1a7ccb44115f427a 192.168.65.3:6443

$sudo ./boot.sh 4 –token 4cec03.1a7ccb44115f427a 192.168.65.3:6443

[/simterm]

 

It brings up 5-Node Kubernetes cluster as shown below:

 

Deploying WordPress Application on Kubernetes Cluster:

Let us try to deploy a WordPress application on top of this Kubernetes Cluster.

Head over to the below directory:

[simterm]

$cd linuxkit/projects/kubernetes/wordpress

[/simterm]

 

Creating a Persistent Volume

[simterm]

$kubectl create -f local-volumes.yaml

[/simterm]

The content of local-volumes.yml look like:


This creates 2 persistent volumes:

Verifying the volumes:

 

Creating a Secret for MySQL Password

[simterm]

kubectl create secret generic mysql-pass --from-literal=password=mysql123

[/simterm]

[simterm]

 

Verifying the secrets using the below command:

$kubectl get secrets

[/simterm]

 

Deploying MySQL:

[simterm]

$kubectl create -f mysql-deployment.yaml

[/simterm]

 

Verify that the Pod is running by running the following command:

[simterm]

$kubectl get pods

[/simterm]
 

Deploying WordPress:

[simterm]

$kubectl create -f wordpress-deployment.yaml

[/simterm]

By now, you should be able to access WordPress UI using the web browser.

Did you find this blog helpful?  Feel free to share your experience. Get in touch @ajeetsraina

If you are looking out for contribution/discussion, join me at Docker Community Slack Channel.

 

Building a secure Docker Host VM on VMware ESXi using LinuxKit & Moby

Post Dockercon 2017 @ Austin TX,  I raised a feature request titled “LinuxKit command to push vmware.vmdk to remote ESXi datastore”. Within few weeks time, the feature was introduced by LinuxKit team. A Special thanks goes to Daniel Finneran who worked hard to get this feature merged into the LinuxKit main branch.

 

LinuxKit project is 5 month old now. It has already bagged 3100+ stars, added up 69 contributors and 350+ forks till date. If you are pretty new, LinuxKit is not a full host operating system, as it primarily has two jobs: run containerd containers, and be secure. It uses modern kernels, and updates frequently following new releases. As such, the system does not contain extraneous packages or drivers by default. Because LinuxKit is customizable, it is up to individual operators to include any additional bits they may require.

LinuxKit is undoubtedly Secure

The core system components included in LinuxKit userspace are key to security, and written in type safe languages, such as RustGo and OCaml, and run with maximum privilege separation and isolation. The project is currently leveraging MirageOS to construct unikernels to achieve this, and that progress can be tracked here: as of this writing, dhcp is the first such type safe program. There is ongoing work to remove more C components, and to improve, fuzz test and isolate the base daemons. Further rationale about the decision to rewrite system daemons in MirageOS is explained at length in this document. I am planning to come up with blog post to brief on “LinuxKit Security” aspect. Keep an eye on this space in future..

Let’s talk about building a secure Docker Host VM…

I am a great fan of VMware PowerCLI. I have been using it since the time I was working full time in VMware Inc.(during 2010-2011 timeframe). Today the most quickest way to get VMware PowerCLI up and running is by using PhotonOS based Docker Image. Just one Docker CLI and you are already inside Photon OS running PowerShell & PowerCLI to connect to remote ESXi to build up VMware Infrastructure. Still this mightn’t give you a secure Docker host environment. If you are really interested to build a secure, portable and lean Docker Host operating system, LinuxKit is the right tool. But how?

Under this blog post, I am going to show how Moby & LinuxKit can help you in building a secure Docker 17.07 Host VM on top of VMware ESXi.

Pre-requisites:

  • VMware vSphere ESXi 6.x
  • Linux or MacOS with Go packages installed
  • Docker 17.06/17.07 installed on the system

The below commands has been executed on one of my local Ubuntu 16.04 LTS system which can reach out to ESXi system flawlessly.

Cloning the LinuxKit Repository:

[simterm]

$git clone https://github.com/linuxkit/linuxkit

[/simterm]

Building Moby & LinuxKit:

[simterm]

$cd linuxkit

$make

[/simterm]

Configuring the right PATH for Moby & LinuxKit

[simterm]

$cp bin/moby /usr/local/bin

$cd bin/linuxkit /usr/local/bin

[/simterm]

A Peep into vmware.yml File

https://gist.github.com/ajeetraina/38e6c668674026a33971478c7649bd92

The first 3 lines shows modern, securely configured kernel. The init section spins up containerd to run services. The onboot section allows dhcpd for networking. The services includes getty service container for shell, runs nginx service container. The trust section indicates all images signed and verified.

Building VMware ISO Image using Moby

[simterm]

$moby build -output iso-bios -name vmware vmware.yml

[/simterm]

 

Pushing VMware ISO Image to remote ESXi datastore

[simterm]

$linuxkit push vcenter -datastore=datastore1 -hostname=myesxi.dell.com -url https://root:xxx@100.98.x.x/sdk -folder=linuxkit vmware.iso

[/simterm]

Usage of linuxkit push vcenter :-

 

Running a secure VM directly from the ESXi datastore using LinuxKit

[simterm]

dell@redfish-ubuntu:~/linuxkit/examples$ sudo linuxkit run vcenter -cpus 8 -datastore datastore1 -mem 2048 -network ‘VM Network’ -hostname myesxi.dell.com -powerOn -url  https://root:xxx@100.98.x.x/sdk vmware.iso

Creating new LinuxKit Virtual Machine

Adding ISO to the Virtual Machine

Adding VM Networking

Powering on LinuxKit VM

[/simterm]

Now let us verify that VM is up and running using either VMware vSphere Client or SDK URL.

You will find that VM is already booted up with the latest Docker 17.07 platform up and running.

Building a Docker Host VM using Moby & LinuxKit

In case you want to build a Docker Host VM, you can refer to the below vmware.yml file:

https://gist.github.com/ajeetraina/a9ff0f41208b3ff66f5afb62de25954c

Just re-run the below command to get the new VM image:

[simterm]

$moby build -output iso-bios -name vmware docker-vmware.yml

[/simterm]

Follow the above steps to push it to remote datastore and run it using LinuxKit. Hence, you have a secured Docker 17.07 Host ready to build Docker Images and build up application stack.

How about building Photon OS based Docker Image using Moby & LinuxKit? Once you build it and push it to VM , its all ready to build Virtual Infrastructure. Interesting, Isn’t it?

Did you find this blog helpful?  Feel free to share your experience. Get in touch @ajeetsraina

If you are looking out for contribution/discussion, join me at Docker Community Slack Channel.

 

 

Building a Secure VM based on LinuxKit on Microsoft Azure Platform

LinuxKit GITHUB repository recently crossed 3000 stars, forked around 300+ times and added 60+ contributors. Just 5 months old project and it has already gained lot of momentum across the Docker community. Built with a purpose that enables community to create secure, immutable, and minimal Linux distributions, LinuxKit is matured enough to support number of Cloud Platforms like Azure, AWS, Google Cloud Platform, VMware, Packets.net and many more..

 

In my recent blogs, I showcased how to get LinuxKit OS built for Google Cloud Platform, Amazon Web Services and VirtualBox. ICYMI, I recently published few of the the video on LinuxKit too. Check it out.

 

Under this blog post, I will walkthrough how to build secure and portal VM based on LinuxKit image on Microsoft Azure Platform.

Pre-requisite:

I will be leveraging macOS Sierra running Docker 17.06.1-ce-rc1-mac20 version. I tested it on Ubuntu 16.04 LTS edition too running on one of Azure VM and it went fine. Prior knowledge of Microsoft Azure / Azure CLI 2.0 will be required to configure Service Principle for VHD image to get uploaded to Azure smoothly.

 

Step-1: Pulling the latest LinuxKit repository

Pull the LinuxKit repository using the below command:

[simterm]

$git clone https://github.com/linuxkit/linuxkit

[/simterm]

 

Step-2: Build Moby & LinuxKit tool

[simterm]

$cd linuxkit

$make

[/simterm]

 

Step-3: Copying the tools into the right PATH

[simterm]

$cp -rf bin/moby /usr/local/bin/

$cp -rf bin/linuxkit /usr/local/bin/

[/simterm]

 

Step-4: Preparing Azure CLI tool

[simterm]

$curl -L https://aka.ms/InstallAzureCli | bash

[/simterm]

 

Step-5: Run the below command to restart your shell

[simterm]

$exec -l $SHELL

[/simterm]

 

Step-6: Building LinuxKit OS for Azure Platform

[simterm]

$cd linuxkit/examples/

$moby build -output vhd azure.yml

[/simterm]

This will build up VHD image which now has to be pushed to Azure Platform.

In order to push the VHD image to Azure, you need to authenticate LinuxKit with your Azure subscription, hence you  will need to set up the following environment variables:

[simterm]

   export AZURE_SUBSCRIPTION_ID=43b263f8-XXXX–XXXX–XXXX–XXXXXXXX

   export AZURE_TENANT_ID=633df679-XXXX–XXXX–XXXX–XXXXXXXX

   export AZURE_CLIENT_ID=c7e4631a-XXXX–XXXX–XXXX–XXXXXXXX

   export AZURE_CLIENT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXX=

[/simterm]

Alternatively, the easy way to get all the above details is through the below command:

[simterm]

$az login

To sign in, use a web browser to open the page https://aka.ms/devicelogin and enter the code XXXXXX to authenticate.

[/simterm]

The above command lists out Subscription ID and tenant ID which can be exported therein.

Next, follow this link to create an Azure Active Directory application and service principal that can access resources. If you want to stick to CLI rather than UI, you can follow the below steps:

Step-7: Pushing the VHD image to Azure Platform

[simterm]

$linuxkit run azure –resourceGroupName mylinuxkit –accountName mylinuxkitstore -location eastasia azure.vhd
Creating resource group in eastasia
Creating storage account in eastasia, resource group mylinuxkit

[/simterm]

The command will end up with the below message:

 

 Completed: 100% [     68.00 MB] RemainingTime: 00h:00m:00s Throughput: 0 Mb/sec    

Creating virtual network in resource group mylinuxkitresource, in eastasia

Creating subnet linuxkitsubnet468 in resource group mylinuxkitresource,

within virtual network linuxkitvirtualnetwork702

Creating public IP Address in resource group mylinuxkitresource, with name publicip159

Started deployment of virtual machine linuxkitvm941 in resource group mylinuxkitresource

Creating virtual machine in resource group mylinuxkitresource, with name linuxkitvm941, in location eastasia

NOTE: Since you created a minimal VM without the Azure Linux Agent,

the portal will notify you that the deployment failed. After around 50 seconds try connecting to the VM

ssh -i path-to-key root@publicip159.eastasia.cloudapp.azure.com

 

By this time, you should be able to see LinuxKit VM coming up under Azure Platform as shown below:

Wait for next 2-3 minutes till you try SSHing to this Azure instance and its all set to be up an running smoothly.

Known Issue:

  • Since the image currently does not contain the Azure Linux Agent, the Azure Portal will report the creation as failed.
  • The main workaround is the way the VHD is uploaded, specifically by using a Docker container based on Azure VHD Utils. This is mainly because the tool manages fast and efficient uploads, leveraging parallelism
  • There is work in progress to specify what ports to open on the VM (more specifically on a network security group)
  • The metadata package does not yet support the Azure metadata.

 

Did you find this blog helpful?  Feel free to share your experience. Get in touch @ajeetsraina

If you are looking out for contribution/discussion, join me at Docker Community Slack Channel.

Further Reference: