Docker 17.06 Swarm Mode: Now with built-in MacVLAN & Node-Local Networks support

Docker 17.06.0-ce-RC5 got announced 5 days back and is available for testing. It brings numerous new features & enablements under this new upcoming release. Few of my favourites includes support for Secrets on Windows,  allows specifying a secret location within the container, adds --format option to docker system df command, adds support for placement preference to docker stack deploy, adds monitored resource type metadata for GCP logging driver and adding build & engine info prometheus metrics to list a few. But one of the notable and most awaited feature include support of swarm-mode services with node-local networks such as macvlan, ipvlan, bridge and host.

Under the new upcoming 17.06 release, Docker provides support for local scope networks in Swarm. This includes any local scope network driver. Some examples of these are bridgehost, and macvlan though any local scope network driver, built-in or plug-in, will work with Swarm. Previously only swarm scope networks like overlay were supported. This is a great news for all Docker Networking enthusiasts.

A Brief Intro to MacVLAN:

Picture1

 

macvlan

In case you’re new , the MACVLAN driver provides direct access between containers and the physical network. It also allows containers to receive routable IP addresses that are on the subnet of the physical network.

MACVLAN offers a number of unique features and capabilities. It has positive performance implications by virtue of having a very simple and lightweight architecture. It’s use cases includes very low latency applications and networking design that requires containers be on the same subnet as and using IPs as the external host network.The macvlan driver uses the concept of a parent interface. This interface can be a physical interface such as eth0, a sub-interface for 802.1q VLAN tagging like eth0.10 (.10representing VLAN 10), or even a bonded host adaptor which bundles two Ethernet interfaces into a single logical interface.

To test-drive MacVLAN under Swarm Mode, I will leverage the existing 3 node Swarm Mode clusters on my VMware ESXi system. I have tested it on bare metal system and VirtualBox and it works equally great.  

[Updated: 9/27/2017 – I have added docker-stack.yml at the end of this guide to show you how to build services out of docker-compose.yml file. DO NOT FORGET TO CHECK IT OUT]

Installing Docker 17.06 on all the Nodes:

[simterm]

$curl -fsSL https://test.docker.com > install-docker.sh

$sh install-docker.sh

[/simterm]

 

Verifying the latest Docker version:

Screen Shot 2017-06-26 at 12.51.18 AM

 

Setting up 2 Node Swarm Mode Cluster:

 

 

Attention VirtualBox Users: – In case you are using VirtualBox,  the MACVLAN driver requires the network and interfaces to be in promiscuous mode. 

A local network config is created on each host. The config holds host-specific information, such as the subnet allocated for this host’s containers. --ip-range is used to specify a pool of IP addresses that is a subset of IPs from the subnet. This is one method of IPAM to guarantee unique IP allocations.

Manager:

[simterm]

manager1==>sudo docker network create --config-only --subnet 100.98.26.0/24 -o parent=ens160.60 --ip-range 100.98.26.100/24 collabnet

[/simterm]

 

Worker-1:

[simterm]

worker1==>sudo docker network create --config-only --subnet 100.98.26.0/24 -o parent=ens160.60 --ip-range 100.98.26.100/24 collabnet

[/simterm]

 

 

Instantiating the macvlan network globally

Manager:

[simterm]

manager1==> $sudo docker network create -d macvlan --scope swarm --config-from collabnet swarm-macvlan

[/simterm]

 

Deploying a service to the swarm-macvlan network:

Let us go ahead and deploy WordPress application. We will be creating 2 services – wordpressapp and wordpressdb1 and attach it to “swarm-macvlan” network as shown below:

Creating Backend Service:

[simterm]

$docker service create –replicas 1 –name wordpressdb1 –network swarm-macvlan –env MYSQL_ROOT_PASSWORD=collab123 –env MYSQL_DATABASE=wordpress mysql

[/simterm]

Let us verify if MacVLAN network scope holds this container:

 

Creating Frontend Service

Next, it’s time to create wordpress application i.e. wordpressapp

[simterm]

#docker service create –env WORDPRESS_DB_HOST=wordpressdb1 –env WORDPRESS_DB_PASSWORD=collab123 –network swarm-macvlan –replicas 4 –name wordpressapp –publish 80:80/tcp wordpress:latest

[/simterm]

Verify if both the services are up and running:

 

Verifying if all the containers on the master node picks up desired IP address from the subnet:

 

Docker Compose File showcasing MacVLAN Configuration

https://gist.github.com/ajeetraina/71ecd6f81021cd6c178d3d8462511096

Ensure that you run the below commands to setup MacVLAN configuration for your services before you execute the above docker stack deploy CLI:

[simterm]

root@ubuntu-1610:~# docker network create –config-only –subnet 100.98.26.0/24 –gateway 100.98.26.1 -o parent=ens160.60 –ip-range 100.98.26.120/24 collabnet
da2912d762cbf5f5ea412e6e4d69352a3285f720e23740529af9e533c7168729

 

root@ubuntu-1610:~#docker network create -d macvlan –scope swarm –config-from collabnet swarm-macvlan
jp76lts6hbbheqlbbhggumujd

[/simterm]

 

Verify that the containers inspection shows the correct information:

[simterm]

root@ubuntu-1610:~/docker101/play-with-docker/wordpress/example1# docker network inspect swarm-macvlan
[
{
“Name”: “swarm-macvlan”,
“Id”: “jp76lts6hbbheqlbbhggumujd”,
“Created”: “2017-09-27T02:12:00.827562388-04:00”,
“Scope”: “swarm”,
“Driver”: “macvlan”,
“EnableIPv6”: false,
“IPAM”: {
“Driver”: “default”,
“Options”: null,
“Config”: [
{
“Subnet”: “100.98.26.0/24”,
“IPRange”: “100.98.26.120/24”,
“Gateway”: “100.98.26.1”
}
]
},
“Internal”: false,
“Attachable”: false,
“Ingress”: false,
“ConfigFrom”: {
“Network”: “collabnet”
},
“ConfigOnly”: false,
“Containers”: {
“3c3f1ec48225ef18e8879f3ebea37c2d0c1b139df131b87adf05dc4d0f4d8e3f”: {
“Name”: “myapp2_wordpress.1.nd2m62alxmpo2lyn079x0w9yv”,
“EndpointID”: “a15e96456870590588b3a2764da02b7f69a4e63c061dda2798abb7edfc5e5060”,
“MacAddress”: “02:42:64:62:1a:02”,
“IPv4Address”: “100.98.26.2/24”,
“IPv6Address”: “”
},
“d47d9ebc94b1aa378e73bb58b32707643eb7f1fff836ab0d290c8b4f024cee73”: {
“Name”: “myapp2_db.1.cxz3y1cg1m6urdwo1ixc4zin7”,
“EndpointID”: “201163c233fe385aa9bd8b84c9d6a263b18e42893176271c585df4772b0a2f8b”,
“MacAddress”: “02:42:64:62:1a:03”,
“IPv4Address”: “100.98.26.3/24”,
“IPv6Address”: “”
}
},
“Options”: {
“parent”: “ens160”
},
“Labels”: {},
“Peers”: [
{
“Name”: “ubuntu-1610-1633ea48e392”,
“IP”: “100.98.26.60”
}
]
}
]

[/simterm]

Docker Stack Deploy CLI:

[simterm]

$docker stack deploy -c docker-stack.yml myapp2
Ignoring unsupported options: restart

Creating service myapp2_db
Creating service myapp2_wordpress

[/simterm]

Verifying if the services are up and running:

[simterm]

root@ubuntu-1610:~/# docker stack ls
NAME SERVICES
myapp2 2

root@ubuntu-1610:~/# docker stack ps myapp2

ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
nd2m62alxmpo myapp2_wordpress.1 wordpress:latest ubuntu-1610 Running Running 15 minutes ago
cxz3y1cg1m6u myapp2_db.1 mysql:5.7 ubuntu-1610 Running Running 15 minutes ago

[/simterm]

Looking for Docker Compose file for Single Node?

https://gist.github.com/ajeetraina/defff61db96b88144509d728196d7853

 

Cool..I am going to leverage this for my Apache JMeter Setup so that I can push loads from different IPs using Docker containers.

Did you find this blog helpful?  Feel free to share your experience. Get in touch @ajeetsraina

If you are looking out for contribution/discussion, join me at Docker Community Slack Channel.

Know more what’s new upcoming under Docker 17.06 CE release by clicking on this link.

Topology Aware Scheduling under Docker v17.05.0 Swarm Mode Cluster

 

Docker 17.05.0 Final release went public exactly 2 week back.This community release was the first release as part of new Moby project.  With this release, numerous interesting features  like Multi-Stage Build support to the builder, using build-time ARG in FROM, DEB packaging for Ubuntu 17.04(Zesty Zapus)  has been included. With this latest release, Docker Team brought numerous new features and improvements in terms of Swarm Mode. Example – synchronous service commands, automatic service rollback on failure, improvement over raft transport package, service logs formatting etc. 

swarm_1705

 

Placement Preference under Swarm Mode:

One of the prominent new feature introduced is placement preference  under 17.05.0-CE Swarm Mode . Placement preference feature allows you to divide tasks evenly over different categories of nodes. It allows you to balance tasks between multiple datacenters or availability zones. One can use a placement preference to spread out tasks to multiple datacenters and make the service more resilient in the face of a localized outage. You can use additional placement preferences to further divide tasks over groups of nodes.  Under this blog, we will setup 5-node Swarm Mode cluster on play-with-docker platform and see how to balance them over multiple racks within each datacenter. (Note – This is not real time scenario but based on assumption that nodes are being placed in 3 different racks).

Assumption:  There are 3 datacenter Racks holding respective nodes as shown:

{Rack-1=> Node1, Node2 and Node3},

{Rack-2=> Node4}  &

{Rack-3=> Node5}

 

Creating Swarm Master Node:

Open up Docker Playground to build up Swarm Cluster.

[simterm]

$ docker swarm init –advertise-addr 10.0.116.3

[/simterm]

Screen Shot 2017-05-20 at 7.04.26 AM

 

Adding Worker Nodes to Swarm Cluster

[simterm]

$ docker swarm join –token <token-id> 10.0.116.3:2377

[/simterm]

Screen Shot 2017-05-20 at 7.07.53 AM

Create 3 more instances and add those nodes as worker nodes. This should build up 5 node Swarm Mode cluster.

Screen Shot 2017-05-20 at 7.08.51 AM

 

Setting up Visualizer Tool 

To showcase this demo, I will leverage a fancy popular Visualizer tool.

 [simterm]

$git clone https://github.com/ajeetraina/docker101

$cd docker101/play-with-docker/visualizer

[/simterm]

 

All you need is to execute docker-compose command to bring up visualizer container:

 

[simterm]

$docker-compose up -d

[/simterm]

 

Screen Shot 2017-05-20 at 7.09.57 AM

 

Click on port “8080” which gets displayed on top centre of this page and it should display a fancy visualiser depicting Swarm Mode cluster nodes.

Screen Shot 2017-05-20 at 7.13.50 AM

Creating an Overlay Network:

[simterm]

 $docker network create -d overlay collabnet

[/simterm]

Screen Shot 2017-05-20 at 7.15.15 AM

 

Let us try to create service with no preference placement or no node labels.

Setting up WordPress DB service:

[simterm]

$docker service create –replicas 10 –name wordpressdb1 –network collabnet –env MYSQL_ROOT_PASSWORD=collab123 –env MYSQL_DATABASE=wordpress mysql:latest

[/simterm]

Screen Shot 2017-05-20 at 7.19.02 AM

When you run the above command, the swarm will spread the containers evenly node-by-node. Hence, you will see 2-containers per node as shown below:

 Screen Shot 2017-05-20 at 7.22.58 AM 

Setting up WordPress Web Application:

[simterm]

$docker service create –env WORDPRESS_DB_HOST=wordpressdb1 –env WORDPRESS_DB_PASSWORD=collab123 –network collabnet –replicas 3 –name wordpressapp –publish 80:80/tcp wordpress:latest

[/simterm]

Screen Shot 2017-05-20 at 7.30.55 AM

Visualizer:

Screen Shot 2017-05-20 at 7.34.51 AM 

As per the visualizer, you might end up with uneven distribution of services. Example., Rack-1 holding node-1, node-2 and node-3 looks to have almost equal distribution of services, Rack-2 which holds node3 lack WordPress fronted application.

Here Comes Placement Preference for a rescue…

Under the latest release, Docker team has introduced a new feature called “Placement Preference Scheduling”. Let us spend some time to understand what it actually means. You can set up the service to divide tasks evenly over different categories of nodes. One example of where this can be useful is to balance tasks over a set of datacenters or availability zones. 

This uses --placement-pref with a spread strategy (currently the only supported strategy) to spread tasks evenly over the values of the datacenter node label. In this example, we assume that every node has a datacenter node label attached to it. If there are three different values of this label among nodes in the swarm, one third of the tasks will be placed on the nodes associated with each value. This is true even if there are more nodes with one value than another. For example, consider the following set of nodes:

  • Three nodes with node.labels.datacenter=india
  • One node with node.labels.datacenter=uk
  • One node with node.labels.datacenter=us

Considering the last example, since we are spreading over the values of the datacenter label and the service has 5 replicas, at least 1 replica should be available  in each datacenter. There are three nodes associated with the value “india”, so each one will get one of the three replicas reserved for this value. There is 1 node with the value “uk”, and hence 1 replica for this value will be receiving it. Finally, “us” has a single node that will again get atleast 1 replica of each service reserved.

To understand more clearly, let us assign node labels to Rack nodes as shown below:

Rack-1 : 

Node-1

[simterm]

$ docker node update –label-add datacenter=india node1

[/simterm]

[simterm]
$ docker node update –label-add datacenter=india node2

[/simterm]

[simterm]
$ docker node update –label-add datacenter=india node3

[/simterm]

Rack-2

[simterm]
$ docker node update –label-add datacenter=uk node4

[/simterm]

Rack-3

[simterm]
$ docker node update –label-add datacenter=us node5

[/simterm]

Screen Shot 2017-05-20 at 7.46.33 AM

 

Removing both the services:

[simterm]

$ docker service rm wordpressdb1 wordpressapp

[/simterm]

Let us now pass placement preference parameter to the docker service command:

[simterm]

$docker service create –replicas 10 –name wordpressdb1 –network collabnet –placement-pref “spread=node.labels.datacenter” –env MYSQL_ROOT_PASSWORD=collab123 –env MYSQL_DATABASE=wordpress mysql:latest

[/simterm]

Screen Shot 2017-05-20 at 8.05.52 AM

 

Visualizer:

Screen Shot 2017-05-20 at 8.05.14 AM

Rack-1(node1+node2+node3) has 4 copies, Rack-2(node4) has 3 copies and Rack-3(node5) has 3 copies.

Let us run WordPress Application service likewise:

[simterm]

$docker service create –env WORDPRESS_DB_HOST=wordpressdb1 –env WORDPRESS_DB_PASSWORD=collab123 –placement-pref “spread=node.labels.datacenter” –network collabnet –replicas 3 –name wordpressapp –publish 80:80/tcp wordpress:latest

[/simterm]

Screen Shot 2017-05-20 at 8.09.29 AM

Visualizer: As shown below, we have used placement preference feature to ensure that the service containers get distributed across the swarm cluster on both the racks.

Screen Shot 2017-05-20 at 8.10.41 AM

 

As shown above, –placement-pref ensures that the task is spread evenly over the values of the datacenter node label. Currently spread strategy is only supported.Both engine labels and node labels are supported by placement preferences.

Please Note: If you want to try this feature with Docker compose , you will need Compose v3.3 which is slated to arrive under 17.06 release.

Did you find this blog helpful?  Feel free to share your experience. Get in touch @ajeetsraina

If you are looking out for contribution/discussion, join me at Docker Community Slack Channel.

Know more about the latest Docker releases clicking on this link.

 

Docker Service Inspection Filtering & Template Engine under Swarm Mode

Go programming language has really helped in shaping Docker as a powerful software and enabling fast development for distributed systems. It has been helping developers and operations team to quickly construct programs and tools for Cloud computing environment. Go offers built-in support for JSON  (JavaScript Object Notation) encoding and decoding, including to and from built-in and custom data types.

golang

 

In last 1 year, Docker Swarm Mode has matured enough to become production-ready. The Orchestration platform is quite stable with numerous features like Logging, Secrets Management, Security Scanning, improvement over Scheduling, networking etc. making it more simple to use and scale-out in just few liner commands. With the introduction of new APIs like swarm, node, volume plugins, services etc., Swarm Mode brings dozens of features to control every aspect of swarm cluster sitting on the master node. But when you start building services in the range of 100s & 1000s and that too distributed across another 100s and 1000s of nodes, there arise a need of quick and handy way of filtering the output, especially when you are interested to capture one specific data out of the whole cluster-wide configuration. Here comes ‘a filtering flag’ as a rescue.

swarm11

The filtering flag (-f or --filter) format is a key=value pair which is actually a very powerful weapon for developers & system administrators.If you have ever played around with Docker CLI, you must have used docker inspect command to get metadata on a container/ image. Using it with -f provides you more specific information like IP address, network etc. There are numerous guide on how to use filters with standalone host holding the Docker images but I found lack of guides talking about Swarm Mode filters.

Under this blog post, I have prepared a quick list of consolidated filtering commands and outputs in tabular format for Swarm Mode Cluster(shown below).

I have 3 node Swarm Mode cluster running on one of my Google Cloud Engine. Let us first create a simple wordpress application as shown below:

 

swarm-1

 

We will create a simple wordpress service as shown:

master==>docker service create –env WORDPRESS_DB_HOST=wordpressdb1 –env WORDPRESS_DB_PASSWORD=collab123 –network collabnet –replicas 4 –name wordpressapp –publish 80:80/tcp wordpress:latest

Below is the tabular list of commands and outputs which talks about various filtering mode for docker service inspect command:

 

Inspecting the “wordpress” service:

[wpsm_comparison_table id="3" class=""]

To retrieve the network information for specific service:

[wpsm_comparison_table id="6" class=""]

To list out the port which wordpress is using for specific service:

[wpsm_comparison_table id="7" class=""]

To list out the protocol detail for specific service:

[wpsm_comparison_table id="8" class=""]

To list out the type of Published port:

[wpsm_comparison_table id="9" class=""]

To verify if its DNS RR or Virtual IP(VIP) based for specific service:

[wpsm_comparison_table id="10" class=""]

 

To list out the type of Published port for specific service:

[wpsm_comparison_table id="11" class=""]

To list out the replication factor for a specific service:

[wpsm_comparison_table id="12" class=""]

To list out the environmental variable passed for specific service:

[wpsm_comparison_table id="13" class=""]

To list out the image detail for specific service:

[wpsm_comparison_table id="14" class=""]

To list out the complete networking details for specific service:

[wpsm_comparison_table id="15" class=""]

To list out detailed consolidated  meta data information for the specific service::

[wpsm_comparison_table id="4" class=""]
 
To list out further detailed information of the service :

[wpsm_comparison_table id="5" class=""]

Let us create another service called “wordpressdb1” which is actually a database service under Docker Swarm Mode:

$docker service create –replicas 1 –name wordpressdb1 –network collabnet –env MYSQL_ROOT_PASSWORD=collab123 –env MYSQL_DATABASE=wordpress mysql:latest

Inspecting the service “collabdb1”:

[wpsm_comparison_table id="16" class=""]

Displaying the output in human-friendly way:

[wpsm_comparison_table id="17" class=""]

To list out the VIPs details for specific service:

[wpsm_comparison_table id="18" class=""]

Retrieving the last StatusUpdate details:

[wpsm_comparison_table id="19" class=""]

I have plans to add more examples during the course of time based on experience and exploration. I hope you will find it handy.