Docker 17.06 Swarm Mode: Now with built-in MacVLAN & Node-Local Networks support

Docker 17.06.0-ce-RC5 got announced 5 days back and is available for testing. It brings numerous new features & enablements under this new upcoming release. Few of my favourites includes support for Secrets on Windows,  allows specifying a secret location within the container, adds --format option to docker system df command, adds support for placement preference to docker stack deploy, adds monitored resource type metadata for GCP logging driver and adding build & engine info prometheus metrics to list a few. But one of the notable and most awaited feature include support of swarm-mode services with node-local networks such as macvlan, ipvlan, bridge and host.

Under the new upcoming 17.06 release, Docker provides support for local scope networks in Swarm. This includes any local scope network driver. Some examples of these are bridgehost, and macvlan though any local scope network driver, built-in or plug-in, will work with Swarm. Previously only swarm scope networks like overlay were supported. This is a great news for all Docker Networking enthusiasts.

A Brief Intro to MacVLAN:

Picture1

 

macvlan

In case you’re new , the MACVLAN driver provides direct access between containers and the physical network. It also allows containers to receive routable IP addresses that are on the subnet of the physical network.

MACVLAN offers a number of unique features and capabilities. It has positive performance implications by virtue of having a very simple and lightweight architecture. It’s use cases includes very low latency applications and networking design that requires containers be on the same subnet as and using IPs as the external host network.The macvlan driver uses the concept of a parent interface. This interface can be a physical interface such as eth0, a sub-interface for 802.1q VLAN tagging like eth0.10 (.10representing VLAN 10), or even a bonded host adaptor which bundles two Ethernet interfaces into a single logical interface.

To test-drive MacVLAN under Swarm Mode, I will leverage the existing 3 node Swarm Mode clusters.

Installing Docker 17.06 on all the Nodes:

curl -fsSL https://test.docker.com > install-docker.sh
sh install-docker.sh

 

Verifying the latest Docker version:

Screen Shot 2017-06-26 at 12.51.18 AM

 

Setting up 3 Node Swarm Mode Cluster:

Screen Shot 2017-06-26 at 12.52.21 AM

 

Attention VirtualBox Users: – In case you are using VirtualBox,  the MACVLAN driver requires the network and interfaces to be in promiscuous mode. 

A local network config is created on each host. The config holds host-specific information, such as the subnet allocated for this host’s containers. --ip-range is used to specify a pool of IP addresses that is a subset of IPs from the subnet. This is one method of IPAM to guarantee unique IP allocations.

Manager:

manager1==>sudo docker network create --config-only --subnet 10.140.0.0/24 --gateway 10.140.0.1 -o parent=ens4 --ip-range 10.140.0.10/24 collabnet

Screen Shot 2017-06-26 at 1.19.14 AM

 

Worker-1:

worker1==>sudo docker network create --config-only --subnet 10.140.0.0/24 --gateway 10.140.0.1 -o parent=ens4 --ip-range 10.140.0.30/24 collabnet

Screen Shot 2017-06-26 at 1.22.55 AM

 

Worker-2:

worker2==>sudo docker network create --config-only --subnet 10.140.0.0/24 --gateway 10.140.0.1 -o parent=ens4 --ip-range 10.140.0.30/24 collabnet

Screen Shot 2017-06-26 at 1.27.37 AM

 

Instantiating the macvlan network globally

Manager:

manager1==> $sudo docker network create -d macvlan --scope swarm --config-from collabnet swarm-macvlan

Screen Shot 2017-06-26 at 1.06.42 AM

 

Deploying a service to the swarm-macvlan network:

Let us go ahead and deploy WordPress application. We will be creating 2 services – wordpressapp and wordpressdb1 and attach it to “swarm-macvlan” network as shown below:

Screen Shot 2017-06-26 at 1.35.33 AM

Let us verify if MacVLAN network scope holds this container:

Screen Shot 2017-06-26 at 1.39.17 AM

Next, it’s time to create DB backend i.e. wordpressdb1.

Screen Shot 2017-06-26 at 7.36.31 AM

Verify if both the services are up and running:

Screen Shot 2017-06-26 at 8.33.52 AM

 

Inspecting the “wordpressdb1” service:

Screen Shot 2017-06-26 at 8.40.40 AM

 

Verifying if all the containers on the master node picks up desired IP address from the subnet:

Screen Shot 2017-06-26 at 8.56.46 AM

 

Cool..I am going to leverage this for my Apache JMeter Setup so that I can push loads from different IPs using Docker containers.

Did you find this blog helpful?  Feel free to share your experience. Get in touch @ajeetsraina

If you are looking out for contribution/discussion, join me at Docker Community Slack Channel.

Know more what’s new upcoming under Docker 17.06 CE release by clicking on this link.

  • EXCELLENT Post.thanks for share..more wait.

  • Great post. I am confronting a couple of these difficulties.

  • prakash

    Hello,

    I tried your steps described above for macvlan setup in our lab, my db service is not starting. here is the service ps output. Any pointers to troubleshoot

    [root@docker-vm-1 ~]# docker service ps wordpressdb1
    ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
    6c7hefd2prw9 wordpressdb1.1 mysql:latest docker-vm-2.padc.local Ready Rejected 4 seconds ago “invalid subinterface vlan nam…”
    y2dh61x29xm6 _ wordpressdb1.1 mysql:latest docker-vm-2.padc.local Shutdown Rejected 9 seconds ago “invalid subinterface vlan nam…”
    l64i82v2x5av _ wordpressdb1.1 mysql:latest docker-vm-1.padc.local Shutdown Rejected 14 seconds ago “invalid subinterface vlan nam…”
    dmde1xsqp6na _ wordpressdb1.1 mysql:latest docker-vm-2.padc.local Shutdown Rejected 19 seconds ago “invalid subinterface vlan nam…”
    x95v97kzcxyt _ wordpressdb1.1 mysql:latest docker-vm-1.padc.local Shutdown Rejected 24 seconds ago “invalid subinterface vlan nam…”

    • Ajeet Singh Raina

      Can you let me know which Docker version are you running?

      • prakash

        thank you for responding. let me recheck on the parent interface name

        • Ajeet Singh Raina

          Approved.

          • prakash

            thanks for pointing out the parent interface name. after using the right interface name, service is deployed and each container got an IP. But my understanding of macvlan was, each container will be in isolated network – what I mean by that is – containers shouldn’t communicate with each other. for example, if I ping container 2 IP from container 1, it shouldn’t reach.

            My requirement is – I would like to build each container in an isolated network. we have a lot of customers, we want each customer to be isolated with its own network and container. Any pointers to design this.

          • Ajeet Singh Raina

            Macvlan & host network support in swarm-mode essentially are the easiest way to bring a traditional/legacy application into the container world.essentially macvlan networks will make it easy for the operators to make a container work like a “VM” with its dedicated IP-Address and direct access to the containers from their on-prem data-center networks.

            If you are looking out for the way to put each service in isolation, then all you need is isolated subnet IPs. Think of how you do that in terms of Virtual machines, the same way you do it here. So, if I want to create 2 application stack – WordPress with MYSQL as backend and another – WordPress with postgresql, you will need to put them in separate subnet.

          • prakash

            thanks. I tried below solution too
            https://docs.docker.com/engine/userguide/networking/get-started-macvlan/#dual-stack-ipv4-ipv6-macvlan-bridge-mode

            It works but there is no swarm ( service/scheduler ) way of deploying the containers. This solution has portability issue.

          • Ajeet Singh Raina

            Looking at your comments earlier, you don’t need macvlan but a general Swarm mode overlay networks. Try it out and it should work.

            See the attached diagram https://uploads.disquscdn.com/images/828a78272be820b0411e0981b70a6588c525d65484eb84350d5dda4b9fd4abd9.jpg

          • Ajeet Singh Raina

            So there are two overlay network and one can’t be accessible from another unless you allow them to.

          • Ajeet Singh Raina

            Thanks for the comments.i will go through it when I am in front of the laptop.

  • reportados

    Hi,

    I’m not too experienced with networking, but I want to use a private network interface (eth1) for my service to communicate over. DigitalOcean (as well as other cloud providers) allows for private networking, which enables another net interface, eth1 (eth0 for public). My swarm nodes are deployed in various regions, so I must use the public network interface (eth0) to communicate for the service, which is done via an overlay network, but for the same service in the same region, I wanted to create another overlay network which is for communication in the same region. The problem is that there is no way to specify a network interface/ip to use for the overlay network. I’ve looked into several different network drivers, and it seems like macvlan does what I want.

    I tried `docker network create –config-only –subnet 10.132.0.0/16 –gateway 10.132.0.1 -o parent=eth1 –ip-range 10.132.150.30/24 myregion` for several nodes, and on the manager node, I do
    `docker network create -d macvlan –scope swarm –config-from myregion region-net`

    However, I cannot ping any of my container ips from inside the container, or ping any of the hosts running the containers from inside the container. Also, it appears that all the containers get the same ip address, 10.132.150.0. My manager private ip address is 10.132.27.71. I chose 150 to be part of the ip address for the containers because from my understanding, as long as it’s within the same subnet, it can be discovered and routed. I do not have control over the private network’s gateway, which is apparently 10.132.0.1 according to digitalocean, so it seems like I cannot do any routing.

    Did I misconfigure something? Am I misunderstanding how macvlan works? Most of the examples I’ve seen run the macvlan driver on a single host with multiple containers, but your example seems to run on multiple hosts with a container (potentially more containers as well). Also, does macvlan support the DNS service name lookup/load balancing? Say, I have a couple of replicas for a service named, db. And from inside a container running in the same network, I ping db, will it automatically go through the routing mesh, choosing any of the nodes that is running the db service like the overlay driver?

    Thanks.

  • bb

    I wrote up a small tutorial on running swarm + macvlan with consul + autoscaling https://gist.github.com/killcity/d95e231ea98af4432f200e6d7ceb4961 if anyone is interested.

    • Ajeet Singh Raina

      Cool..

  • Gabriel Sousa

    how can containers ( from different host ) communication which other

    • Ajeet Singh Raina

      Isn’t this blog talking about the same?? Do let me know if you have specific queries?

    • Ajeet Singh Raina

      You will need to set it up on either bare metal system or VirtualBox. Tested it on Virtualbox and it works fine.