Ajeet Raina Docker Captain, ARM Innovator & Docker Bangalore Community Leader.

Docker 1.12 Swarm Mode – Under the hood

3 min read

Today Docker Inc. released Engine 1.12 Release Candidate 4 with numerous improvements and added security features. With an optional “Swarm Mode” feature rightly integrated into core Docker Engine, a native management of a cluster of Docker Engines, orchestration, decentralized design, service and application deployment, scaling, desired state reconciliation, multi-host networking, service discovery and routing mesh implementation is just a matter of few liner commands.

In the previous posts, we introduced Swarm Mode, implemented a simple service applications and went through 1.12 networking model. Under this post, we will deep dive into Swarm Mode and study what kind of communication gets generated between master and worker nodes in the Swarm cluster.

Setting up Swarm Master Node

Let’s start setting up Swarm Mode cluster and see how underlying communication takes place. I will be using docker-machine to setup master and worker nodes on my Google Cloud Engine.

$docker-machine create -d google –google-project <project-id> –engine-url https://test.docker.com test-master1

If you have less time setting up Swarm Cluster, do refer https://github.com/ajeetraina/google-cloud-swarm. I have forked it from here.

As you see below, Docker Hosts machines gets created through docker-machine with all the nodes running Docker Engine 1.12-rc4.

Let’s initialize the swarm mode on the first master node as shown below:

I have used one liner docker-machine command to keep it clean and simple. The docker-machine command will SSH to the master node and initialize the swarm mode.

The newly released RC4 version holds improvement in terms of security which is enabled by default. In earlier release, one has to pass –secret parameter to secure and control which worker node can join and which can’t. But going forward, the swarm mode automatically generates random secret key. This is just awesome !!!

[Under the hood] – Whenever we do “docker swarm init”, a TLS root CA (Certificate Authority) gets created as shown below.

Then a key-pair is issued for the first node and signed by root CA.

Let’s add the first worker node as shown below:

Looking at inotify output:

When further nodes joins the swarm, they are issues their own keypair, signed by the root CA, and they also receive the root CA public key and certificate. All the communication is encrypted over TLS.

The node keys and certificates are automatically renewed on regular intervals (by default 90 days) but one can tune it with docker swarm update command.

Let us spend some time understanding the master and worker architecture in detail.

 

Every node in Swarm Mode has a role which can be categorized as  Manager and Worker. Manager node has responsibility to actually orchestrate the cluster, perform the health-check, running containers serving the API and so on. The worker node just execute the tasks which are actually containers. It can-not decide to schedule the containers on the different machine. It can-not change the desired state. The workers only takes work and report back the status. You can enable node promotion or demotion easily through one-liner command.

Managers and Workers uses two different communication models. Managers have built-in RAFT system that allows them to share information for new leader election. At one time, only manager is actually performing the scaling and they use a leader follower model to figure out which one is supposed to be what. No External K-V store is required as built-in internal distributed state store is available.

Workers, on the other side, uses GOSSIP network protocol which is quite fast and consistent. Whenever any new container/tasks gets generated in the cluster, the gossip is going to broadcast it to all the other containers in a specific overlay network that this new container has started. Please remember that ONLY the containers which are running in the specific overlay network will be communicated and NOT globally. Gossip is optimized for heavy traffic.

Let us go one level more deeper to understand how the underlying service is created and dispatched to the worker nodes. Before creating the service, let us first create a new overlay network called mynetwork.

The inotify triggers the relevant output accordingly:

Let’s create our first service:

$sudo docker-machine ssh test-master1 ‘sudo docker service create –name collabnix –replicas 3 \
   –network mynetwork dockercloud/hello-world

Once you run the above command, 3 replicas of services gets generated and distributed across the cluster nodes.

[Under the hood] – Let’s understand what happens whenever a new service is created.

 

Whenever we create overlay network through “docker network create -d overlay” command, it basically goes to manager. Manager is built up of multiple pipeline stages. One of them is Allocator. Allocator takes the network creation request and choose particular pre-defined sub network that is available. Allocation purely happen in the memory and hence it goes quick. Once network is created, it’s time to connect service to that network. Say, you start with service creation, orchestrator is involved and try to generate the requisite number of tasks which is nothing but containers in real world. But the tasks needs IP address, VXLAN ids as the overlay network needs that too. The allocation happens in the manager nodes. Once allocation gets completed, tasks are created and the state is preserved in the raft store. Once allocation is done, only then the scheduler will be able to move that particular task into the assigned state which is then dispatched to one of the worker node. Manager can also be worker. Every task goes through multiple stages – New, Allocated, Assigned etc. if the task has not been moved to allocator stage, it will not be assigned to worker nodes. With the help of network control plane(gossip protocol), multiple tasks distributed across the multiple worker node is taken care and managed effectively.

I hope you liked reading this deep-dive article. In future blog post, I will try to cover deep dive session into Docker network and volume aspects. Till then, Happy Swarming !!!

 

 

 

Have Queries? Join https://launchpass.com/collabnix

Ajeet Raina Docker Captain, ARM Innovator & Docker Bangalore Community Leader.

65 Replies to “Docker 1.12 Swarm Mode – Under the hood”

  1. I’m not sure exactly why but this blog is loading incredibly slow for me. Is anyone else having this issue or is it a issue on my end? I’ll check back later and see if the problem still exists.

  2. Hello there! Do you know if they make any plugins to help with Search Engine Optimization? I’m trying to get my blog to rank for some targeted keywords but I’m not seeing very good gains. If you know of any please share. Many thanks!

  3. I love your blog.. very nice colors & theme. Did you create this website yourself or did you hire someone to do it for you? Plz reply as I’m looking to construct my own blog and would like to know where u got this from. thanks

  4. obviously like your website but you have to check the spelling on several of your posts. Many of them are rife with spelling problems and I to find it very troublesome to inform the truth nevertheless I’ll certainly come back again.

  5. Hiya! I simply would like to give an enormous thumbs up for the nice info you have here on this post. I will likely be coming back to your blog for extra soon.

  6. The very crux of your writing whilst appearing agreeable at first, did not really settle properly with me after some time. Someplace throughout the paragraphs you were able to make me a believer unfortunately just for a very short while. I nevertheless have got a problem with your leaps in logic and you would do well to fill in all those gaps. In the event you can accomplish that, I could undoubtedly be fascinated.

  7. Whats Going down i am new to this, I stumbled upon this I have discovered It positively useful and it has helped me out loads. I’m hoping to give a contribution & aid other users like its helped me. Great job.

  8. It is really a nice and helpful piece of information. I’m glad that you shared this useful info with us. Please keep us up to date like this. Thanks for sharing.

  9. Perfectly pent subject matter, appreciate it for selective information. “The bravest thing you can do when you are not brave is to profess courage and act accordingly.” by Corra Harris.

  10. I’ll right away take hold of your rss as I can’t in finding your e-mail subscription hyperlink or newsletter service. Do you have any? Please let me recognize in order that I may just subscribe. Thanks.

  11. I cling on to listening to the news update speak about receiving free online grant applications so I have been looking around for the best site to get one. Could you advise me please, where could i get some?

  12. What i don’t realize is actually how you’re not really much more well-liked than you may be now. You are very intelligent. You realize thus significantly relating to this subject, produced me personally consider it from numerous varied angles. Its like men and women aren’t fascinated unless it is one thing to accomplish with Lady gaga! Your own stuffs excellent. Always maintain it up!

  13. Perfectly indited written content, thank you for selective information. “He who establishes his argument by noise and command shows that his reason is weak.” by Michel de Montaigne.

  14. Thank you for some other great post. The place else could anyone get that type of info in such a perfect way of writing? I’ve a presentation subsequent week, and I am on the look for such information.

  15. Good – I should definitely pronounce, impressed with your website. I had no trouble navigating through all tabs as well as related information ended up being truly easy to do to access. I recently found what I hoped for before you know it at all. Reasonably unusual. Is likely to appreciate it for those who add forums or anything, web site theme . a tones way for your client to communicate. Nice task.

  16. I’d have to examine with you here. Which is not one thing I usually do! I take pleasure in reading a post that may make folks think. Additionally, thanks for permitting me to comment!

  17. I cling on to listening to the news speak about receiving boundless online grant applications so I have been looking around for the finest site to get one. Could you tell me please, where could i find some?

  18. Great post. I used to be checking constantly this weblog and I am impressed! Very helpful information specifically the ultimate phase 🙂 I deal with such info a lot. I used to be seeking this particular information for a very lengthy time. Thank you and good luck.

  19. It’s in reality a nice and helpful piece of info. I’m satisfied that you simply shared this useful info with us. Please keep us up to date like this. Thanks for sharing.

  20. 93,435 orgasm super best orgazm FREE videos found on XVIDEOS for this search.
    y mi mujer infieles cowgirl getting orgasm by large cock find6 xyz slud exlatinahotx legal porns porns pretty teenager super hardcore lesbians pretty anal
    piledriver vintage casting xxxxxx boss xxx orgasm orgasms xxx orgazm casting feet mom and san xxx video.

  21. Hmm is anyone else experiencing problems with the images on this blog loading? I’m trying to find out if its a problem on my end or if it’s the blog. Any responses would be greatly appreciated.

  22. Thanks , I’ve just been looking for info about this topic for ages and yours is the best I’ve discovered so far. But, what about the conclusion? Are you sure about the source?

  23. I wish to get across my passion for your kind-heartedness for visitors who really need guidance on that concept. Your special commitment to passing the message across ended up being definitely beneficial and have continuously allowed somebody much like me to attain their endeavors. Your entire informative information denotes a great deal to me and especially to my mates. Thanks a ton; from each one of us.

Leave a Reply

Your email address will not be published.

© Copyright Collabnix Inc

Built for Collabnix Community, by Community