A Docker Swarm consists of multiple Docker hosts which run in swarm mode and act as managers (to manage membership and delegation) and workers (which run swarm services). When you create a service, you define its optimal state (number of replicas, network and storage resources available to it, ports the service exposes to the outside world, and more). Docker works to maintain that desired state. For instance, if a worker node becomes unavailable, Docker schedules that node’s tasks on other nodes. A task is a running container which is part of a swarm service and managed by a swarm manager, as opposed to a standalone container.
Let us talk a bit more about Services…
A Swarm service is a 1st class citizen and is the definition of the tasks to execute on the manager or worker nodes. It is the central structure of the swarm system and the primary root of user interaction with the swarm. When one create a service, you specify which container image to use and which commands to execute inside running containers.Swarm mode allows users to specify a group of homogenous containers which are meant to be kept running with the docker service CLI. Its ever running process.This abstraction which is undoubtedly powerful, may not be the right fit for containers which are intended to eventually terminate or only run periodically. Hence, one might need to run some containers for specific period of time and terminate it acccordingly.
Let us consider few example:
You are a System Administrator who wishes to allow users to submit long-running compiler jobs on a Swarm cluster
A website which needs to process all user uploaded images into thumbnails of various sizes
An operator who wishes to periodically run docker rmi $(docker images –filter dangling=true -q) on each machine
Today Docker Swarm doesn’t come with this feature by default. But there are various workaround to make it work. Under this tutorial, we will show you how to run on-off cron-job on 5-Node Swarm Mode Cluster.
Click on Spanner on the left side of the screen to bring up 5-Node Swarm Mode Cluster
Verifying 5-Node Swarm Mode Cluster
$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
y2ewcgx27qs4qmny9840zj92p * manager1 Ready Active Leader
qog23yabu33mpucu9st4ibvp5 manager2 Ready Active Reachable
tq0ed0p2gk5n46ak4i1yek9yc manager3 Ready Active Reachable
tmbcma9d3zm8jcx965ucqu2mf worker1 Ready Active
dhht9gr8lhbeilrbz195ffhrn worker2 Ready Active
Cloning the Repository
git clone https://github.com/crazy-max/swarm-cronjob
Last week I attended Dockercon 2018 EU which took place at Centre de Convencions Internacional de Barcelona (CCIB) in Barcelona, Spain. With over 3000+ attendees from around the globe, 52 breakout sessions, 11 Community Theatres, 12 workshops, over 100+ total sessions, exciting Hallway Tracks & Hands-on Labs/Trainings, paid trainings, women’s networking event, DockerPals and so on..Dockercon allowed developers, sysadmins, Product Managers & industry evangelists come closer to share their wealth of experience around the container technology. This time I was lucky enough to get chance to Emcee for Docker for Developer Track for the first time. Not only this, I conducted Hallway Track for OpenUSM project & DockerLabs community contribution effort. Around 20-30 participants showed up their interest to learn more around this system management, monitoring & Log Analytics tool.
This Dockercon we had Docker Captains Summit for the first time where the entire day was dedicated to Captains. On Dec #3 ( 10:00 AM till 3:00 PM), we got chance to interact with Docker Staffs, where we put all our queries around Docker Future roadmap. It was amazing to meet all young Captains who joined us this year as well as getting familiar to what they have been contributing to during the initial introductory rounds.
This Dockercon, there has been couple of exciting announcements. 3 of the new features were targeted at Docker Community Edition, while the two were for Docker Enterprise customers. Here’s a rundown of what I think are the most 5 exciting announcements made last week –
#1. Announcement of Cloud Native Application Bundles(CNAB)
Microsoft and Docker have captured a great piece of attention with announcement around CNAB – Cloud Native Application Bundles.
What is CNAB?
Cloud Native Application Bundles (CNAB) are a standard packaging format for multi-component distributed applications. It allows packages to target different runtimes and architectures. It empowers application distributors to package applications for deployment on a wide variety of cloud platforms, cloud providers, and cloud services. It also provides the capabilities necessary for delivering multi-container applications in disconnected environments.
Is it platform-specific tool?
CNAB is not a platform-specific tool. While it uses containers for encapsulating installation logic, it remains un-opinionated about what cloud environment it runs in. CNAB developers can bundle applications targeting environments spanning IaaS (like OpenStack or Azure), container orchestrators (like Kubernetes or Nomad), container runtimes (like local Docker or ACI), and cloud platform services (like object storage or Database as a Service). CNAB can also be used for packaging other distributed applications, such as IoT or edge computing. In nutshell, CNAB are a package format specification that describes a technology for bundling, installing, and managing distributed applications, that are by design, cloud agnostic.
Why do we need CNAB?
The current distributed computing landscape involves a combination of executable units and supporting API-based services. Executable units include Virtual Machines (VMs), Containers (e.g. Docker and OCI) and Functions-as-a-Service (FaaS), as well as higher-level PaaS services. Along with these executable units, many managed cloud services (from load balancers to databases) are provisioned and interconnected via REST (and similar network-accessible) APIs. The overall goal of CNAB is to provide a packaging format that can enable application providers and developers with a way of installing a multi-component application into a distributed computing environment, supporting all of the above types.
Is it open source? Tell me more about CNAB format?
It is an open source, cloud-agnostic specification for packaging and running distributed applications. It is a nascent specification that offers a way to repackage distributed computing apps
The CNAB format is a packaging format for a broad range of distributed applications. It specifies a pairing of a bundle definition(bundle.json) to define the app, and an invocation image to install the app.
The bundle definition is a single file that contains the following information:
Information about the bundle, such as name, bundle version, description, and keywords
Information about locating and running the invocation image (the installer program)
A list of user-overridable parameters that this package recognizes
The list of executable images that this bundle will install
A list of credential paths or environment variables that this bundle requires to execute
What’s Docker future plan to do with CNAB?
This project was incubated by Microsoft and Docker 1 year back. The first implementation of the spec is an experimental utility called Docker App, which Docker officially rolled out this Dockercon and expected to be integrated with Docker Enterprise in near future. Microsoft and Docker plan to donate CNAB to an open source foundation publicly which is expected to happen early next year.
#2. Support for using Docker Compose on Kubernetes.
On the 2nd day of Dockercon, Docker Inc. open sourced Compose on Kubernetes project. Docker Enterprise Edition already had this capability enabled starting Compose File version 3.3 where one can use the same docker-compose.yml file for Swarm deployment as well as one can specify Kubernetes workloads whenever stack is deployed.
What benefit does this bring to Community Developers?
By making it open source, Docker, Inc has really paved a way of infinite possibilities around simplified way of deploying Kubernetes application. Docker Swarm gained popularity because of its simplified approach of application deployment using docker-compose.yml file. Now the community developers can use the same YAML file to deploy their K8s application.
Imagine, you are using Docker Desktop on your Macbook. Docker Desktop provides capability of running both Swarm & Kubernetes. You have context set to GKE cluster which is running on Google Cloud Platform. You just deployed your app using docker-compose.yml on your local Macbook. Now you want to deploy it in the same way but this time on your GKE cluster. Just use docker stack deploy command to deploy it to GKE cluster. Interesting, Isn’t it?
How does Compose on Kubernetes architecture look like?
Compose on Kubernetes is made up of server-side and client-side components. This architecture was chosen so that the entire life cycle of a stack can be managed. The following image is a high-level diagram of the architecture:
If you’re interested to learn further, I would suggest you to visit this link.
This controller uses the standard Kubernetes extension points to introduce the `Stack` to the Kubernetes API. You can use any Kubernetes cluster you like, but if you don’t already have one available then remember that Docker Desktop comes with Kubernetes and the Compose controller built-in, and enabling it is as simple as ticking a box in the settings
Check out the latest doc which shows how to make it work with AKS here.
#3. Introducing Docker Desktop Enterprise
The 3rd Big announcement was an introduction to Docker Desktop Enterprise. With this, Docker Inc. made a new addition to their desktop product portfolio which currently includes the free Docker Desktop Community products for MacOS and Windows. Docker Desktop is the simplest way to get started with container-based development on both Windows 10 and macOS with a set of features now available for the enterprise.
How will Docker Desktop Enterprise be different from Docker Desktop Community Edition?
Good question. Docker Desktop has Docker Engine and Kubernetes built-in and with the addition of swappable version packs you can now synchronize your desktop development environment with the same Docker API and Kubernetes versions that are used in production with Docker Enterprise. You get the assurance that your application will not break due to incompatible API calls, and if you have multiple downstream environments running different versions of the APIs, you can quickly change your desktop configuration with the click of a button.
Not only this, with Docker Desktop Enterprise, you get access to the Application Designer which is a new workflow that provides production-ready application and service templates that let you get coding quickly, with the reassurance that your application meets architectural standards
For those who are interested in Docker Desktop Enterprise – Please note that it is expected to be available for preview in January & General Availability is slated to happen during 1H 2019.
#4. From Zero to Docker in Seconds with “docker assemble” CLI
This time, Docker Team announced a very interesting docker subcommand rightly named as “assemble” to the public. Ann Rahma and Gareth Rushgrove from Docker, Inc. announced assemble, a new command that generates optimized images from non dockerized apps. It will get you from source to an optimized Docker images in seconds.
Here are few of interesting facts around docker assemble utility:
Dockerassemble has capability to build an image without a Dockerfile, all about auto detecting the code framework.
It generates docker images (and lot more) from your code with single command and zero effort! which mean no more dockerfile needed for your app till you have a config (.pom file there).
It can analyze your applications, dependencies, and caches, and give you a sweet Docker image without having to author your own Dockerfiles.
It is built on top of buildKit, will auto detect framework, versions etc. from a config file (.pom file) and automatically add dependencies to the image label, optimize image size and push.
Docker Assemble can also figure out what ports need to be published and what healthchecks are relevant.
The docker–assemble builds app without configuration files, without Dockerfile, just a git repository to deploy
Is it an open source project?
It’s an enterprise feature for now — not in the community version. It is available for a couple languages and frameworks (like Java as demonstrated on Dockercon stage).
How is it different from buildpack?
By reading all through its feature, Docker assemble might look very similar to buildpacks as it overlap with some of the stuff docker-assemble does. But the huge benefit with assemble is that it’s more than just an image (also ports, healthchecks, volume mounts, etc), and it’s integrated into the enterprise toolchain. The docker-assemble is sort of an enterprise-grade buildpack to help with digitalization.
Keep eye on my next blog post to get more detail around the fancy docker assemblecommand.
#5. Docker-app & CNAB together for the first time
On the 2nd day of Dockercon, Docker confirmed that they are the first to implement CNAB for containerized applications and will be expanding it across the Docker platform to support new application development, deployment and lifecycle management. Initially CNAB support will be released as part of our docker-app experimental tool for building, packaging and managing cloud-native applications. With this, Docker now lets you package CNAB bundles as Docker images, so you can distribute and share through Docker registry tools including Docker Hub and Docker Trusted Registry. Additionally, Docker will enable organizations to deploy and manage CNAB-based applications in Docker Enterprise in the upcoming months.
Can I test the preview binaries of docker-app which comes with CNAB support?
Yes, you can find some preview binaries of docker-app with CNAB support here.The latest release of Docker App is one such tool that implements the current CNAB spec. Tt can be used to both build CNAB bundles for Compose (which can then be used with any other CNAB client), and also to install, upgrade and uninstall any other CNAB bundle.
Under the newer Docker Engine 18.09 release, a new feature called CE-EE Node Activate has been introduced. It allows a user to perform an in-place seamless activation of the Enterprise engine feature set on an existing Community Edition (CE) node through the Docker command line.CE-EE Node Activate applies a license, and switch the Docker engine to the Enterprise engine binary.
Docker Community Edition (CE) version must be 18.09 or higher.
All of the Docker packages must be installed: docker-cli, docker-server, and containerd.
Node-level Engine activation between CE and EE is only supported in the same version of Docker Enterprise Engine for Docker
Number of Instance
Google Cloud Platform
Create an account with Google Cloud Engine (Free Tier)
~$ sudo docker version
API version: 1.39
Go version: go1.10.4
Git commit: 4d60db4
Built: Wed Nov 7 00:49:01 2018
Server: Docker Engine - Community
API version: 1.39 (minimum version 1.12)
Go version: go1.10.4
Git commit: 4d60db4
Built: Wed Nov 7 00:16:44 2018
Running Nginx Docker container
$ sudo docker run -d -p 80:80 nginx
Unable to find image 'nginx:latest' locally
latest: Pulling from library/nginx
a5a6f2f73cd8: Pull complete
67da5fbcb7a0: Pull complete
e82455fa5628: Pull complete
Status: Downloaded newer image for nginx:latest
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ba4a5822d7c9 nginx "nginx -g 'daemon of…" 5 seconds ago Up 3 seconds 0.0.0.0:80->80/tcp peaceful_swanson
Verifying Nginx Docker container Up and Running
~$ sudo curl localhost:80
<title>Welcome to nginx!</title>
margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif;
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
Commercial support is available at
<p><em>Thank you for using nginx.</em></p>
Connect your system to DockerHub Account
$sudo docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.
WARNING! Your password will be stored unencrypted in /home/joginderkour1950/.docker/config.json.
Configure a credential helper to remove this warning. See