Docker Compose v3.1 file format now supports Docker 1.13.1 Secret Management

Estimated Reading Time: 5 minutes

Docker Engine 1.13.1 went GA last week and introduced one of the most awaited feature called Secrets Management . With a mission to introduce a container native solution that strengthens the Trusted Delivery component of container security, new Secrets API is rightly integrated into Docker 1.13.1 Orchestration engine.The new secrets-management capabilities are also included in Docker Datacenter as part of the Docker 1.13.1 release.

docker secrets

What are secrets all about?

It is a blob of data, such as password, SSH private keys, certificates,API keys, and encryption keys etc..In broader term, it can be anything that can be tightly control access to.The secrets-management capability is the latest security enhancement integrated into the Docker platform so as to ensure applications are safer in a containerized environment.This is going to benefit financial sector players who look for hybrid cloud security strategy.

 

Why do we need Docker secrets?

There has been numerous concerns over environmental variables which are being used to pass configuration & settings to the containers.Environmental variables are easily leaked when debugging and exposed into many places including child processes, hosting secrets on a server etc.

Consider a Docker compose file for WordPress application:

wordpress:
image: wordpressapp
links:
– mariadb:mysql
environment:
– WORDPRESS_DB_PASSWORD=<password>
ports:
– “80:80”
volumes:
– ./code:/code
– ./html:/var/www/html

As shown above, environmental variables are insecure in nature because they are accessible by any process in the container, preserved in intermediate layers of an image, easily accessible through docker inspect and lastly, it can get shared with any container linked to the container. To overcome this, one can use secrets to manage any sensitive data which a container needs at runtime aand there is no need to store in the image . A  given secret is only accessible to those services which have been granted explicit access to it, and only while those service tasks are running.

How does it actually work?

Docker secrets is currently supported for Swarm mode only starting Docker Engine 1.13.1. If you are using Docker 1.12.x you might need to upgrade to the latest 1.13.x release to use this feature. To understand how secret works under Docker Swarm mode, you can follow the below process flow:

 

secret_compose

 

Docker Compose v3.1 File Format now supports Secrets

Docker compose file format v3.1 is available and requires Docker Engine 1.13.0+. It introduced support for secrets for the first time which means that now you can use secrets inside your docker-compose file.

compose_matrix

Let us test-drive Compose v3.1 file format to see how secrets can be implemented using the newer docker stack deploy utility as shown below:

Ensure that you have the latest Docker 1.13.1 running on your Swarm Mode cluster:

secret_0

I will leverage 4-node Swarm Mode cluster to test the secret API:

secret_1

 

Let us first create a secret using docker secret create utility as shown:

$date | md5sum | docker secret create collab_mysqlpasswd –
72flsq9lhuj8je20y7bzfxyld

ollab# date | md5sum | docker secret create collab_mysqlrootpasswd –
1g329zm35umunim61r8q49res

collab# date | md5sum | docker secret create collab_wordpressdbpasswd –
tfxq1bm2cn54he03uzdaar91i

 

Listing the secret using the below command:

secret_101

Create a docker-compose.yml file with the below entry:

PLEASE NOTE: No Compose binaries are required to run the below command. All you require is Compose v3.1 file format for this to work.

compose-1compose-2compose-3

You can copy the whole code from here

Let us now use docker stack deploy to build up the services containing secrets:

$docker stack deploy –compose-file=./docker-compose.yml collab

Updating service collab_mysql (id: yn9fqojgmtmzukqnn3tfa6wlk)

Updating service collab_web (id: xw7kx49sqrkaqriikm5lsbqmj)

Verify the services are up and running:

collab_5

Let us verify if secret got stored under every container:

root@master101:/collab# docker exec -it 35f cat /run/secrets/mysqlpasswd
050a58c339431a5c9a6a6b8a15bead91  –

As shown above, one can use docker exec to connect to the container and read the contents of the secret data file, which defaults to being readable by all and has the same name as the name of the secret.

Key Takeaways:

  • Docker secrets are only available to swarm services, not to standalone containers. To use this feature, consider adapting your container to run as a service with a scale of 1.
  • No Compose binaries are required to run docker stack deploy. All you require is Compose v3.1 file format for this to work.
  • Raft data is encrypted in Docker 1.13 and higher.
  • It is recommended to update all of your manager nodes to Docker 1.13 to prevent secrets from being written to plain-text Raft logs.

 

What’s new upcoming in Docker Compose v1.9.0?

Estimated Reading Time: 5 minutes

Docker Compose has gained lots of attention in the recent past due to its easy one-liner installation(on Linux, Windows & Mac OS X), easy-to-use JSON & YAML format support , available sample docker-compose files on GITHUB  and a one-liner command to create and start all the services from your configuration. If you are looking out for Microservices implementation, Docker Compose is a great tool to get started with. With Compose, you can define and run complex application with Docker. Also, you define a multi-container application in a single file, then spin up your application in a single command which takes care of linking services together through Service Discovery.

cmp011

Docker Compose 1.9 is currently under RC4 phase and nearing the Final Release. Several new features and improvements in terms of Networking, Logging & Compose CLI has been introduced. With this release, Docker Compose version 2.1 has been introduced for the first time.This release will support the setting up of  volume labels and network labels in YAML specification. BUT there is a good news for Microsoft Windows enthusiasts. Interactive mode for docker-compose run and docker-compose exec is now supported on Windows platforms and this is surely going to help Microsoft enthusiasts to play around with the services flawlessly.

The below picture shows what major features has been introduced since last year in Docker Compose release:

pic009

In case you are very new to Docker Compose, I suggest you to read this official documentation. If you are an experienced Compose user and curious to know how Docker Compose fits into Swarm Mode, don’t miss out my recent blog post. Under this blog post, we will look at the new features which are being introduced under Docker Compose 1.9 release.

Installation of Docker Compose v1.9

On Windows Server 2016 system, you can run the below command to get started with Docker Compose 1.9-rc4 release.

cmp1

If you are on Linux host, the installation just goes flawless as shown below:

# curl -L https://github.com/docker/compose/releases/download/1.9.0-rc3/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose

# chmod +x /usr/local/bin/docker-compose$ sudo docker-compose -vdocker-compose version 1.9.0-rc3, build fcd38d3

cmp4

Introduction of Version 2.1 YAML specification format for the first time

Docker 1.9 introduces the newer version of Docker Compose YAML specification format rightly called “Version: 2.1” for the first time. To test drive, I created a docker-compose file for my wordpress application and it just worked well.

cmp5

The docker-compose up -d just went good as shown below:

cmp6

We can have a look at the list of services running using Docker compose as shown below:

cmp7

Interactive Mode for docker exec & docker run

Though this feature has been there for Linux users quite for sometime, it has been newly introduced and supported on Windows Platform too. In case you are new to docker-compose run command, here is the simplified way to demonstrate it.

On Linux Host:

cmp9

Note: In case you are new to docker-compose config command, it is a CLI tool which validates your Docker compose file.

Cool. One can use docker-compose run command to target one service out of several services mentioned under docker-compose.yml file and interact with that particular service without any issue.

On Windows Host:

To quickly test this feature, I spun up Windows Server 2016 on Azure, installed Docker and Docker Compose and forked https://github.com/ajeetraina/Virtualization-Documentation repository which has collection of Windows Docker images. Though it was quite slow in the beginning, but once pulled bringing up services using Docker Compose was pretty quick.

NOTE: When running docker-compose, you will either need to explicitly reference the host port by adding the option “-H tcp://localhost:2375” to the end of this command (e.g. docker-compose -H “tcp://localhost:2375” or by setting your DOCKER_HOST environment variable to always use this port (e.g. $env:DOCKER_HOST=”tcp://localhost:2375”

cmp31

cmp32

cmp33As shown below, the services finally were up and running and one can easily check through docker-compose ps command as shown below:

 

cmp11

cmp14

Let us test docker-compose run feature now. I tried targeting the db service and running cmd command to see if it works well.

cmp21

cmp23

That’s really cool. Believe me, it was quite quick in bringing up command prompt.

Support for setting volume labels and network labels in docker-compose.yml

This is an important addition to Docker compose release. There has been several ask from Docker community user to bring up this feature and Docker team has done a great job in introducing it under this release.

cmp5

If you look at the last few lines, the volume labels has been specified in the following format:

volumes:

    volume_with_labels:

        labels:

            –   “alpha=beta”

To verify if it rightly build up the container with the volume labels, one can issue the below command:

cmp10

In the upcoming posts, I will be covering more features and bug fixes introduced under Docker Compose 1.9. Keep watching this space for further updates.

Running Prometheus Docker container for monitoring Microservices on Raspberry Pi

Estimated Reading Time: 5 minutes

In the previous post, we talked about running Nagios container on Raspberry Pi for the first time.Nagios is a free and open source monitoring and alerting service which basically collects the statistics of your server using agents like nrpe, check_mk or using SNMP and sends the alert if the metric value is above the predefined threshold. It is a great tool for monitoring the nodes or monolithic application but with the advent of the ‘microservices’ architecture and Docker, monitoring the small piece of services running across the cluster becomes very important.With that also grows the need to monitor these services around the clock, to maintain the healthy functioning of the application.

pi33

One of the most talked-about monitoring tools are Prometheus & Grafana. Prometheus is an open-source systems monitoring and alerting toolkit written in Go. It is a next-generation monitoring system and well appreciated by Docker Core Engineering Team. It’s a service especially well designed for containers and provides perspective about the data intensiveness of this new age and how even Internet scale companies have had to adapt.

On the other hand, Grafana can be rightly called the face of Prometheus. It is also an open-source, general purpose dashboard and graph composer, which runs as a web application. It supports graphite, influxdb or opentsdb as backends. While Prometheus exposes some of its internals like settings and the stats it gathers via basic web front-ends, it delegates the heavy lifting of proper graphical displays and dashboards to Grafana. For anyone who want to collect metrics from the development environment to be graphed and measured, Prometheus as time series store and Grafana as a visualization tool is the perfect solution.

What’s unique about Prometheus?

Prometheus is a self-hosted set of tools which collectively provide metrics storage, aggregation, visualization and alerting. Usually most of the available monitoring tools are push based, i.e. agents on the monitored servers talk to a central server (or set of servers) and send out their metrics. Prometheus is little different on this aspect – It is a pull based server which expects monitored servers to provide a web interface from which it can scrape data. This is a unique characteristic of Prometheus.

The main features of Prometheus can be summed below:

  • Decentralized architecture
  • Scalable data collection
  • Powerful Query language (leverages the data model for meaningful alerting (including easy silencing) and graphing (for dashboards and for ad-hoc exploration).
  • A multi-dimensional data model.( data can be sliced and diced at will, along dimensions like instance, service, endpoint, and method)
  • No reliance on distributed storage; single server nodes are autonomous.
  • Time series collection happens via a pull model over HTTP.
  • Pushing time series is supported via an intermediary gateway.
  • Targets are discovered via service discovery or static configuration.
  • Multiple modes of graphing and dashboard support.

You can deep dive more into its architecture at https://prometheus.io/docs/introduction/overview/#architecture.

I was in need of lightweight monitoring tool like Prometheus and Fancy UI like Grafana for my Raspberry Pi which I can spin up as Docker container in few seconds and then grow /spread it across the cluster primarily to test it for Docker 1.12 Swarm Mode. Unfortunately, I didn’t find any Docker image for Prometheus. I decided to build one for myself to explore it on Raspberry Pi 3 box.

After spending considerable amount of time on Pi box, I was able to build the first Prometheus Docker image and pushed it to Dockerhub. In case you are interested, check out my github page : https://github.com/ajeetraina/prometheus-armv7 to build and run your own Prometheus container.

[Thanks to Alex Ellis, #Docker Captain for reviewing this container and cleaning the Dockerfile.]

 

pi3

Running Prometheus & Grafana Docker Image on Raspberry Pi

Follow the below command to spin up Prometheus container on the fly:

pi11

Luckily, there was already a Grafana ARM image available in Dockerhub which I spent no minute to spin up easily as shown below:

 

pi22

You can also use docker-compose to bring up both the container in a single shot as shown:

pi@raspberrypi:~/testingPi $ cat docker-compose.yml

prometheus:
image: ajeetraina/prometheus-armv7
volumes:
– ./prometheus.yml:/etc/prometheus/prometheus.yml
command:
– ‘-config.file=/etc/prometheus/prometheus.yml’
ports:
– ‘9090:9090’

grafana:
image: fg2it/grafana-armhf:v3.1.1
environment:
– GF_SECURITY_ADMIN_PASSWORD=collabnix

ports:
– “3000:3000”

Ensure that you have either sample or your own customized prometheus.yml file under / directory to run this container.

Just one command and you are ready to spin up both the containers as shown below:

pi432pi90

That’s it. Browse to http://<ipaddress>:9090 to check out the Prometheus UI.

pi323

Also, Browse to http://<ipaddress>:3000 to open up fancy Grafana UI as shown below. Supply admin/collabnix as credentials to enter into Grafana UI.

graf2

Importing Prometheus Data into Grafana

Once you login into Grafana UI, click on Data Sources followed by “Add Data Source” button. You will find the below screen:

pi44

Next, Let’s import the prometheus into Grafana dashboard:

import

pi2345

I firmly believe that the newly built Prometheus container is sure to help developers to monitor their microservice application on top of Raspberry Pi box.In the next post, I will talk about the other available data sources and how it integrates with Prometheus and Grafana as a monitoring stack in detail.

Demonstrating Docker 1.12 Service Discovery with Docker Compose

Estimated Reading Time: 4 minutes

CofvQBxUAAE1UpT

Docker 1.12 is GA now. Thanks to Docker Inc. core Team and community support for releasing it on time as promised during DockerCon 2016.In my earlier post, I briefed around Service Discovery concept and how does it work in terms of  Swarm Mode. As we are well aware, Swarm Mode is an optional mode which can be ON and OFF based on your choice and requirement. Under this blog post, I am going to demonstrate service discovery while playing around with docker-compose.

Compose

I have a Ubuntu 16.04 machine upgraded with Docker 1.12 GA version. I have docker-compose 1.8.0 running on the same machine. To demonstrate how one service discover others through hostname, I am going to build a very simple Nginx application and a proxy which redirects to the default Nginx web application through the following structure:

Docker_11

The collabapp contains a very simple Dockerfile which pulls nginx running alpine image from Dockerhub.

Docker_12

 

The collabproxy contains Dockerfile to use the same nginx image with the customized proxy.conf. All it does is whenever there is any request coming to port: 80 will be redirected to http://collabapp which is nothing but a container running web application.

Docker_13

A quick look at docker-compose.yml file:

Docker_19

One thing which you would notice here is that we are NOT providing any “link:” parameter for linking both the services. Docker 1.12 handles this service name aspect internally through the in-built service discovery concept.

Let’s build up the containers using docker-compose as shown below:

Docker_14

Let’s bring up both the containers:

Docker_15

There are two services running – test_collabapp_1 and test_collabproxy_1. You can view the services running in a single shot as shown:

Docker_16

Let’s try to reach out to collabapp from within collabproxy using hostname and see if internal DNS resolves that or not.

Docker_16

As shown above, we are able to ping web application from proxy container using hostname.

To see how does it resolve that hostname, let us look up few interesting files inside the container:

Docker_17

Also, the mount command too shows how the /etc/resolv.conf, /etc/hostname and related files are created for service discovery to be enabled.

Docker_18

The 127.0.0.11 is an embedded DNS server which enables the service discovery functionality.The embedded DNS server is really a light weight server only interested in responding to container name/alias queries and for everything else it just acts as the proxy to any other DNS server.There is a socket created in the container namespace and which listens on it in the daemon itself and respond queries using that. The socket acts as the demux identifier to identify the network in case of unqualified name queries.

Let’s look at an interesting aspect of Service Discovery while we scale the service to run multiple containers across the same machine.

Docker_20

Now, the docker-compose ps shows scaled value of 5.

Docker_21

If you enter into collabproxy container and try to check the nslookup output:

Docker_23

Wow ! The command returned five different IP addresses from in-built DNS and all it does is round-robin DNS to load-balance across those 5 different containers.