Building Web Frontend/UI for Local Docker Registry using SUSE Portus

If you are looking out for Web UI for your private Docker Registry, I would recommend to test-drive a tool called “Portus” powered by SUSE Team. Portus provides a useful and powerful UI on top of your registry. It is an open source authorization service and a user interface for the next generation of the Docker registry. Portus targets version 2 of the Docker Registry API.

12

Docker Registry(a.k.a Docker Distribution)  is a storage and content delivery system, holding named Docker images. It is a stateless, highly scalable server side application that stores and lets you distribute Docker images. The Registry is open-source, under the permissive Apache license. Basically,  It is the backend behind the Docker Hub.  As it is completely  open source,  it clearly means that you can have your own Docker Registry on your own servers.

I went through Portus official website and found that it holds a large number of great features . Few of the notable features I was really keen to try out are listed below:

In my last blog post,  I tried setting up Portus on Alpine Linux. It requires workaround and few tweaking to get it up and running. Under this blog post, I will share my first experience with Portus on Ubuntu OS in detail . I had few of Docker host instances already up and  running Ubuntu 17.04 OS on which I want to try this tool and see how promising it looks.

Step:1 – Creating daemon.json under /etc/docker directory if it doesn’t exists:

master==>cat /etc/docker/daemon.json

{

“insecure-registries”: [“10.128.0.2:5000”]

}

 

Step:2 – Restart docker daemon so as to enable insecure registry parameter:

master==>systemctl restart docker

master==>docker info | tail -n 5

WARNING: No swap limit support

Experimental: false

Insecure Registries:

10.128.0.2:5000

127.0.0.0/8

Live Restore Enabled: false

 

Step:3 – Cloning Portus repository

master==>git clone https://github.com/ajeetraina/docker101

master==>cd docker101/play-with-docker/Portus

master==>./dependencies.sh

Currently there is a bug and this script helps to fix the issue. Please remember that the above script might or mightn’t work for other distribution like alpine. If you are on play-with-docker platform,you might need to follow the bug ID.

Leave this screen open and open up the new terminal.

Step:4 – Installing Docker Compose

master==> curl -L https://github.com/docker/compose/releases/download/1.12.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose master==> chmod +x /usr/local/bin/docker-compose

Step:5 – Finally run the below script which set up Portus tool

master==> cd docker101/play-with-docker/Portus

master==> cd Portus

master==> ./compose-setup -e `hostname -i`

portus_2

 

 

 

 

portus_3

Step:7 – Ensure that port 3000 is enabled and accessible so as to open it up in the browser.

portus_9

 

Step:8- Open http://10.128.0.2:3000 with your browser and perform the following steps:

1. Create an admin account

2. You will be redirected to a page where you have to register the registry. In this form: – Choose a custom name for the registry.

– Enter 10.128.0.2:5000 as the hostname.

– Do *not* check the “Use SSL” checkbox, since this setup is not using SSL.

portus_5

Once you input the parameter, it will show up as shown above.

Step:9 – It’s time to create Users and Team. Go to Admin > Users > Create New User.

portus_08

 

Step:10 – Create two teams – db_team and cloud_team as we are planning to host seperate private Docker Registry for each team.

portus_009

 

Step:11 – Testing the Local Docker registry from other machine:

worker3==>docker login -u ajeetraina -p ****** 10.128.0.2:5000

Login Succeeded

 

Step:12 – Let us try pulling a busybox Docker image and push it to a private registry:

master==>docker pull busybox

master==> docker tag busybox 10.128.0.2:5000/ajeetraina/busybox

master==> docker push 10.128.0.2:5000/ajeetraina/busybox

portus_111

Let us verify if Portus UI shows up the new Docker image added to the registry:

portus_112

Reference:

https://github.com/SUSE/Portus

http://port.us.org

Did you find this blog helpful?  Feel free to share your experience. Get in touch @ajeetsraina

If you are looking out for contribution, join me at Docker Community Slack Channel.

 

0
0

Test Drive 5 Cool Docker Application Stacks on play-with-docker (PWD) platform

Do you want to learn Docker FOR FREE OF COST? Yes, you read it correct. Thanks to a playground called “play-with-docker” – PWD in short. PWD is a website which allows you to create 5 instances to play around with Docker & Docker Swarm Mode for 4 hours – all for $0 cost. It is a perfect tool for demos, Meetups, beginners & advanced level training. I tested it on number of web browsers like Chrome, Safari, Firefox and IE and it just works flawlessly. You don’t need to install Docker as it comes by default. It’s ready-to-use platform.

learn

 

 

Currently, PWD is hosted on AWS instance type 2x r3.4xlarge  having 16 cores and 120GB of RAM. It comes with the latest Docker 17.05 release, docker-compose 1.11.1 & docker-machine 0.9.0-rc1. You can setup your own PWD environment in your lab using this repository. Credits to Docker Captain – Marcos Nils & Jonathan Leibuisky for building this amazing tool for Docker Community. But one of the most interesting fact about PWD is its based on DIND (Docker-in-a-Docker) concept. When you are playing around with PWD instances & building application stack, you are actually inside Docker container itself. Interesting, isn’t it? PWD gives you an amazing experience of having a free Alpine Linux  3.5 Virtual Machine in the cloud where you can build and run Docker containers and even create Multi-Node Swarm Mode Cluster.

Said that, PWD is NOT just a platform for beginners. Today, it has matured enough to run sophisticated application stack on top of it. Within seconds of time, you can setup Swarm Mode cluster running application stack.Please remember that PWD is just for trying out new stuffs with Docker and its application and NOT to be used for production environment. The instances will vanish after 4:00 hours automatically.

PWDD

 

 

 

Under this blog post, I am going to showcase 3 out of 5 cool Dockerized application stack which you can demonstrate to the advance level audience all running on PWD playground (listed below):

Docker UI Management & Local Registry Service

  • Bringing Portainer & Portus together for PWD under Swarm Mode

Building Monitoring Stack with Prometheus & Grafana:

  • Prometheus, Grafana & cAdvisor Stack on Swarm Mode

Demonstrating Voting Application under Swarm Mode

  • Voting App on Swarm Mode

Highly Available Web Application

  • LAMP Stack under Swarm Mode

Demonstrating RSVP

To make it quite simple, I have collected the list of sample application under https://github.com/ajeetraina/docker101

You can use git clone to pull the repository on the manager node:

$git clone https://github.com/ajeetraina/docker101

First, we need to setup Swarm Mode cluster on PWD. As Docker 17.05 already comes installed by default, we are all set to initialize the swarm mode cluster. Click on “New Instance” button on the left hand side of PWD and this will open up an instance on the right side as shown.Run the below command on the 1st instance as shown below:

$docker swarm init –advertise-addr <manager-ip>:2377 –listen-addr <manager-ip>:2377

This command will initialize the Swarm and suggest a command to join the worker node as shown:

$docker swarm join  –token <token-id> <manager-ip>:2377

Once you run the command on all the 4 worker nodes. You can go back to manager node and check the list of Swarm nodes:

33

 

 

1. Docker UI Management & Local Registry Service:

Setting up Portainer for Swarm Mode Cluster

It’s time to get started with our first application – Portainer. Portainer is an open-source lightweight management UI which allows you to easily manage your Docker host or Swarm cluster. Run the below command to setup Portainer for Swarm Mode cluster:

4

 

Ensure that portainer is up and running:

5

 

The moment Portainer service gets started, PWD displays 9000 port which Portainer works on as shown. You can click on this port number to directly open the Management UI.

6

 

 

This opens up a fancy UI which displays lots of management features related to image, container, swarm, network, volumes etc.

7

 

 

Portainer clearly displays the swarm cluster as shown:

8

 

Setting up Portus for Local Docker Registry:

Portus is an authorization service for your Docker registry. It provide a useful and powerful UI on top of your registry. To test driver Portus, one need to clone the repository:

$git clone https://github.com/SUSE/Portus && cd Portus

Run the below script with your manager IP to get Portus up and running:

9

 

 

10

 

 

15

 

 

Once this looks good, you should be able to browse its UI as shown:

12

 

 

13

 

 

 

 

I faced the issue related to Webpack::Rails::Manifest::ManifestLoadError which I got it fixed within few minutes. You now have control over Docker registry using this fancy UI. You can create Users, Organization for your development team and push/pull Docker images privately.

Hence, you can now open up Portainer and provide Portus as a local Docker registry instead of standard Docker registry. This makes Portainer & Portus work together flawlessly.

2. Building Monitoring Stack with Prometheus & Grafana

Prometheus is an open-source systems monitoring and alerting toolkit. Most Prometheus components are written in Go while some  written in Java, Python, and Ruby. It is designed for capturing high dimensional data. It is designed to be used for monitoring. On the other hand, ELK is a general-purpose NOSQL stack that is also very popular for monitoring and Logging.

To test-drive Prometheus & ELK, you can change the directory to /play-with-docker/docker-prometheus-swarm directory and run the stack deploy command as shown below:

$git clone https://github.com/ajeetraina/docker101

#cd docker101/play-with-docker/docker-prometheus-swarm/

$docker stack deploy -c docker-compose.yml myprom1

[ A Special Credits to Basi for building https://github.com/bvis/docker-prometheus-swarm repository & tremendous effort for enabling this solution]

16

 

 

That’s it. Your Prometheus, Grafana & cadvisor for ELK stack is ready to be used.

22

18

 

 

Demonstrating Voting Application under Swarm Mode

Voting app is a perfect piece of example where it showcase dependencies among the services, and a potential division of services between the manager and worker nodes in a swarm.  You can learn more about voting app and how it actually works under this link. To quickly demonstrate it, let us pick up the right directory under the pulled repository:

$cd play-with-docker/example-voting-app

Run the below command:

$docker stack deploy -c docker-stack.yml myvotingapp

vote1

 

 

vote2

 

 

vote4

 

In the next series of this blog post, I am going to cover on other 2 application stacks – CloudYuga RSVP & WordPress Web Application.

Did you find this blog helpful?  Feel free to share your experience. Get in touch @ajeetsraina

If you are looking out for contribution, join me at Docker Community Slack Channel.

0
0

What’s New in Docker 17.03 Volume Plugin Architecture & Specification?

Docker, Inc announced initial support for volume driver plugins for the first time under Docker 1.8 release. Since then, there has been subtle changes in terms of its volume plugin architecture. With the new Docker 17.03 Volume plugin architecture, writing your own Volume Plugin is quite simplified.

VolPlugin

 

Old Legacy Docker Volume Plugin Specification < 1.12 release

Evolution

Before 17.03 release,  Docker Volume Plugin Specification feature includes the standardization of  API interface. In case Docker daemon is already been running in the system and you write certain extension, the only way Docker daemon talks to extension is based on standardized API. As per Docker official page , you need to write 9 endpoints shown below:

1 2 3 4 5 6 7 8 9

 

New Docker Volume Plugin Specification > 1.12

With Docker 17.03, the new Volume Plugin Spec has been revamped. The new specification extends the standardization and talks about plugin packaging as a Docker Image. What it really mean is now you can now convert your extension/plugin into a Docker image which you can publish on Dockerhub. Interesting, isn’t it? In simple statement, now it is possible to publish Volume plugin in the form of Docker image which anyone can discover, install flawlessly onto their system and easy to configure & manage.New Docker Volume Plugins enable Engine deployments to be integrated with external storage systems such as Amazon EBS, and enable data volumes to persist beyond the lifetime of a single Docker host.

Before we build, store, install and manage the plugin, we need to go deeper in understanding the newer Docker Volume API design.

Understanding Docker Volume API Design:

As per the official Docker Volume Plugin page…

“The new Plugin API is RPC-style JSON over HTTP, much like webhooks.Requests flow from the Docker daemon to the plugin. So the plugin needs to implement an HTTP server and bind this to the UNIX socket mentioned in the “plugin discovery” section. All requests are HTTP POST requests.The API is versioned via an Accept header, which currently is always set to application/vnd.docker.plugins.v1+json.”

How Docker Volume Orchestration Works?

Orches

 

Playing around with RexRay Volume Plugin:

In my previous blog post, I talked about RexRay as a Volume Plugin. Let us look at various CLIs which can be used to play around with this plugin:

  • Listing the RexRay Volume Plugin:

1213

  • Disabling or Enabling the RexRay Volume Plugin:

141516

  • Verify that “Enabled=true” value gets listed once plugin is enabled back:

17

  • If Volume Plugin is in the form of Docker Image, then there should be a way to enter into this container. Right? Yes,it is possible. You can enter into a shell of RexRay Volume Plugin using docker-runc command.

1819

  • Checking the Plugin logs:

log

  • It’s time to use this plugin and create volume for your application:

vol1

 

  • Inspecting the volume:

vol2

 

I hope you found this blog useful.In my future blog post, I will talk further on how Volume Plugin Orchestration works in terms of Swarm Mode cluster.

0
0

Docker Service Inspection Filtering & Template Engine under Swarm Mode

Go programming language has really helped in shaping Docker as a powerful software and enabling fast development for distributed systems. It has been helping developers and operations team to quickly construct programs and tools for Cloud computing environment. Go offers built-in support for JSON  (JavaScript Object Notation) encoding and decoding, including to and from built-in and custom data types.

golang

 

In last 1 year, Docker Swarm Mode has matured enough to become production-ready. The Orchestration platform is quite stable with numerous features like Logging, Secrets Management, Security Scanning, improvement over Scheduling, networking etc. making it more simple to use and scale-out in just few liner commands. With the introduction of new APIs like swarm, node, volume plugins, services etc., Swarm Mode brings dozens of features to control every aspect of swarm cluster sitting on the master node. But when you start building services in the range of 100s & 1000s and that too distributed across another 100s and 1000s of nodes, there arise a need of quick and handy way of filtering the output, especially when you are interested to capture one specific data out of the whole cluster-wide configuration. Here comes ‘a filtering flag’ as a rescue.

swarm11

The filtering flag (-f or --filter) format is a key=value pair which is actually a very powerful weapon for developers & system administrators.If you have ever played around with Docker CLI, you must have used docker inspect command to get metadata on a container/ image. Using it with -f provides you more specific information like IP address, network etc. There are numerous guide on how to use filters with standalone host holding the Docker images but I found lack of guides talking about Swarm Mode filters.

Under this blog post, I have prepared a quick list of consolidated filtering commands and outputs in tabular format for Swarm Mode Cluster(shown below).

I have 3 node Swarm Mode cluster running on one of my Google Cloud Engine. Let us first create a simple wordpress application as shown below:

 

swarm-1

 

We will create a simple wordpress service as shown:

master==>docker service create –env WORDPRESS_DB_HOST=wordpressdb1 –env WORDPRESS_DB_PASSWORD=collab123 –network collabnet –replicas 4 –name wordpressapp –publish 80:80/tcp wordpress:latest

Below is the tabular list of commands and outputs which talks about various filtering mode for docker service inspect command:

 

Inspecting the “wordpress” service:

[wpsm_comparison_table id="3" class=""]

To retrieve the network information for specific service:

[wpsm_comparison_table id="6" class=""]

To list out the port which wordpress is using for specific service:

[wpsm_comparison_table id="7" class=""]

To list out the protocol detail for specific service:

[wpsm_comparison_table id="8" class=""]

To list out the type of Published port:

[wpsm_comparison_table id="9" class=""]

To verify if its DNS RR or Virtual IP(VIP) based for specific service:

[wpsm_comparison_table id="10" class=""]

 

To list out the type of Published port for specific service:

[wpsm_comparison_table id="11" class=""]

To list out the replication factor for a specific service:

[wpsm_comparison_table id="12" class=""]

To list out the environmental variable passed for specific service:

[wpsm_comparison_table id="13" class=""]

To list out the image detail for specific service:

[wpsm_comparison_table id="14" class=""]

To list out the complete networking details for specific service:

[wpsm_comparison_table id="15" class=""]

To list out detailed consolidated  meta data information for the specific service::

[wpsm_comparison_table id="4" class=""]
 
To list out further detailed information of the service :

[wpsm_comparison_table id="5" class=""]

Let us create another service called “wordpressdb1” which is actually a database service under Docker Swarm Mode:

$docker service create –replicas 1 –name wordpressdb1 –network collabnet –env MYSQL_ROOT_PASSWORD=collab123 –env MYSQL_DATABASE=wordpress mysql:latest

Inspecting the service “collabdb1”:

[wpsm_comparison_table id="16" class=""]

Displaying the output in human-friendly way:

[wpsm_comparison_table id="17" class=""]

To list out the VIPs details for specific service:

[wpsm_comparison_table id="18" class=""]

Retrieving the last StatusUpdate details:

[wpsm_comparison_table id="19" class=""]

I have plans to add more examples during the course of time based on experience and exploration. I hope you will find it handy.

 

 
0
0

Running Apache JMeter 3.1 Distributed Load Testing Tool using Docker Compose v3.1 on Swarm Mode Cluster

Apache JMeter is a popular open source software used as a load testing tool for analyzing and measuring the performance of web application or multitude of services. It is 100% pure Java application, designed to test performance both on static and dynamic resources and web dynamic applications. It simulates a heavy load on one or multitude of servers, network or object to test its strength/capability or to analyze overall performance under different load types. The various applications/server/protocol includes – HTTP, HTTPS (Java, NodeJS, PHP, ASP.NET), SOAP / REST Web services, FTP,Database via JDBC, LDAP, Message-oriented middleware (MOM) via JMS,Mail – SMTP(S), POP3(S) and IMAP(S), native commands or shell scripts, TCP, Java Objects and many more.

Logo

JMeter is extremely easy to use and doesn’t take time to get familiar with it.It allows concurrent and simultaneous sampling of different functions by a separate thread group. JMeter is highly extensible which means you can write your own tests. JMeter also supports visualization plugins allow you extend your testing.JMeter supports many testing strategies such as Load Testing, Distributed Testing, and Functional Testing. JMeter can simulate multiple users with concurrent threads, create a heavy load against web application under test.
 
 

Arch

 
 
Architecture of JMeter Distributed Load Testing:
 
JMeter Distributed Load Testing uses client-server model. It is a kind of testing which use multiple systems to perform stress testing. Distributed testing is applied for testing web sites and server applications when they are working with multiple clients simultaneously.
It includes:
 
1. Master –  the system running JMeter GUI, which controls the test.
2. Slave –  the system running JMeter-server, which takes commands from the GUI and send requests to the target system(s).
3. Target (System Under Test)– the web server which undergoes stress test.

 

What Docker has to offer for Apache JMeter?

Good Question ! With every new installation of Apache JMeter, you need to download/install JMeter on every new node participating in Distributed Load Testing. Installing JMeter requires dependencies to be installed first like JAVA, default-jre-headless, iputils etc. The complexity begins when you have multitude OS distributions running on your infrastructure. You can always use automation tools or scripts to enable this solution but again there is an extra cost of maintenance and skills required to troubleshoot with the logs when something goes wrong in the middle of the testing.

With Docker, it is just a matter of 1 Docker Compose file and an one-liner command to get the entire infrastructure ready. Under this blog, I am going to show how a single Docker Compose v3.1 file format can help you setup the entire JMeter Distributed Load Testing tool – all working on Docker 17.03 Swarm Mode cluster.

I will leverage 4-node Docker 17.04 Swarm Cluster running on my Google Cloud Engine platform to showcase this solution as shown below:

gcloud

Under this setup, I will be using instance “master101” as master/client node while the rest of worker nodes as “server/slaves” nodes. All of these instances are running Ubuntu 17.04 with Docker 17.03 installed. You are free to choose any latest Linux distributions which supports Docker executables.

First, let us ensure that you have the latest Docker 17.03 running on your machine:

$curl -sSL https://get.docker.com/ | sh

Version

 

Ensure that the latest Docker compose is installed which supports v3.0 file format:

 $curl -L https://github.com/docker/compose/releases/download/1.11.2/docker-compose-`uname -s`-`uname -m` > /usr/local 
   /bin/docker-compose
 $chmod +x /usr/local/bin/docker-compose

Compose_Version

Docker Compose for Apache JMeter Distributed Load Testing

One can use docker stack deploy command to quickly setup JMeter Distributed Testing environment. This command requires docker-compose.yml file which holds the declaration of services (apache-jmeter-master and apache-jmeter-server) respectively. Let us clone the repository as shown below to get started –

Run the below commands on the Swarm Master node –

$git clone https://github.com/ajeetraina/jmeter-docker/

$cd jmeter-docker

Under this directory, you will find docker-compose.yml file which is shown below:

Picture1

 

In the above docker-compose.yml, there are two service definitions – master and server. Through this docker compose file format, we can push master/client service definition to the master node and server specific service definition which has to be pushed to all the slave nodes. The constraints sections takes care of this functionality. This compose file will automatically create an overlay network called jm-network across the cluster.

Let us start the required services out of the docker-compose file as shown below:

$sudo docker stack deploy -c docker-compose.yml myjmeter

Untitled picture

Checking if the services are up and running through docker service ls command:

service_ls

Let us verify if constraints worked correctly or not as shown below:

1_service

2_service

As shown above, the constraints worked well pushing the containers to the right node.

It’s time to enter into one of the container running on the master node as shown below:

exec_cont

Using docker exec command, one can enter into the container and browse to /jmeter/apache-jmeter-3.1/bin directory.

exec_cont2

Pushing the JMX file into the container

   $docker exec -i <container-running-on-master-node> sh -c 'cat > /jmeter/apache-jmeter-3.1/bin/jmeter-docker.jmx' < jmeter-docker.jmx

Starting the Load testing

  $docker exec -it <container-on-master-node> bash
  root@daf39e596b93:/#cd /jmeter/apache-jmeter-3.1/bin
  $./jmeter -n -t jmeter-docker.jmx -R<list of containers running on slave nodes seperated by commas)

Planning to move from Apache JMeter running on VMs to Containers??

This is very important topic which I don’t want to miss out , though we are at the end of this blog post. You might be using Virtual Infrastructure for setting up Apache JMeter Distributed Load tool. Generally, each  VM is assigned with static IP and it does make sense as your System Under Test will see load coming from different static IP. The above docker-compose file uses an overlay network and IPs are in the range of 172.16.x.x which gets assigned automatically and taken care by Docker  Swarm Networking. In case you are looking out for static IP which should get assigned to each containers automatically, then for sure you need macvlan to be implemented. Follow https://github.com/ajeetraina/jmeter-docker/blob/master/static-README.md to achieve this.

In my next blog post, I will cover further interesting stuffs related to JMeter Distributed Load Testing tool. Drop me a message through tweet (@ajeetsraina) if you face any issue with the above solution.

Loved reading it? Feel free to share it.

[clickandtweet handle=”@ajeetsraina” hashtag=”#Jmeter” related=”@docker” layout=”” position=””]Running Apache JMeter 3.1 Distributed Load Testing via Docker Compose on Swarm Mode[/clickandtweet]

References:

https://github.com/ajeetraina/jmeter-docker

http://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.pdf

https://docs.docker.com/compose/compose-file/

 

0
0

Dockercon 2017 surely gonna be EPIC | Top Sessions Which You Can’t Miss to Attend This Year..

Are you still thinking whether or not to attend Dockercon 2017? Still finding it difficult to convince yourself or your boss/manager to allow you to attend this conference? Then trust me, you have come to the right place. For the next 30 minutes, I will talk about the great sessions which you can’t miss to attend this year.

wordle 10

Dockercon 2017 is just 1 month away. Heavily power-packed with 3 keynotes( includes Solomon Hykes impressive talk), 7 tracks, 60+ breakout sessions, workshops, Ask the Experts, Birds-of-a-feather, Hands-on Lab, Ecosystem expo and lot more.. this year DockerCon 2017 brings a three-day impressive event schedule in capital of the U.S. state of Texas, Austin.Featuring topics, contents & workshops covering all aspects of Docker and it’s ecosystem,Dockercon has always given a chance to meet and talk to like-minded professionals, get familiar about the latest offerings, upcoming Docker releases & roadmap, best practices and solutions for building Docker based applications. Equally it has always provided opportunity to the community users to know what and how are they using Docker in their premises and in the Cloud.

Untitled picture

                     April 17-21 2017 | Austin, TX | DockerCon 2017

Dockercon 2017 is primarily targeted for Developers, DevOps, Ops, System Administrators, Product Manager and IT executives. Whether you are Enablement Solution Architect for DevOps and containers, OR Technical Solution Architect; whether you are part of IoT Development Team OR AWS/Azure DevOps Engineer; whether you are Principal Product Engineer OR Product Marketing Manager, Dockercon is the place to be. Still wondering how would this conference help your organization in adopting containers and improving your offerings in terms of containerized application for your customer? I have categorized the list of topics based on the target audience. Hope it will help you gather data points to convince yourself and your boss.

As a developer, you are a core piece of your organization, busy developing new versions of your flagship software meant to run your software in various platforms. You are responsible for developments leveraging the target containerized platform’s capabilities and adapting and maintaining release artifacts to deliver a compelling experience for your users.Below lists of sessions  might help you to develop the better containerized software –

con_dev

As a Product Manager, you are actually CEO of your product and responsible for the strategy, roadmap, and feature definition for that product or product line. You love to focus on the problems, not on the solutions. You are gifted to excel at getting prospects and customers to express their true needs. Below list of the sessions might interest you to attend:

 con_2

 

As a system administrator, you are the only person who is responsible for the uptime, performance, resources, security, configuration, and reliable operation of systems running Docker applications . Below sessions might interest you to manage your Dockerized environment in a better way –

con_5

 

As a Solution Architect, you are always busy with definition and implementation of reference architectures, capturing business capabilities and transform them into services leveraged across the platform and not to miss out – designing infrastructures for critical applications and business processes in a cost effective manner. Below lists might interest you to shape your containerized solutions in a better way:

  dev_architect

Don’t you think attending Dockercon gonna be a great investment for you and your career?If yes, then what are you waiting for? Docker Team has something really cool for you to get started  –

DockerCon Agenda Builder – Browse and Search Your Session

Register for Dockercon 2017

Dockercon Speakers at a glance

For more information, visit http://2017.dockercon.com/about/

 

0
0

Introducing new RexRay 0.8 with Docker 17.03 Managed Plugin System for Persistent Storage on Cloud Platforms

DellEMC Rex-Ray 0.8 Final Release was announced last week. Graduated as top-level project within {code} community, RexRay 0.8 release has been considered as one of the largest releases till date. The new release introduced support for long lists of new storage platforms like S3FS, EBS, EFS, GCEPD & ScaleIO shown below:

rexray010

Public cloud storage is one of the fastest growing sector in storage with leaders like Amazon AWS, Google Cloud Storage and Microsoft Azure. With the release of RexRay 0.8, {code} community took the right approach in targeting the first community-contributed driver starting with Amazon EFS driver and then quickly adding additional community-contributed drivers like Digital Ocean, FittedCloud, Google Cloud Engine (GCEPD) & Microsoft Azure Unmanaged Disk driver.

Introducing New Docker 17.03 Volume Plugin System

With Docker 17.03 release, a new managed plugin system has been introduced. This is quite different from the old Docker plugin system. Plugins are now distributed as Docker images and can be hosted on Docker Hub or on a private registry.A volume plugin enables Docker volumes to persist across multiple Docker hosts.

rex01

In case you are very new to Docker Plugins, they basically extend Docker’s functionality.  A plugin is a process running on the same or a different host as the docker daemon, which registers itself by placing a file on the same docker host in one of the plugin directories described either as a.sock files( UNIX domain sockets placed under /run/docker/plugins), .spec files( text files containing a URL, such as unix:///other.sock or tcp://localhost:8080 placed under /etc/docker/plugins or /usr/lib/docker/plugins)or.json files ( text files containing a full json specification for the plugin placed under /etc/docker/plugin). You can refer this in case you want to develop your own Docker Volume Plugin.

Running RexRay inside Docker container

Yes, you read it correct ! With the introduction of Docker 17.03 New Plugin Managed System, you can now run RexRay inside Docker container flawlessly. Rex-Ray Volume Plugin is written in Go and provides advanced storage functionality for many platforms including VirtualBox, EC2, Google Compute Engine, OpenStack, and EMC.

You can list out available Docker Volume Plugin for various storage platforms using docker search rexray as shown below:

rexray_plugin2

Let us test-drive RexRay Volume Plugin for Swarm Mode cluster for the first time. I have 4-node Swarm Mode cluster running on Google Cloud Platform as shown below:

rex1

Verify that all the cluster nodes are running the latest 17.03.0-ce (Community Edition).

Installing the RexRay Volume plugin is just one-liner command:

rexray_plugininstallation

 

You can inspect the Rex-Ray Volume plugin using docker plugin inspect command:

rexray_inspect    rexray_inspect22

It’s time to create a volume using docker volume create utility :

$sudo docker volume create –driver rexray/gcepd –name storage1 –opt=size=32

rexray_volume1

You can verify if it is visible under GCE Console window:

RexRay_GCE

Let us try running few application which uses RexRay volume plugin as shown:

{code}==>docker run -dit –name mydb -e MYSQL_ROOT_PASSWORD=wordpress -e MYSQL_USER=wordpress -e MYSQL_PASSWORD:wordpress –volume-driver=rexray/gcepd -v dbdata:/var/lib/mysql mysql:5.7

Verify that MySQL service is up and running using docker logs <container-id> command as shown below:

dbdata1

By now, we should be able to see new volume called “dbdata” created and shown under docker volume ls command:

rexray_dbdata

Under GCE console, it should get displayed too:

dbdata_gce

Using Rex-Ray Volume Plugin under Docker 17.03 Swarm Mode

This is the most interesting section of this blog post. RexRay volume plugin worked great for us till now, especially for a single Docker host running multiple number of services.But what if I want to enable RexRay Volume to persist across multiple Docker Hosts(Swarm Mode cluster)? Yes, there is one possible way to achieve this – using Swarm Executor. It executes docker command across the swarm cluster. Credits to Madhu Venugopal @ Docker Team for assisting me testing with this tool.

swarmexec_1

Please remember that this is UNOFFICIAL way of achieving Volume Plugin implementation across swarm cluster.  I found this tool really cool and hope that it gets integrated within Docker official repository.

First, we need to clone this repository:

$git clone https://github.com/mavenugo/swarm-exec

Run the below command to push the plugin across the swarm cluster:

$cd swarm-exec

$./swarm-exec.sh docker plugin install –grant-all-permissions rexray/gcepd GCEPD_TAG=rexray

swarm_exec01

First, let’s quickly  verify the plugin on the master node  as shown below:

swarm_exec2

While verifying it on the worker nodes:

swarm_exec3

rexray_gce11

The docker volume inspect <volname> should display this particular volume as rexray volume driver as shown below:

rexray_gce222

Creating a MySQL service which uses Rex-Ray volume under Swarm Mode cluster:

up_service

Verifying that the service is up and running:

rexray_100

To conclude, the new version of RexRay looks promising and brings support for various Cloud storage platform. It continues to be leading open source container orchestration engine and now with inclusion of Docker 17.03 Managed Plugin architecture, it will definitely reduce the pain for implementing persistent storage solution.

0
0

Docker Compose v3.1 file format now supports Docker 1.13.1 Secret Management

Docker Engine 1.13.1 went GA last week and introduced one of the most awaited feature called Secrets Management . With a mission to introduce a container native solution that strengthens the Trusted Delivery component of container security, new Secrets API is rightly integrated into Docker 1.13.1 Orchestration engine.The new secrets-management capabilities are also included in Docker Datacenter as part of the Docker 1.13.1 release.

docker secrets

What are secrets all about?

It is a blob of data, such as password, SSH private keys, certificates,API keys, and encryption keys etc..In broader term, it can be anything that can be tightly control access to.The secrets-management capability is the latest security enhancement integrated into the Docker platform so as to ensure applications are safer in a containerized environment.This is going to benefit financial sector players who look for hybrid cloud security strategy.

 

Why do we need Docker secrets?

There has been numerous concerns over environmental variables which are being used to pass configuration & settings to the containers.Environmental variables are easily leaked when debugging and exposed into many places including child processes, hosting secrets on a server etc.

Consider a Docker compose file for WordPress application:

wordpress:
image: wordpressapp
links:
– mariadb:mysql
environment:
– WORDPRESS_DB_PASSWORD=<password>
ports:
– “80:80”
volumes:
– ./code:/code
– ./html:/var/www/html

As shown above, environmental variables are insecure in nature because they are accessible by any process in the container, preserved in intermediate layers of an image, easily accessible through docker inspect and lastly, it can get shared with any container linked to the container. To overcome this, one can use secrets to manage any sensitive data which a container needs at runtime aand there is no need to store in the image . A  given secret is only accessible to those services which have been granted explicit access to it, and only while those service tasks are running.

How does it actually work?

Docker secrets is currently supported for Swarm mode only starting Docker Engine 1.13.1. If you are using Docker 1.12.x you might need to upgrade to the latest 1.13.x release to use this feature. To understand how secret works under Docker Swarm mode, you can follow the below process flow:

 

secret_compose

 

Docker Compose v3.1 File Format now supports Secrets

Docker compose file format v3.1 is available and requires Docker Engine 1.13.0+. It introduced support for secrets for the first time which means that now you can use secrets inside your docker-compose file.

compose_matrix

Let us test-drive Compose v3.1 file format to see how secrets can be implemented using the newer docker stack deploy utility as shown below:

Ensure that you have the latest Docker 1.13.1 running on your Swarm Mode cluster:

secret_0

I will leverage 4-node Swarm Mode cluster to test the secret API:

secret_1

 

Let us first create a secret using docker secret create utility as shown:

$date | md5sum | docker secret create collab_mysqlpasswd –
72flsq9lhuj8je20y7bzfxyld

ollab# date | md5sum | docker secret create collab_mysqlrootpasswd –
1g329zm35umunim61r8q49res

collab# date | md5sum | docker secret create collab_wordpressdbpasswd –
tfxq1bm2cn54he03uzdaar91i

 

Listing the secret using the below command:

secret_101

Create a docker-compose.yml file with the below entry:

PLEASE NOTE: No Compose binaries are required to run the below command. All you require is Compose v3.1 file format for this to work.

compose-1compose-2compose-3

You can copy the whole code from here

Let us now use docker stack deploy to build up the services containing secrets:

$docker stack deploy –compose-file=./docker-compose.yml collab

Updating service collab_mysql (id: yn9fqojgmtmzukqnn3tfa6wlk)

Updating service collab_web (id: xw7kx49sqrkaqriikm5lsbqmj)

Verify the services are up and running:

collab_5

Let us verify if secret got stored under every container:

root@master101:/collab# docker exec -it 35f cat /run/secrets/mysqlpasswd
050a58c339431a5c9a6a6b8a15bead91  –

As shown above, one can use docker exec to connect to the container and read the contents of the secret data file, which defaults to being readable by all and has the same name as the name of the secret.

Key Takeaways:

  • Docker secrets are only available to swarm services, not to standalone containers. To use this feature, consider adapting your container to run as a service with a scale of 1.
  • No Compose binaries are required to run docker stack deploy. All you require is Compose v3.1 file format for this to work.
  • Raft data is encrypted in Docker 1.13 and higher.
  • It is recommended to update all of your manager nodes to Docker 1.13 to prevent secrets from being written to plain-text Raft logs.

 

0
0

Docker For Mac 1.13.0 brings support for macOS Sierra, now runs ARM & AARCH64 based Docker containers

Recently I purchased a brand new slim 13.3 inch Apple Mac Book  Air with an amazing 1.6GHz dual-core Intel Core i5 processor. Introducing Siri to newly re-branded macOS for the first time along with dozens of new features, it came by default running macOS 10.12.1 Sierra. macOS Sierra is the 13th major release of macOS(previously OS X)and successor of OS X El Capiton. ICYMI – Apple released macOS 10.12 Sierra open source Darwin code last November and can be accessed via https://opensource.apple.com/release/os-x-1012.html . One of the first thing I wanted to try out was to see how easy is it to bring Docker 1.13.0 up and running on this system. Trust me, it was an amazing experience. Under this blog post, I am going to share my experience with Docker 1.13.0 and what you really need to know about Docker for Mac on macOS Sierra.

16487451_10209801703852550_7429605017657917198_o

In case you are too new to Docker For Mac…

Last year during March time frame, Docker announced and released a native beta support for Mac and Windows, rightly termed as “Docker for Mac” & “Docker for Windows” respectively. They started with a closed beta & provided access to a couple of early adopters. During Dockercon 2016, they announced Final GA release for both the platforms.

Docker for Mac Vs. Docker Toolbox

difference_between_MacOSToolbox_D4M

 

Pre-Requisite:

  1. Docker for Mac only works on OS X 10.11 or newer macOS release.
  2. Apple Mac must be running 2010 or newer Mac system, with Intel HW support for MMU Virtualization.
  3. Docker for Mac require atleast 4GB of RAM to function smoothly.

Currently the installer comes in the form of 2 channels – stable and Beta. Under the stable channel, the installer is fully tested and comes with the latest GA version of Docker Engine. The Experimental feature is enabled by default.Under Beta channel, the installer provides the latest beta release of Docker for Mac with experimental features enabled by default.

Features enablement under Docker 1.13.0:

Features_1130

Getting Started with Docker Engine 1.13.0 on macOS Sierra

Installing Docker for Mac is one of fantastic experience I ever had installing any software. Just 3 simple steps:

  • Download Docker for Mac by clicking on this link and double-click Docker.dmg which opens up the installer.
  • Drag Moby the whale to the Application Folder as shown below:

Screen Shot 2017-01-29 at 2.52.53 PM

 

3.  Authorize Docker.app with your system password and double-click Docker.app to get started as shown below:

Screen Shot 2017-01-29 at 2.59.02 PM

Screen Shot 2017-01-29 at 2.59.12 PM

Screen Shot 2017-01-29 at 2.59.30 PM

 

Now this is really amazing. Docker for Mac comes with the default availability of docker-compose, docker-machine and experimental feature by default as shown below:

Screen Shot 2017-01-29 at 3.00.25 PM

  1. Verify the docker compose version:

bash-3.2# docker-compose version
docker-compose version 1.10.0, build 4bd6f1a
docker-py version: 2.0.1
CPython version: 2.7.12
OpenSSL version: OpenSSL 1.0.2j 26 Sep 2016

2. Verify the docker-machine version:

bash-3.2# docker-machine version

docker-machine version 0.9.0, build 15fd4c7

Kudos to Docker Engineers, the whale in the status bar is all you need to have a glimpse of overall Docker daemon running and easy way to configure Docker preferences & environment as per your need:

Screen Shot 2017-01-29 at 3.03.25 PM

Open the terminal and you can see that Docker for Mac runs on top of Alpine Linux v3.5, default storage driver as overlay2, Plugins support and many more.

Screen Shot 2017-01-29 at 6.04.50 PM

Multi-CPU Architecture Support(ARM, AARCH64 & PPC64le):

Apple Inc. added support for an ARM chip to the latest macOS Sierra 10.12 kernel. At the right time, Docker for Mac made binfmt_misc multi architecture support available,which means that now you can run containers for different Linux architectures, such as arm, aarch64 ppc64le and even s390x.

I couldn’t wait to test-drive few of Raspberry Pi ARM-based Docker images on my Mac Book Air. Here is how I tested few of ARM -based images:

bash-3.2# docker run -itd ajeetraina/centos7-arm /bin/bash

Unable to find image ‘ajeetraina/centos7-arm:latest’ locally

latest: Pulling from ajeetraina/centos7-arm

5584ea4c92c5: Pull complete

a3ed95caeb02: Pull complete

Digest: sha256:10989b1b7a3bdf826857ba5b3348956d495b9be1ced644a3aa7321cfbd705b04

Status: Downloaded newer image for ajeetraina/centos7-arm:latest

a719fb80aac98c2f2daddd7c5374bb2739d03285e74f7523cb0cc36258fc605e

bash-3.2# docker ps

CONTAINER ID        IMAGE                    COMMAND             CREATED             STATUS              PORTS               NAMES

a719fb80aac9        ajeetraina/centos7-arm   “/bin/bash”         2 minutes ago       Up 2 minutes                            suspicious_goldwasser

bash-3.2# docker exec -it a719f cat /etc/issue\S

Kernel \r on an \m

bash-3.2#

Please Note – The containers need to have the appropriate qemu binary inside the container to work, without qemu you can pull ARM-based Docker images but can’t run it.

Filesystem Sharing with OSXFS:

Docker for Mac brings a new shared filesystem, rightly called OSXFS  solution . OSXFS provides a close-to-native user experience for bind mounting OS X file system trees into Docker containers. It brings a number of unique capabilities as well as differences from a classical Linux file system. It does not use OSXFUSE and doesn’t run under, inside, or between OS X user space processes and the OS X kernel.

There are number of concerns around performance degradation around OSXFS and I think Docker team has rightly addressed it. It is worth reading  https://docs.docker.com/docker-for-mac/osxfs/#/performance-issues-solutions-and-roadmap to understand the problem statement and taking the right step. Said that, Docker team is continuously working to bring improvement under this space.

In the future blog post, I will talk about newer features and bug fixes which comes with Engine 1.13.0. Feel free to share if you like and reach out to me via @ajeetsraina(t).

 

0
0

Test-Drive Docker 1.12 on first 64-bit ARM OpenSUSE running on Raspberry Pi 3

Raspberry Pi 3 Model B is the first 64 bit version and the third generation Pi box which runs on 1.2GHz 64 bit quad-core ARMv8 CPU.(Broadcom BCM2837 A53 ARM processor). Despite its processor upgrade, there wasn’t an official 64-bit OS available for it till the first week of Jan 2017. Kudos to SUSE Team, they came up providing first commercial enterprise Linux distribution optimized for ARM AARCH64 servers. This is definitely a BIG news. Reason –  To build a solution to meet specific market needs while maintaining a common code base. Enterprise vendors & customers demanding workload-optimized server platforms can now radically expand it for their modern data centers.

sles22

In the last couple of months, Docker enthusiasts have been working hard to get Docker running on ARM 32bit systems (like Raspberry Pi). With Docker Engine 1.12.1, a FIRST ARM Debian package was officially made available by Docker Inc. which happened late last year. This year, SUSE Team did a great job in bringing capabilities of SUSE Linux(a.k.a SLES for ARM) to the ARM AArch64 hardware platform. This is BIG news for Docker community too as more innovation and development is expected to grow building containers which will run across the  AARCH64 platform.

Under this blog, I am going to test drive Docker 1.12.3 on first 64-bit ARM Open SUSE distribution running on Raspberry Pi 3 box.

Hardware:

  1. Raspberry Pi 3 ( You can order it from Amazon in case you are in India for $35)
  2. Micro-SD card reader ( I got it from here )
  3. Any Windows or Linux Desktop or Laptop
  4. HDMI cable ( I used the HDMI cable of my plasma TV)
  5. Internet Connectivity(WiFi/Broadband/Tethering using Mobile) – to download Docker 1.12.3 package
  6. Keyboard & mouse connected to Pi’s USB ports

1

                                                                                            Source~http://www.rlocman.ru/i/Image/2016/02/29/RaspberryPi_3_1.jpg

Software:

  1. SD-Formatter – to format microSD card
  2. Rufus(in case you have Windows OS running on your laptop) – to burn OpenSUSE Leap 42.2 ARM XZ format directly into microSD card.(No need to extract XZ using any tool)

Steps:

  1. Format the microSD card using SD Formatter as shown below:

4

2. Download OpenSUSE Leap 42.2 ARM  OS from here and use Win32 imager(in case you are on Windows OS  running on your laptop) to burn it on microSD card.

rufus

3. Insert the microSD card into your Pi box. Now connect the HDMI cable  from one end of Pi’s HDMI slot to your TV or display unit and mobile charger(recommended 5.1V@1.5A) as shown:


23

4.You will see the fancy GUI coming up on your display:

sles2

5. To enable VNC, run the below command:

linux:~ # zypper install xorg-x11-Xvnc

linux:~# vncserver

Installing Docker 1.12.3 on first 64-bit ARM OpenSUSE

6. Run the below command:

$curl -sSLk https://get.docker.com/ | sh

sles4

7. Verify the docker version:

sles9

8. Let us try searching for Docker images present in Dockerhub based on aarch64:

sles10

Please be aware that the usual Docker containers are not going to work for this architecture. You need to pick up AARCH64 bit Docker images to run containers.

linux:~ # docker run busybox
Unable to find image ‘busybox:latest’ locally
latest: Pulling from library/busybox

557a0c95bfcd: Pull complete
Digest: sha256:ae007bdb45fc0d56e3d705b97640ac24844bcc9ce4c8b8493f216a57ab6af0d5
Status: Downloaded newer image for busybox:latest
panic: standard_init_linux.go:175: exec user process caused “exec format error” [recovered]
panic: standard_init_linux.go:175: exec user process caused “exec format error”

goroutine 1 [running, locked to thread]:
panic(0x7bcaa0, 0x4820161ce0)

sles13

Installing Docker 1.13 on 64-bit ARM OS:

Docker doesn’t officially support AARCH64 system. You can’t use one-liner command to install Engine 1.13 as shown below:

Untitled picture

If you really want to try 1.13, the easier way is downloading it from the below link:

sles25

Installing docker-compose

linux:~ # zypper install python-pip

linux:~ # pip install docker-compose

sles19
Tested docker-compose v2.0 functionality by bringing up microservices and it went fine.

sles66

Testing Swarm Mode:

sles100

sles90

linux:~/collabnix # docker service ls
ID            NAME        REPLICAS  IMAGE          COMMAND
0x7dlmr4pjm7  backend-db  1/1       aarch64/redis

linux:~/collabnix # docker service ps backend-db
ID                         NAME          IMAGE          NODE   DESIRED STATE  CURRENT STATE                 ERROR
7jmkwcbzq8wix3umfkcqc92rw  backend-db.1  aarch64/redis  linux  Running        Preparing about a minute ago

 

0
0