How to create a Local Private Docker Registry on Play with Docker in 5 Minutes?

Estimated Reading Time: 4 minutes

DockerHub is a service provided by Docker for finding and sharing container images with your team. It is the world’s largest repository of container images with an array of content sources including container community developers, open source projects and independent software vendors (ISV) building and distributing their code in containers. Users get access to free public repositories for storing and sharing images or can choose subscription plan for private repos.

One can easily pull the Docker Image from Dockerhub and run application in their environment flawlessly. But in case you want to setup a private registry, it is still possible to accomplish. Under this blog post, I will demonstrate how to build a private registry on Play with Docker in just 5 minutes.

Caution – Please note that Play with Docker platform is just for demo or training purpose. As the instance wipe away after 4 hours of limited time, it’s better to save your data before it gets wiped up.

Tested Infrastructure

PlatformNumber of InstanceReading Time
Play with Docker15 min

Pre-requisite

  • Create an account with DockerHub
  • Open PWD Platform on your browser
  • Click on Add New Instance on the left side of the screen to bring up Alpine OS instance on the right side

Create a directory to permanently store images.

$ mkdir -p /opt/registry/data

Authenticate with DockerHub

$docker login

Start the registry container.

$ docker run -d \
  -p 5000:5000 \
  --name registry \
  -v /opt/registry/data:/var/lib/registry \
  --restart always \
  registry:2

Display running containers.

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS
                    NAMES
3a056bf96c6d        registry:2          "/entrypoint.sh /etc…"   About an hour ago   Up About an hour    0.0.0
.0:5000->5000/tcp   registry

Pull Debian Stretch image from official repository.

$ docker pull debian:stretch

stretch: Pulling from library/debian
723254a2c089: Pull complete
Digest: sha256:0a5fcee6f52d5170f557ee2447d7a10a5bdcf715dd7f0250be0b678c556a501b
Status: Downloaded newer image for debian:stretch

Tag local Debian Stretch image with an additional tag – local repository address.

$ docker tag debian:stretch localhost:5000/debian:stretch

Push image to the local repository.

[node1] (local) root@192.168.0.23 ~
$ docker push localhost:5000/debian:stretch
The push refers to repository [localhost:5000/debian]
90d1009ce6fe: Pushed
stretch: digest: sha256:38236c068c393272ad02db100e09cac36a5465149e2924a035ee60d6c60c38fe size: 529

Remove local images.

[node1] (local) root@192.168.0.23 ~
$ docker image remove debian:stretch
Untagged: debian:stretch
Untagged: debian@sha256:df6ebd5e9c87d0d7381360209f3a05c62981b5c2a3ec94228da4082ba07c4f05
[node1] (local) root@192.168.0.23 ~
$ docker image remove localhost:5000/debian:stretch
Untagged: localhost:5000/debian:stretch
Untagged: localhost:5000/debian@sha256:38236c068c393272ad02db100e09cac36a5465149e2924a035ee60d6c60c38fe
Deleted: sha256:4879790bd60d439cfe39c063660eef7af525d5f6f1cbb701a14c7cfc11cbfcf7

Pull Debian Stretch image from local repository.

[node1] (local) root@192.168.0.23 ~
$ docker pull localhost:5000/debian:stretch
stretch: Pulling from debian
54f7e8ac135a: Pull complete
Digest: sha256:38236c068c393272ad02db100e09cac36a5465149e2924a035ee60d6c60c38fe
Status: Downloaded newer image for localhost:5000/debian:stretch

List stored images.

[node1] (local) root@192.168.0.23 ~
$ docker image ls
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
localhost:5000/debian   stretch             4879790bd60d        12 days ago         101MB
registry                2                   2e2f252f3c88        2 months ago        33.3MB

Shared local registry

Create a directory to permanently store images.

$ mkdir -p /srv/registry/data

Create a directory to permanently store certificates and authentication data.

$ mkdir -p /srv/registry/security

Store domain and intermediate certificates using /srv/registry/security/registry.crt file, private key using /srv/registry/security/registry.key file. Use valid certificate and do not waste time with self-signed one. This step is required do use basic authentication.

Install apache2-utils to use htpasswd utility.

[node1] (local) root@192.168.0.23 ~
$ apk add apache2-utils
OK: 302 MiB in 110 packages

Create initial username and password. The only supported password format is bcrypt.

$ : | sudo tee /srv/registry/security/htpasswd
[node1] (local) root@192.168.0.23 ~
$ echo "password" | sudo htpasswd -iB /srv/registry/security/htpasswd username
Adding password for user username

Adding password for user username

$(local) root@192.168.0.23 ~ $ cat /srv/registry/security/htpasswd username:$2y$05$q9R5FSNYpAppB4Vw/AGWb.RqMCGE8DmZ4q5HZC/1wC87oTWyvB9vy 

Stop and Remove all old containers

$ docker stop $(docker ps -a -q)
3a056bf96c6d
$ docker rm -f $(docker ps -a -q) 
3a056bf96c6d 

Start the registry container.

[node1] (local) root@192.168.0.23 ~
$ docker run -d   -p 443:5000   --name registry   -v /srv/registry/data:/var/lib/registry   -v /srv/registry/security:/etc/security   -e REGISTRY_HTTP_TLS_CERTIFICATE=/etc/security/registry.crt   -e REGISTRY_HTTP_TLS_KEY=/etc/security/registry.key   -e REGISTRY_AUTH=htpasswd   -e REGISTRY_AUTH_HTPASSWD_PATH=/etc/security/htpasswd   -e REGISTRY_AUTH_HTPASSWD_REALM="Registry Realm"   --restart always   registry:2
e7755af8cbd70ea84ab77237a87cb97fd1abb18c7726fbc116c40f081d3b7098

Display running containers.

[node1] (local) root@192.168.0.23 ~
$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS        PORTS               NAMES
e7755af8cbd7        registry:2          "/entrypoint.sh /etc…"   About a minute ago   Restarting (1) 22 seconds ago                       registry

Pull Debian Stretch image from official repository.

$ docker pull debian:stretch

stretch: Pulling from library/debian
723254a2c089: Pull complete
Digest: sha256:0a5fcee6f52d5170f557ee2447d7a10a5bdcf715dd7f0250be0b678c556a501b
Status: Downloaded newer image for debian:stretch

Tag local Debian Stretch image with an additional tag – local repository address.

$ docker tag debian:stretch registry.collabnix.com/debian:stretch

This time you need to provide login credentials to use local repository.

$ docker push registry.collabnix.com/debian:stretch

e27a10675c56: Preparing
no basic auth credentials
$ docker pull registry.collabnix.com/debian:stretch

Error response from daemon: Get https://registry.collabnix.com/v2/debian/manifests/stretch: no basic auth credentials

Log in to the local registry.

$ docker login --username username registry.collabnix.com
Password: ********

Login Succeeded

Push image to the local repository.

$ docker push registry.collabnix.com/debian:stretch
The push refers to repository [registry.collabnix.com/debian]
e27a10675c56: Pushed
stretch: digest: sha256:02741df16aee1b81c4aaff4c48d75cc2c308bade918b22679df570c170feef7c size: 529

Remove local images.

$ docker image remove debian:stretch

Untagged: debian:stretch
Untagged: debian@sha256:0a5fcee6f52d5170f557ee2447d7a10a5bdcf715dd7f0250be0b678c556a501b
$ docker image remove registry.collabnix.com/debian:stretch

Untagged: registry.collabnix.com/debian:stretch
Untagged: registry.sl.collabnix.com/debian@sha256:02741df16aee1b81c4aaff4c48d75cc2c308bade918b22679df570c170feef7c
Deleted: sha256:da653cee0545dfbe3c1864ab3ce782805603356a9cc712acc7b3100d9932fa5e
Deleted: sha256:e27a10675c5656bafb7bfa9e4631e871499af0a5ddfda3cebc0ac401dfe19382

Pull Debian Stretch image from local repository.

$ docker pull registry.collabnix.com/debian:stretch

stretch: Pulling from debian
723254a2c089: Pull complete
Digest: sha256:02741df16aee1b81c4aaff4c48d75cc2c308bade918b22679df570c170feef7c
Status: Downloaded newer image for registry.collabnix.com/debian:stretch

List stored images.

$ docker image ls

REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
registry                             2                   d1fd7d86a825        4 weeks ago         33.3MB
registry.collabnix.com/debian   stretch             da653cee0545        2 months ago        100MB
hello-world                          latest              f2a91732366c        2 months ago     

How I built Elastic Stack for Docker Swarm using Docker Application Packages(docker-app)

Estimated Reading Time: 6 minutes 

Let’s begin with Problem Statement !

DockerHub is a cloud-based registry service which allows you to link to code repositories, build your images, test them, store manually pushed images so you can deploy images to your hosts. It provides a centralized resource for container image discovery, distribution and change management as well as workflow automation throughout the development pipeline. We share Docker images all the time, but let’s agree to the fact that we don’t have a good way of sharing the multi-service applications that use them.

Let us take an example of Elastic Stack. Built on an open source foundation, the Elastic Stack lets you reliably and securely take data from any source, in any format, and search, analyze, and visualize it in real time with the help of Elasticsearch, Logstash, Kibana and multiple other tools and technique. In order to build these tools in the form of containers, one need to start building Docker Image for each of these tools. The recommended way is constructing a Dockerfile for each of these tools. In turn, Docker Compose uses these images to build required services. Whenever the docker stack deploy CLI is used to deploy the application stack, these Docker images are pulled from Dockerhub for the first time and then picked up locally once downloaded to your system. What if you could upload your whole application stack to DockerHub? Yes, it’s possible today and docker-app is the tool which can make Compose-based applications shareable on Docker Hub and DTR.

Docker-app v0.5.0 is now Available !

Docker Application Package v0.5.0 is the latest offering from  Docker, Inc. You can download it from this link. The binaries are available for Linux, Windows and MacOS Platform. If you are looking out for source code,  this is the direct link.

   

The docker-app v0.5.0 comes with notable features and improvements which are listed below:

  • The improved docker-app inspect command to shows a summary of services, networks, volumes and secrets.

  • The docker-app push CLI now works on Windows and bypasses the local docker daemon by talking directly to the registry.
  • The docker-app save and docker-app ls have been obsoleted.
  • All commands now accept an application package as a URL.
  • The docker-app push command now accepts a custom repository name.
  • The docker-app completion command can generate zsh completion in addition to bash.

In my last blog post, I talked about docker-app for the first time and showcased its usage soon after I returned back from Dockercon. Under this post, I will show how I built Elastic Stack using docker-app for 5-Node Docker Swarm cluster.

Prerequisite:

  • Click on Icon near to Instance to choose 3 Managers & 2 Worker Nodes

Deploy 5 Node Docker Swarm Cluster

$ docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUSENGINE VERSION
iy9mbeduxd4mmjxoikbn5ulds *   manager1            Ready               Active              Reachable18.03.1-ce
mx916kgqg6gfgqdr2gn1eksxy     manager2            Ready               Active              Leader18.03.1-ce
xaeq943o84g9spy6mebj64tw3     manager3            Ready               Active              Reachable18.03.1-ce
8umdv6m82nrpevuris1e45wnq     worker1             Ready               Active18.03.1-ce
o3yobqgg7wjvjw2ec5ythszgw     worker2             Ready               Active18.03.1-ce

Cloning the Repository

$ git clone https://github.com/ajeetraina/app
Cloning into 'app'...remote: Enumerating objects: 134, done.
remote: Counting objects: 100% (134/134), done.remote: Compressing objects: 100% (134/134), done.
remote: Total 14511 (delta 95), reused 0 (delta 0), pack-reused 14377Receiving objects: 100% (14511/14511), 17.37 MiB | 13.35 MiB/s, done.
Resolving deltas: 100% (5391/5391), done.

Install Docker-app

$ cd app/examples/elk/
[manager1] (local) root@192.168.0.30 ~/app/examples/elk$ ls
README.md          devel              elk.dockerapp      install-dockerapp  prod
[manager1] (local) root@192.168.0.30 ~/app/examples/elk
$ chmod +x install-dockerapp[manager1] (local) root@192.168.0.30 ~/app/examples/elk
$ sh install-dockerappConnecting to github.com (192.30.253.112:443)
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (52.216.131.187:443)docker-app-linux.tar 100% |*************************************************************|  8895k  0:00:00 ETA
[manager1] (local) root@192.168.0.30 ~/app/examples/elk

Verifying Docker-app Version

$ docker-app version
Version:      v0.4.0
Git commit:   525d93bc
Built:        Tue Aug 21 13:02:46 2018
OS/Arch:      linux/amd64
Experimental: off
Renderers:    none

I assume you have a docker compose file for ELK stack application already available with you. If not, you can download a sample file from this link. Place this YAML file under the same directory(app/examples/elk/). Now with docker-app installed, let’s create an Application Package based on this Compose file:

$ docker-app init elk

Once you run the above command, it create a new directory elk.dockerapp/ that contains three different YAML files:

docker-compose.yml  elk.dockerapp
[manager1] (local) root@192.168.0.30 ~/myelk
$ tree elk.dockerapp/
elk.dockerapp/
├── docker-compose.yml
├── metadata.yml
└── settings.yml

0 directories, 3 files

Edit each of these files as shown to look similar to what are placed under this link.

Rendering Docker Compose file

$ docker-app render elk
version: "3.4"
services:
  elasticsearch:    command:
    - elasticsearch    - -Enetwork.host=0.0.0.0
    - -Ediscovery.zen.ping.unicast.hosts=elasticsearch
    deploy:
      mode: replicated
      replicas: 2
    environment:
      ES_JAVA_OPTS: -Xms2g -Xmx2g
    image: elasticsearch:5
    networks:
      elk: null
    volumes:
    - type: volume
      target: /usr/share/elasticsearch/data
  kibana:
    deploy:
      mode: replicated
      replicas: 2
    environment:
      ELASTICSEARCH_URL: http://elasticsearch:9200
    healthcheck:
      test:
      - CMD-SHELL
      - wget -qO- http://localhost:5601 > /dev/null
      interval: 30s
      retries: 3
    image: kibana:latest
    networks:
      elk: null
    ports:
    - mode: ingress
      target: 5601
      published: 5601
      protocol: tcp
  logstash:
    command:
    - sh
    - -c
    - logstash -e 'input { syslog  { type => syslog port => 10514   } gelf { } } output
      { stdout { codec => rubydebug } elasticsearch { hosts => [ "elasticsearch" ]
      } }'
    deploy:
      mode: replicated
      replicas: 2
    hostname: logstash
    image: logstash:alpine
    networks:
      elk: null
    ports:
    - mode: ingress
      target: 10514
      published: 10514
      protocol: tcp
    - mode: ingress
      target: 10514
      published: 10514
      protocol: udp
    - mode: ingress
      target: 12201
      published: 12201
      protocol: udp
networks:
  elk: {

Setting the kernel parameter for ELK stack

sysctl -w vm.max_map_count=262144

Deploying the Application Stack


[manager1] (local) root@192.168.0.30 ~/app/examples/elk
$ docker-app deploy elk --settings-files elk.dockerapp/settings.yml
Creating network elk_elk
Creating service elk_kibana
Creating service elk_logstash
Creating service elk_elasticsearch
[manager1] (local) root@192.168.0.30 ~/app/examples/elk
$

Inspecting ELK Stack

[manager1] (local) root@192.168.0.30 ~/app/examples/elk
$ docker-app inspect elk
myelk 0.1.0
Maintained by: Ajeet_Raina <ajeetraina@gmail.com>

ELK using Dockerapp

Setting                       Default
-------                       -------
elasticsearch.deploy.mode     replicated
elasticsearch.deploy.replicas 2
elasticsearch.image.name      elasticsearch:5
kibana.deploy.mode            replicated
kibana.deploy.replicas        2
kibana.image.name             kibana:latest
kibana.port                   5601
logstash.deploy.mode          replicated

Verifying Stack services are up & running

[manager1] (local) root@192.168.0.30 ~/app/examples/elk/docker101/play-with-docker/visualizer
$ docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE               PORTS
uk2whax6f3jq        elk_elasticsearch   replicated          2/2                 elasticsearch:5
nm4p3yswvh5y        elk_kibana          replicated          2/2                 kibana:latest       *:5601->56
01/tcp
g5ubng6rhcyp        elk_logstash        replicated          2/2                 logstash:alpine     *:10514->1
0514/tcp, *:10514->10514/udp, *:12201->12201/udp
[manager1] (local) root@192


Pushing the App Package to Dockerhub

Password:[manager1] (local) root@192.168.0.30 ~/app/examples/elk
$ docker login
Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to
 https://hub.docker.com to create one.
Username: ajeetraina
Password:Login Succeeded

Pushing the App package to DockerHub


[manager1] (local) root@192.168.0.30 ~/app/examples/elk$ docker-app push --namespace ajeetraina --tag 1.0.2
The push refers to repository [docker.io/ajeetraina/elk.dockerapp]
15e73d68a400: Pushed
1.0.2: digest: sha256:c5a8e3b7e2c7a5566a3e4247f8171516033e7e9791dfdb6ebe622d3830884d9b size: 524
[manager1] (local) root@192.168.0.30 ~/app/examples/elk
$

Important Note: If you are using Docker-app v0.5.0, you might face issue related to pulling the image from Dockerhub as it report unsupported OS error message. Here’s a link to this open issue.

Testing the Application Package

Open up a new PWD window. Install docker-app as shown above and try to run the below command:

docker-app deploy ajeetraina/elk.dockerapp:1.0.2

This should bring up your complete Elastic Stack Platform.

Did you find this blog helpful?  Feel free to share your experience. Get in touch with me at twitter @ajeetsraina.

If you want to keep track of latest Docker related information, follow me at https://www.linkedin.com/in/ajeetsraina/.

Docker’s Birthday Celebration in Bangalore – The Fifth Kind

Estimated Reading Time: 5 minutes“As you grow up, make sure you have more networking opportunities than chances, and more collaborative approach than just an acquaintance. Reason  ~ “Sharing is Caring”.

Docker’s Birthday Celebration is not just about cakes, food and party. It’s actually a global tradition that is near and dear to our heart because it gives each one of us an opportunity to express our gratitude to our community of contributors, customers, partners and users. The goal of this global celebration is to welcome those new to Docker and bring them up to speed through hands-on labs with the help of more advanced users who act as mentors on-site.

 

This year, celebrations all over the world took place March 19-25, 2018. Yesterday we celebrated Docker’s 5th Birthday in Bangalore at Altimetrik Office, Electronic City. This time I was lucky enough to get chance to lead this event.

I started planning for this event in the first week of March. Started interlocking with  Community Marketing Co-ordinator, Docker Inc. and ordered for Docker Mentor T-shirts, birthday stickers & Docker crafted Balloons. It just took 1 week time to get it shipped to Bangalore address. Quite quick, isn’t it? A Special thanks to Lisa McNikol for helping me with these stuffs. Next, Neependra Khare, Founder of CloudYuga & Docker Bangalore Meetup organizer initiated a thread for looking out Mentors for the event. Around 10+ community members turned up to mentor the participants for lab sessions. I started interlocking with them for the lab preparation. A BIG SHOUT OUT to each and every mentors (names listed towards the end) as I find them quite interested and were very quick in completing the testing with the lab sessions. Few of the issues reported were fixed and handled smoothly.

Next, it was time to work with the organizers at Altimetric India. A special thanks to Sonali Baluni, Altimetrik Employee for multiple interactions on phone and emails for planning this event. Over 30+ balloons, 5kg Docker Birthday Cake, goodies for organizer and participants, smoothening WiFi connectivity & Backups were planned for the day.

Over 200+ participants signed up while 50+ turned up for this event. This was expected as we just completed Joint Meetup with 8 vibrant groups last weekend in Walmart Office. I reached venue at around 8:30 AM and was stoked to find wonderful ambience in place. Worked with organizers to arrange Docker T-shirts and Balloons for the participants, distributed Mentors T-Shirts so that they can get ready for the celebration.

I started the Birthday celebration with an attention grabber –

 

Talked about the agenda for the day which included more networking, mentorship & lab practice as shown –

Next, I asked each of Docker Mentors to introduce themselves to the group. Mentors are group of experts targeted to help the participants with the lab sessions.

 

Most of the participants followed the instructions on their own and where ever needed, mentors helped the participants. There were around 5 Lab sessions categorised under Docker 101, MTA with Java and .Net applications, Kubernetes and Hybrid OS applications. You can find it under https://github.com/dockersamples/docker-fifth-birthday

 

At 11:30 sharp, we did a cake cutting. We gave chance to youngest member in the group to do this honour.

This time I focused on networking and spent considerable amount of time interlocking with the participants. I met with new Community members and tried to understand how they are using Docker. Few of interesting questions from the community were –

  1. What is Serverless?
  2. How to get started with OpenFaas?
  3. How to become Docker Mentor?
  4. Why we switched to Kubernetes? Issues around Docker EE UCP.
  5. How to containerise 20-years old Monolithic Legacy Application inside my organization?
  6. What is RexRay & its architecture?
  7. When can we expect K8s under Docker CE on Linux Platform?
  8. A Query around AWX – An Alternative to RedHat’s Ansible Tower?
  9. When shall I use Kubernetes & when to use Swarm?

I am planning to come up with blog post around most of the queries in the near future.

Thank You –

  • Mentors (Suraj Narwade, Yogesh Kad, Mamta Jha, Shanthi Kumar, Jibin George, Shankar, Kumara K, Vishan Ghule, Y Ramesh Naik, Savitha Ashture, Saurav Kumar, Swapnasagar Pradhan – For effective Mentorship
  • Mano Marks, Elton & Marcos Nills – For the robust PWD Platform for Lab workshop
  • Sonali Baluni & Team, Organizer ( Altimetrik India)
  • Lisa McNikol – Community Marketing Co-ordinator, Docker Inc.
  • Jenny Burcio – Sr. Program Manager, Developer Relations and Advocacy, Docker Inc.
  • All Participants

What I liked most about this Meetup is the energy and curiosity that the participants have. It was a blast discussing with, answering and questioning all the keen minds that make up this community.

Dockercon 2018 is happening this June 12-15. I have plan to attend this event.  Would be completely excited to meet more members of the Docker community over at DockerCon in June and hope to bring home interesting knowledge and tips & tricks for the larger Docker Bangalore Meetup to gain from.

Thanks to all you there who participated to make this event a blast !