5 Minutes to Run Your First Docker Container on Google Cloud Platform using Terraform

Estimated Reading Time: 11 minutes

 

It’s 2018 ~ Define Your whole Cloud infrastructure via Terraform

I still remember those days(back in 2006-07′) when I started my career as IT Consultant in Telecom R&D centre where I used to administer Subversion & CVS repositories running on diversified Linux platforms. I considered it as a dark age where fear of downtime, fear of accidental misconfiguration and slow network impacted the overall development, testing & go-to-market process.

Thanks to today’s DevOps Era, we now have a better way to do things: Infrastructure-as-Code (IAC). The goal of DevOps is to perform software delivery more efficiently, and we need  tools to make this delivery quick and efficient, this is where the tools like Terraform help companies with infrastructure as code and automation.

Terraform is an open source tool that allows you to define infrastructure for a variety of cloud providers (e.g. AWS, Azure, Google Cloud, DigitalOcean, etc) using a simple, declarative programming language and to deploy and manage that infrastructure using a few CLI commands.Terraform is a tool to Build, Change and Version Control your Infrastructure.

Building Infrastructure includes:

  • Talking to Multiple Cloud/Infrastructure Provider
  • Ensuring Creation & Consistency
  • Express in an API-agnostic DSL

Change Infrastructure includes:

  • Apply Incremental Changes
  • Destroy when needed
  • Preview Changes
  • Scale Easily

Version Control includes:

  • HCL Language( HashiCorp Configuration Language)
  • State File(don’t store it in GitHub Repo)

Wait a sec..I have been using Ansible & Puppet. How is Terraform different from these CM tools?

You might have used technologies like Ansible, Chef, or Puppet to automate and provision the software. Terraform starts from the same law, infrastructure as code, but focuses on the automation of the infrastructure itself. Your whole Cloud infrastructure (instances, volumes, networking, IPs) can be easily defined in terraform.

Chef, Puppet & Ansible are “Configuration management” tools whereas Terraform is actually an orchestration tool. Terraform is designed to provision the servers themselves. Tools like Chef, Puppet, & Ansible typically default to a mutable infrastructure paradigm which means if you tell Puppet to install a new version of Docker, it’ll run the software update on your existing servers and the changes will happen in-place accordingly. If you’re using an orchestration tool such as Terraform to deploy machine images created by Docker or Packer, then every “change” is actually a deployment of a new server (just like every “change” to a variable in functional programming actually returns a new variable). I recommend you to read this,  if you have spare time to deep-dive into use cases around Terraform.

Under this blog post, I will show you how to run your first Docker Web container on Google Cloud Platform using Terraform in just 5 minutes. I will be running the below command under macOS High Sierra v10.13.3.

Installing Terraform on macOS

Installing Terraform on macOS is super easy. You are just a “brew-far”.

[Captains-Bay]🚩 >  brew install terraform
Updating Homebrew...
==> Auto-updated Homebrew!
Updated 4 taps (jenkins-x/jx, homebrew/cask-versions, homebrew/core, homebrew/cask).
==> Updated Formulae
agda                globus-toolkit      pango               shibboleth-sp
bacula-fd           hadolint            pdftoedn            subversion
chronograf          jenkins-x/jx/jx     pdftoipe            tkdiff
conan               json-fortran        php@5.6             triton
convox              jsonnet             php@7.0             vdirsyncer
diff-pdf            kitchen-sync        php@7.1             xapian
fdroidserver        libdill             picard-tools        xml-tooling-c
fn                  lmod                poppler
git-annex           pandoc              rust

Warning: terraform 0.11.7 is already installed and up-to-date
To reinstall 0.11.7, run `brew reinstall terraform`

Clone the Repository

I have put all the required files for Terraform in my docker101 repository. Just clone it and you are ready to go..

[Captains-Bay]🚩 >  git clone https://github.com/ajeetraina/docker101
Cloning into 'docker101'...
remote: Counting objects: 5637, done.
remote: Compressing objects: 100% (155/155), done.
remote: Total 5637 (delta 117), reused 114 (delta 57), pack-reused 5422
Receiving objects: 100% (5637/5637), 17.67 MiB | 432.00 KiB/s, done.
Resolving deltas: 100% (1821/1821), done.

Change directory to Terraform-GCP location

As I am planning to write dozens of articles around Terraform as IaC, I have rightly arranged it under automation/terraform/<platform> folder. You can keep an eye on this repository to learn more from my exploration:

[Captains-Bay]🚩 >  pwd
/Users/ajeetraina/docker101/automation/terraform/googlecloud/building-first-instance
[Captains-Bay]🚩 >  tree
.
├── README.md
├── compute.tf
├── first-docker-container
│   ├── README.md
│   ├── main.tf
│   ├── output.tf
│   ├── terraform-provider-google
│   └── variables.tf
├── google-compute-firewall.tf
├── provider.tf
└── terraform-account.json

2 directories, 9 files
[Captains-Bay]🚩 >

Let us spend some time in understanding the essential concepts of Terraform before we move ahead.

A Quick Look at Terraform Module

Modules in the Terraform ecosystem are a way to organize the code to be more reusable, to avoid code duplication & to improve the code organisation and its readability. By using modules, you will save time because you write the code once, test it and reuse it many times with different parameters.

The below main.tf is the main configuration file for Terraform. It starts with definition of a provider which is responsible for understanding API interactions and exposing resources. Providers generally are an IaaS (e.g. AWS, GCP, Microsoft Azure, OpenStack), PaaS (e.g. Heroku), or SaaS services (e.g. Terraform Enterprise, DNSimple, CloudFlare). The Google Cloud provider is used to interact with Google Cloud services. The provider needs to be configured with the proper credentials before it can be used.We are targeting Google Cloud as our provider, hence the definition look like as shown below:

provider "google" {
  region      = "${var.region}"
  project     = "${var.project_name}"
  credentials = "${file("${var.credentials_file_path}")}"
}


resource "google_compute_instance" "docker" {
  count = 1

  name         = "tf-docker-${count.index}"
  machine_type = "f1-micro"
  zone         = "${var.region_zone}"
  tags         = ["docker-node"]

  boot_disk {
    initialize_params {
      image = "ubuntu-os-cloud/ubuntu-1404-trusty-v20160602"
    }
  }

  network_interface {
    network = "default"

    access_config {
      # Ephemeral
    }
  }

  metadata {
    ssh-keys = "root:${file("${var.public_key_path}")}"
  }


  provisioner "remote-exec" {
    connection {
      type        = "ssh"
      user        = "root"
      private_key = "${file("${var.private_key_path}")}"
      agent       = false
    }

    inline = [
      "sudo curl -sSL https://get.docker.com/ | sh",
      "sudo usermod -aG docker `echo $USER`",
      "sudo docker run -d -p 80:80 nginx"
    ]
  }

  service_account {
    scopes = ["https://www.googleapis.com/auth/compute.readonly"]
  }
}

resource "google_compute_firewall" "default" {
  name    = "tf-www-firewall"
  network = "default"

  allow {
    protocol = "tcp"
    ports    = ["80"]
  }

  source_ranges = ["0.0.0.0/0"]
  target_tags   = ["docker-node"]
}

If you have used Google Cloud in the past, you will surely find it easy to understand. If not, I suggest you to dirty your hands in creating your first Google Cloud instance and running your first Docker container.

File: variables.tf

Input variables serve as parameters for a Terraform module. When used in the root module of a configuration, variables can be set from CLI arguments and environment variables. Below is the variables file one can set for our GCP instance which specified region, project ID, credential file, private and public SSH key.

[Captains-Bay]🚩 >  cat variables.tf
variable "region" {
  default = "us-central1"
}

variable "region_zone" {
  default = "us-central1-f"
}

variable "project_name" {
  description = "The ID of the Google Cloud project"
}

variable "credentials_file_path" {
  description = "Path to the JSON file used to describe your account credentials"
  default     = "~/.gcloud/Terraform.json"
}

variable "public_key_path" {
  description = "Path to file containing public key"
  default     = "~/.ssh/gcloud_id_rsa.pub"
}

variable "private_key_path" {
  description = "Path to file containing private key"
  default     = "~/.ssh/gcloud_id_rsa"
}

Generating Key

[Captains-Bay]🚩 >  ssh-keygen -f ~/.ssh/gcloud_id_rsa
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /Users/ajeetraina/.ssh/gcloud_id_rsa.
Your public key has been saved in /Users/ajeetraina/.ssh/gcloud_id_rsa.pub.
The key fingerprint is:
SHA256:ebJlwBe0MCFQv49bLHtfXZ2havVf89sulIYE2PDHXBI ajeetraina@Ajeets-MacBook-Air.local
The key's randomart image is:
+---[RSA 2048]----+
|    .oo =*o E..  |
|       +.+o= o   |
|        + +.+  . |
|         = .. . +|
|        S +. + oo|
|         X  + =..|
|        + +o o.oo|
|         =o  .. *|
|        o. ..  +*|
+----[SHA256]-----+
[Captains-Bay]🚩 >

Download the credential File from Google Cloud Console

You need to download credential file that contains your service account private key in JSON format. You can download your existing Google Cloud service account file from the Google Cloud Console, or you can create a new one from the same page.

Ensure that you create an empty directory .gcloud under the home directory and place this JSON file under this location.

[Captains-Bay]🚩 >  mkdir ~/.gcloud
[Captains-Bay]🚩 >  cd ~/.gcloud/

Terraform

The terraform init command is used to initialize a working directory containing Terraform configuration files. This is the first command that should be run after writing a new Terraform configuration or cloning an existing one from version control.By default, terraform init assumes that the working directory already contains a configuration and will attempt to initialize that configuration. You can run this command multiple times, it’s safe !

terraform init
[Captains-Bay]🚩 >  terraform plan
var.project_name
  The ID of the Google Cloud project

  Enter a value: i-guru-209217

Refreshing Terraform state in-memory prior to plan...
The refreshed state will be used to calculate this plan, but will not be
persisted to local or remote state storage.

google_compute_instance.www: Refreshing state... (ID: tf-www-0)
google_compute_firewall.default: Refreshing state... (ID: tf-docker-firewall)

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  + google_compute_firewall.default
      id:                                                  <computed>
      allow.#:                                             "1"
      allow.272637744.ports.#:                             "1"
      allow.272637744.ports.0:                             "80"
      allow.272637744.protocol:                            "tcp"
      destination_ranges.#:                                <computed>
      direction:                                           <computed>
      name:                                                "tf-www-firewall"
      network:                                             "default"
      priority:                                            "1000"
      project:                                             <computed>
      self_link:                                           <computed>
      source_ranges.#:                                     "1"
      source_ranges.1080289494:                            "0.0.0.0/0"
      target_tags.#:                                       "1"
      target_tags.1090984259:                              "docker-node"

  + google_compute_instance.docker
      id:                                                  <computed>
      boot_disk.#:                                         "1"
      boot_disk.0.auto_delete:                             "true"
      boot_disk.0.device_name:                             <computed>
      boot_disk.0.disk_encryption_key_sha256:              <computed>
      boot_disk.0.initialize_params.#:                     "1"
      boot_disk.0.initialize_params.0.image:               "ubuntu-os-cloud/ubuntu-1404-trusty-v20160602"
      boot_disk.0.initialize_params.0.size:                <computed>
      boot_disk.0.initialize_params.0.type:                <computed>
      can_ip_forward:                                      "false"
      cpu_platform:                                        <computed>
      create_timeout:                                      "4"
      deletion_protection:                                 "false"
      guest_accelerator.#:                                 <computed>
      instance_id:                                         <computed>
      label_fingerprint:                                   <computed>
      machine_type:                                        "f1-micro"
      metadata.%:                                          "1"
      metadata.ssh-keys:                                   "root:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDW4qyWPIaZg0fu5QMMgVRc96Nv1C2ft2k+cy6bkf0fz5WjZTDWaGRlvkdt7eZqFd5I7C+9frYfwUpBMAJ+lu2nK2xKxTjPUC/PGuhgIVz+AzJX1Rz1RxaOr//xMDvlYDvoQesRO/EMqb31uYPTY/WZVz8k+joj7OMQHkDwZo/Al5a8uSmkHQ6sPQ2mPusT7p7bFfe9M/xQxVBeWtvfAXtXTFRhGecLPByQQ3RogDMO5TvUh3/tURt54OmQNnqzRf36o9Nh69jxhSpbMrRr3ViWZADcyNnD0eECec+1d/3JzbZqoMmUhm5Jpiua+iEPYOj8WbvrU6j4GCuhth0HWSuP ajeetraina@Ajeets-MacBook-Air.local\n"
      metadata_fingerprint:                                <computed>
      name:                                                "tf-docker-0"
      network_interface.#:                                 "1"
      network_interface.0.access_config.#:                 "1"
      network_interface.0.access_config.0.assigned_nat_ip: <computed>
      network_interface.0.access_config.0.nat_ip:          <computed>
      network_interface.0.access_config.0.network_tier:    <computed>
      network_interface.0.address:                         <computed>
      network_interface.0.name:                            <computed>
      network_interface.0.network:                         "default"
      network_interface.0.network_ip:                      <computed>
      network_interface.0.subnetwork_project:              <computed>
      project:                                             <computed>
      scheduling.#:                                        <computed>
      self_link:                                           <computed>
      service_account.#:                                   "1"
      service_account.0.email:                             <computed>
      service_account.0.scopes.#:                          "1"
      service_account.0.scopes.2862113455:                 "https://www.googleapis.com/auth/compute.readonly"
      tags.#:                                              "1"
      tags.1090984259:                                     "docker-node"
      tags_fingerprint:                                    <computed>
      zone:                                                "us-central1-f"


Plan: 2 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Note: You didn't specify an "-out" parameter to save this plan, so Terraform
can't guarantee that exactly these actions will be performed if
"terraform apply" is subsequently run.

Verifying if Docker is installed on GCP instance

@tf-docker-0:~$ sudo docker version
Client:
 Version:           18.06.0-ce
 API version:       1.38
 Go version:        go1.10.3
 Git commit:        0ffa825
 Built:             Wed Jul 18 19:10:22 2018
 OS/Arch:           linux/amd64
 Experimental:      false
Server:
 Engine:
  Version:          18.06.0-ce
  API version:      1.38 (minimum version 1.12)
  Go version:       go1.10.3
  Git commit:       0ffa825
  Built:            Wed Jul 18 19:08:26 2018
  OS/Arch:          linux/amd64
  Experimental:     false
@tf-docker-0:~$ 

[Captains-Bay]🚩 >  terraform show
google_compute_firewall.default:
  id = tf-www-firewall
  allow.# = 1
  allow.272637744.ports.# = 1
  allow.272637744.ports.0 = 80
  allow.272637744.protocol = tcp
  deny.# = 0
  description =
  destination_ranges.# = 0
  direction = INGRESS
  disabled = false
  name = tf-www-firewall
  network = https://www.googleapis.com/compute/v1/projects/i-guru-209217/global/networks/default
  priority = 1000
  project = i-guru-209217
  self_link = https://www.googleapis.com/compute/v1/projects/i-guru-209217/global/firewalls/tf-www-firewall
  source_ranges.# = 1
  source_ranges.1080289494 = 0.0.0.0/0
  source_service_accounts.# = 0
  source_tags.# = 0
  target_service_accounts.# = 0
  target_tags.# = 1
  target_tags.1090984259 = docker-node
google_compute_instance.docker:
  id = tf-docker-0
  attached_disk.# = 0
  boot_disk.# = 1
  boot_disk.0.auto_delete = true
  boot_disk.0.device_name = persistent-disk-0
  boot_disk.0.disk_encryption_key_raw =
  boot_disk.0.disk_encryption_key_sha256 =
  boot_disk.0.initialize_params.# = 1
  boot_disk.0.initialize_params.0.image = https://www.googleapis.com/compute/v1/projects/ubuntu-os-cloud/global/images/ubuntu-1404-trusty-v20160602
  boot_disk.0.initialize_params.0.size = 10
  boot_disk.0.initialize_params.0.type = pd-standard
  boot_disk.0.source = https://www.googleapis.com/compute/v1/projects/i-guru-209217/zones/us-central1-f/disks/tf-docker-0
  can_ip_forward = false
  cpu_platform = Intel Ivy Bridge
  create_timeout = 4
  deletion_protection = false
  guest_accelerator.# = 0
  instance_id = 5050855407468093023
  label_fingerprint = 42WmSpB8rSM=
  labels.% = 0
  machine_type = f1-micro
  metadata.% = 1
  metadata.ssh-keys = root:ssh-rsa XXXX

  metadata_fingerprint = CXEyE8jgfhM=
  metadata_startup_script =
  min_cpu_platform =
  name = tf-docker-0
  network_interface.# = 1
  network_interface.0.access_config.# = 1
  network_interface.0.access_config.0.assigned_nat_ip = 35.226.155.224
  network_interface.0.access_config.0.nat_ip = 35.226.155.224
  network_interface.0.access_config.0.network_tier = PREMIUM
  network_interface.0.access_config.0.public_ptr_domain_name =
  network_interface.0.address = 10.128.0.2
  network_interface.0.alias_ip_range.# = 0
  network_interface.0.name = nic0
  network_interface.0.network = https://www.googleapis.com/compute/v1/projects/i-guru-209217/global/networks/default
  network_interface.0.network_ip = 10.128.0.2
  network_interface.0.subnetwork = https://www.googleapis.com/compute/v1/projects/i-guru-209217/regions/us-central1/subnetworks/default
  network_interface.0.subnetwork_project = i-guru-209217
  project = i-guru-209217
  scheduling.# = 1
  scheduling.0.automatic_restart = false
  scheduling.0.on_host_maintenance = MIGRATE
  scheduling.0.preemptible = false
  scratch_disk.# = 0
  self_link = https://www.googleapis.com/compute/v1/projects/i-guru-209217/zones/us-central1-f/instances/tf-docker-0
  service_account.# = 1
  service_account.0.email = 737359258701-compute@developer.gserviceaccount.com
  service_account.0.scopes.# = 1
  service_account.0.scopes.2862113455 = https://www.googleapis.com/auth/compute.readonly
  tags.# = 1
  tags.1090984259 = docker-node
  tags_fingerprint = KMHM74J1xug=
  zone = us-central1-f

Verifying if Nginx container is running

Run the below command directly on Google cloud instance

inc@tf-docker-0:~$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS         
       NAMES
a6df1767bb64        nginx               "nginx -g 'daemon of…"   3 minutes ago       Up 3 minutes        0.0.0.0:80->80
/tcp   elastic_pare
[Captains-Bay]🚩 >  curl 35.226.155.224
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

In my future blogs, I will show how to setup Docker Swarm Mode and Kubernetes cluster using Terraform. Stay tuned !

Did you find this blog helpful?  Feel free to share your experience. Get in touch with me at twitter @ajeetsraina.

If you are looking out for contribution/discussion, join me at Docker Community Slack Channel.

 

Clap