Integrating Docker Scout with Prometheus

Software vulnerabilities can propagate from one component to another, leading to widespread system compromises. Docker Scout is a tool designed to bring visibility into Docker images, helping identify vulnerabilities and enforcing security policies throughout the software supply chain.

To maximize the potential of Docker Scout, integrating it with Prometheus allows you to actively monitor and visualize vulnerability data. This integration provides a real-time view of your application’s security posture, especially crucial in environments where containerized applications are essential. In this guide, we’ll walk through how to integrate Docker Scout with Prometheus using a real-world example with nginx to enhance visibility into software vulnerabilities and security policy adherence.

Docker Scout Metrics Exporter

Docker Scout provides a metrics exporter that exposes image vulnerability data via an HTTP endpoint. These metrics include vulnerability counts by severity level, policy violations, and other data points critical to maintaining a secure software pipeline. Accessing these metrics requires configuring Docker Scout to output them in a format compatible with Prometheus, enabling seamless integration and data collection.

Types of Available Metrics

The Docker Scout metrics exporter provides two major types of metrics:

1. Vulnerability Metrics: Shows vulnerabilities by severity across your Docker images.
2. Policy Metrics: Highlights any policy violations that may arise from non-compliant configurations or dependencies.

With these metrics, you can quickly identify images with critical or high-severity vulnerabilities, understand which policies are being violated, and track remediation over time. For more details on specific metrics and their meaning, refer to Docker’s official metrics documentation.

Setting up Prometheus for Docker Scout

Prerequisites

Before beginning, ensure the following prerequisites are met:

1. Docker and Docker Compose installed: Docker Scout runs as a service within Docker.
2. Prometheus installed: We will be configuring Prometheus to pull metrics from Docker Scout.
3. Access Token for Docker Scout: Required to authorize Prometheus to access Docker Scout’s metrics.

Step 1: Creating an Access Token

To securely integrate Docker Scout with Prometheus, you’ll need an access token:

1. Log in to your Docker account, and go to Settings.



2. Under Access Tokens, create a new token specifically for Prometheus integration.



3. Copy and securely save the token, as it will be used for authentication in the Prometheus configuration.

Step 2: Configuring Prometheus to Scrape Docker Scout Metrics

Open your Prometheus configuration file, prometheus.yml, and add a new job specifically for Docker Scout. This will allow Prometheus to fetch metrics directly from Docker Scout’s metrics endpoint.

scrape_configs:
  - job_name: 'docker_scout_metrics'
    metrics_path: '/metrics'
    scheme: 'https'
    static_configs:
      - targets: ['scout.docker.com']
    authorization:
      credentials: '<YOUR_ACCESS_TOKEN>'

Replace <YOUR_ACCESS_TOKEN> with the token you generated in the previous step. Save and restart Prometheus to apply the changes. This configuration enables Prometheus to securely fetch vulnerability and policy metrics from Docker Scout.

Creating a Prometheus Project with Nginx

In this section, we’ll set up a sample Prometheus project using Docker Compose to monitor the service — nginx — along with Docker Scout. This setup will allow us to test and visualize how Docker Scout’s metrics are integrated into an application stack.

Step 1: Setting Up the Project Directory

1. Create a project directory for your Prometheus setup:

mkdir prometheus-docker-scout
cd prometheus-docker-scout

2. Within this directory, create a docker-compose.yml file, which will define the services for this project.

Step 2: Creating the docker-compose.yml File

In the docker-compose.yml file, define Prometheus, Docker Scout, and the nginx service as follows:

services:
  prometheus:
    image: prom/prometheus:latest
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - ./token:/etc/prometheus/token  # Ensure you have the token file in your project directory
    ports:
      - "9090:9090"
    networks:
      - prometheus-network

  nginx:
    image: nginx:latest
    ports:
      - "80:80"
    networks:
      - prometheus-network

networks:
  prometheus-network:
    driver: bridge

In this configuration:

• prometheus is set up with a volume mounting prometheus.yml, which includes the scrape configurations for Docker Scout and other services.
• scout-exporter is Docker Scout’s exporter container, configured with the required access token.
• nginx is the standard service container added to simulate a real-world environment where Prometheus and Docker Scout monitor production-like services.

Step 3: Create the Token File

Create a file named token in your project directory. This file should contain your Docker Scout Personal Access Token (PAT).

echo "ENTER_YOUR_PAT" > token
chmod 600 token

Step 4: Writing the Prometheus Configuration

In the root of your project directory, create a prometheus.yml file with the following content:

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: '<ORG>'  # Replace <ORG> with your actual organization name
    metrics_path: '/v1/exporter/org/<ORG>/metrics'  # Adjust based on the API path
    scheme: https
    static_configs:
      - targets:
          - api.scout.docker.com
    authorization:
      type: Bearer
      credentials_file: /etc/prometheus/token 
  - job_name: 'nginx'
    static_configs:
      - targets: ['nginx:80']

In this configuration:

• docker_scout job pulls metrics from Docker Scout’s exporter.
• nginx jobs allow Prometheus to monitor the service, providing a realistic view of how Prometheus collects data from both application services and Docker Scout.

Step 5: Running the Project

Start all services with Docker Compose:

docker-compose up -d

Verify that the services are running by visiting http://localhost:9090 for Prometheus.

http://localhost:8081/metrics for Docker Scout metrics, and http://localhost for nginx.

Sample Application to Test Metrics (Optional)

In this section, we will create a sample application that can be monitored by Prometheus using the Nginx service. We can set up a simple HTTP server that serves metrics in a format that Prometheus can scrape. Below are the steps to create a basic Node.js application that generates metrics and configures Nginx to serve it.

Step 1: Create a Simple Node.js Application

mkdir nginx-metrics-app
cd nginx-metrics-app

Step 2: Initialize a New Node.js Project

npm init -y

Step 3: Install Required Packages

npm install express prom-client

Step 4: Create a File Named server.js

touch server.js

Step 5: Add the Following Code

const express = require('express');
const { collectDefaultMetrics, register, Counter } = require('prom-client');

const app = express();
const port = 3000;

// Collect default metrics
collectDefaultMetrics();

// Create a custom counter metric
const requestsCounter = new Counter({
  name: 'http_requests_total',
  help: 'Total number of requests',
  labelNames: ['method'],
});

app.use((req, res, next) => {
  // Increment the counter on each request
  requestsCounter.inc({ method: req.method });
  next();
});

// Route to expose metrics
app.get('/metrics', (req, res) => {
  res.set('Content-Type', register.contentType);
  res.send(register.metrics());
});

// Route for the homepage
app.get('/', (req, res) => {
  res.send('Hello, World! Visit /metrics to see the metrics.');
});

app.listen(port, () => {
  console.log(`Sample app listening at http://localhost:${port}`);
});

Step 6: Update the docker-compose.yml File

version: '3.7'
services:
  prometheus:
    image: prom/prometheus:latest
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"
    networks:
      - prometheus-network

  scout-exporter:
    image: docker/scout-exporter:latest
    environment:
      - ACCESS_TOKEN="ACCESS_TOKEN_HERE"
    ports:
      - "8081:8080"
    networks:
      - prometheus-network

  nginx:
    image: nginx:latest
    ports:
      - "80:80"
    networks:
      - prometheus-network
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf

  app:
    build: ./nginx-metrics-app
    ports:
      - "3000:3000"
    networks:
      - prometheus-network

networks:
  prometheus-network:
    driver: bridge

Step 7: Create an nginx.conf File

Create an nginx.conf file in your project directory with the following content:

worker_processes 1;

events { worker_connections 1024; }

http {
    server {
        listen 80;

        location / {
            proxy_pass http://app:3000;  # Redirect to Node.js app
        }

        location /metrics {
            proxy_pass http://app:3000/metrics;  # Redirect to Node.js metrics
        }
    }
}

Step 8: Navigate to Your Project Directory

cd ~/prometheus-docker-scout

Step 9: Build and Start the Containers

docker-compose up -d --build

Step 10: Create a Dockerfile

# Use the official Node.js image.
FROM node:14

# Set the working directory
WORKDIR /usr/src/app

# Copy package.json and install dependencies
COPY package.json ./
RUN npm install

# Copy the application code
COPY index.js ./

# Expose the port the app runs on
EXPOSE 3000

# Command to run the app
CMD ["npm", "start"]

Accessing and Visualizing Data

1. Querying Docker Scout Data in Prometheus

Prometheus’s expression browser allows you to query Docker Scout data and monitor real-time metrics on vulnerabilities and policy compliance. Examples of useful queries include:

• scout_vulnerabilities_total{severity="high"}: Lists all high-severity vulnerabilities across images.
• scout_policy_violations_total: Shows policy violations in images.

2. Integrating with Grafana for Dashboards

For a more user-friendly visualization, connect Prometheus to Grafana. Here’s how:

1. Add Prometheus as a Data Source: In Grafana, go to Settings > Data Sources and add Prometheus with http://prometheus:9090.
2. Import a Docker Scout Dashboard: Grafana provides pre-configured dashboards for Docker metrics. Customize these dashboards or use them as a base to include Docker Scout metrics.

Docker’s documentation provides additional information on integrating Grafana with Docker Scout.

Best Practices and Considerations

Optimal Scrape Intervals

Setting scrape intervals that balance system load with real-time visibility is essential. For most environments, a 30-second interval is recommended. In high-security environments or CI/CD pipelines, reducing this interval may be beneficial for faster alerts.

Security for Access Tokens

Access tokens for Docker Scout should be kept secure. Use environment variables or secure secrets management tools to store tokens, avoid including them in version-controlled files, and rotate them periodically.

Conclusion

Integrating Docker Scout with Prometheus brings critical vulnerability data and policy compliance monitoring directly into your observability stack. This integration allows DevOps teams to gain visibility across their containerized software supply chains, helping to prevent potential vulnerabilities from reaching production.

Through Prometheus’s data collection and Grafana’s visualization capabilities, you can establish a comprehensive monitoring system that enhances security and optimizes infrastructure management for the future.

Docker Scout’s capabilities continue to grow, and by combining them with Prometheus, you lay the groundwork for a secure, scalable, and observable software supply chain.