Docker volumes is an extensive feature of Docker that allows data to be stored and managed persistently. The volumes allow data to be stored outside the container system, and this is important because even in a situation where the container is removed or updated, the data is not tied to the lifecycle of the container, ensuring the integrity of the data.
In the latest Docker Desktop 4.28.0, users can filter volumes to see which ones are in use in the Volumes tab. You can now search for volumes in Global Search. The Docker Dashboard Volume Management feature offers the ability to efficiently clean up volumes using multi-select checkboxes.
In addition, Docker volumes simplify the process of creating backups or snapshots, and since data is not tied to the container, it makes it easier to protect against data loss during system failures or disasters.
Difference Between Docker Volumes and Bind Mounts
There are multiple methods of handling persistent data in Docker, and the two most popular methods are Docker volumes and Bind mounts.
Here is why Docker Volumes excels over Bind Mounts:
- Container Orchestration: Docker volumes are deeply integrated with Docker’s ecosystem, making them more suitable for container orchestration systems like Kubernetes and Docker Swarm as they can be managed and scaled more effectively.
- Data Isolation and Portability: Volumes ensure better data isolation since they are handled by Docker and are not dependent on the host’s filesystem. This makes them inherently more portable across different systems and environments.
- Extensibility and Integration: Docker Volumes support volume drivers, allowing for additional functionalities such as encryption or integration with cloud storage services, providing greater extensibility compared to Bind Mounts.
Docker Volumes Key Features
Here are some of the key features of Docker volumes:
- Persistence: Docker volumes provide a mechanism for persisting data generated and used by Docker containers. Volumes keep data separate from the lifecycle of a container; this is different from data stored in a container’s writable layer, which is ephemeral and disappears when a container is deleted.
- Compatibility: Docker volumes are compatible with both Linux and Windows containers, which means they can be used in a wide range of environments and with various applications. This cross-platform compatibility is essential for teams and applications that operate in a diverse set of environments.
- Portability: Volumes can be used across different Docker environments. You can attach it to containers on different hosts, making it easy to move data without the need to bind it to the host’s filesystem. This feature is very important for developers that want to share data between development, testing, and production environments or across different team members.
- Shared Access: This feature means that multiple containers can mount the same volume at the same time, allowing datasets to be shared at the same time.
- Backup and Restore: Docker volumes make it easier to back up container data because they are managed by Docker, meaning that they can be manipulated independently and are not dependent on the containers using them. Meaning that backups can be performed without stopping the containers, and data can be restored to a specific point in time without affecting container operation.
- Performance: Volumes can offer improved performance for I/O operations compared to bind mounts, especially on platforms like Docker Desktop for Mac and Windows. This is because volumes are managed by Docker and can be optimized for the underlying host system.
- Volume Drivers: Docker supports volume drivers, which are plugins that provide additional capabilities. For example, volume drivers can enable Docker volumes to be stored on remote hosts or cloud providers, to be encrypted, or to integrate with other storage systems. This extensibility allows developers to tailor the storage to their specific needs.
Using Docker Volumes
In this section, you will Create, Use, and Remove a Docker volume.
Creating a Docker Volume
Step 1:
To start a container volume, run the docker run command and set the -v flag.
$ docker run -it -v my_volume:/dconfig debian:latest
In the command above, you started a Debian container attached to your terminal with the interactive mode -it. The command also mounts a volume inside the container.
Step 2:
List the contents of your container’s ‘/config‘ directory to verify the volume mount.
$ ls /config
There should not be any output since no files have been created in ‘my_volume‘ yet.
Step 3:
Create a new file with some content inside the ‘/config‘ directory.
$ echo "Lopeum Serum" > /config/sample.txt
Step 4:
Display the content of the newly created file to confirm the operation.
$ cat /config/sample.txt
Lopeum Serum
Step 5:
Detach from your container by pressing Ctrl+P followed by Ctrl+Q or running ‘exit’.
$ exit
This will stop the container.
Step 6:
Now, you will start a new container using the ‘busybox’ image and attach the same volume you have created.
$ docker run -it -v my_volume:/settings busybox:latest
This command mounts the volume to ‘/settings‘ inside the container.
Step 7:
Check the ‘/settings‘ directory for the file created by the first container.
$ cat /settings/sample.txt
Lopeum Serum
Docker has persisted with the content of ‘my_volume‘ after the first container stopped, allowing you to access the data in the new container.
Creating and Linking Volumes Manually
In the previous section, the Docker volume was automatically linked. However, you can also pre-create using the docker volume create command. This method allows you to prepare volumes before they are needed by containers.
Step 1:
Create a new volume named ‘newdata‘.
$ docker volume create newdata
Step 2:
Confirm the volume ‘newdata‘ has been created.
$ echo newdata
Step 3:
Once you have created new volumes, you can now attach them to containers in a manner similar to the automatic method:
# Start a new container and mount the pre-created 'newdata' volume to '/app'
docker run -it -v newdata:/app alpine:latest
Manually creating and linking volumes offers several benefits. It allows you to be consistent and predictable. It also allows you to set up complex environments that need specific settings and provides more control over the volume’s lifecycle.
Docker Volumes in Dockerfiles
In Docker, when you want to start a new container that shares volumes with an already running container, you can bypass the need to list all the volume flags by using the --volumes-from option. This command clones the volume configuration from the source container:
Step 1:
Set the primary container to a named volume.
$ docker run -d --name db -v newdata:/data database-image:latest
Step 2:
Start a new container that inherits volumes from the ‘db’ container.
$ docker run -d --name backup --volumes-from db backup-image:latest
This makes Docker replicate the volume attachments from the original container to the new one, maintaining the same mount paths.
In Dockerfiles, use the VOLUME instruction to declare mount points within the image. This is to make sure that when a container is created from an image, Docker will automatically provide new volumes at the mounts you have specified.
Try the following Dockerfile:
FROM ubuntu:22.04
VOLUME /newdata
This Dockerfile makes sure that a volume is mounted at /newdata inside any container created from this image, even if no -v flag is used during the docker run command.
However, it’s possible to override this automatic behavior and mount a specific volume to the path declared in the Dockerfile:
$ docker run -v my_volume:/newdata app-image:latest
It’s important to note that while the VOLUME instruction can’t specify a host path, it does create volumes that are managed by Docker, independent of the host’s directory structure. This adds a degree of flexibility and security, as it prevents direct access to the host file system and allows for the use of volume drivers for additional functionality, such as remote storage or encryption. Docker volumes created in this manner are easier to back up, migrate, and share among multiple containers.
Removing Volumes
If you want to remove unused volumes, like for example, dangling volumes, which are volumes that are no longer referenced by any container
Firstly, identify the dangling volumes.
docker volume ls -qf dangling=true
Then remove the dangling volumes with the following command:
docker volume prune
Alternatively, you can use the docker volume rm command to delete a specific docker volume.
$ docker volume rm my_volume
Managing Docker Volumes
Listing Docker Volumes
Use the docker volume ls command to list all your volumes:
$ docker volume ls
Backing Up and Restoring Docker Volumes
You can backup Docker volumes to prevent data loss during disasters or system failures.
docker run --rm -v my_volume:/data -v $(pwd):/backup ubuntu tar czvf /backup/my_volume_backup.tar.gz /data
In this command, a temporary container is started that mounts the volume my_volume to its /data directory and the current working directory to /backup. It then uses the tar command to compress the contents of /data into a tarball in /backup.
To restore data from backup:
$ docker run --rm -v my_volume:/data -v $(pwd):/backup ubuntu tar xzvf /backup/my_volume_backup.tar.gz -C /data
In this command, a similar temporary container is started, which mounts the same volume and backup location. The tar command is used to extract the contents of the tarball into the volume’s directory.
Inspecting Docker Volumes
docker volume inspect my_volume
Removing Docker Volumes Still in Use
Volumes that are currently still in use with a container cannot be deleted until you use the -f flag:
$ docker volume rm app_data -f
Conclusion
In Docker containerized environments, Docker volumes provide persistent and flexible storage. This enables your data to outlive the ephemeral nature of containers. Here are some of the best practices for optimizing Docker volumes:
- Restrict the number of volumes to only what you need for your application to run properly.
- When an application only requires read access to data, use read-only volumes.
- Use systematic and structured processes to connect and disconnect volumes in containers. Improper disconnection could lead to data loss.
- Make use of volume plugins to help increase the functionality of Docker volumes, giving you access to features like snapshotting, replication, or interaction with cloud storage providers.
- Use atomic operations so that changes can be made in one step and not divided.
If you want to fully understand Docker volumes, it is advisable to check out their official documentation and practice different use cases. With Docker volumes, you gain the ability to maintain data consistency across container restarts and system reboots, share data between containers, and facilitate the backup and recovery processes.
