“LinuxKit is NOT designed with an intention to replace any of traditional OS like Alpine, Ubuntu, Red Hat etc. It is an open-source toolbox for building fine-tuned Linux-based operating systems that run in containers and hence, lean, portable & secure out-of-the-box.”
~ Collabnix
How is LinuxKit different from other available GNU operating system? How is it different from Hashicorp Packer & Cloud instances? Why should I consider LinuxKit? I came across these queries in various community forums and discussion and thought to come up a write-up which talks about the factor which differentiate LinuxKit from other available operating system, tools & utilities.
In case you’re new, LinuxKit is a toolkit for building minimal Linux distributions that is lean, secure and portable. LinuxKit is built with an intention to build a distributions that has just enough Linux to accomplish a task and just do it in a way that is more secure. It uses Moby to build the distribution images and the LinuxKit tool to run them on the local hypervisor, cloud and bare metal environment.
Under this blog post, I will go through 10 good reasons why LinuxKit is better than other available GNU / open source distributions:
1. Smaller in size(< 200MB) compared to GBs size of traditional OS
The idea behind LinuxKit is that you start with a minimal Linux kernel — the base distro is only 35MB — and add literally only what you need. Once you have that, you can build your application on it, and run it wherever you need to.
~ source
LinuxKit is not a full host operating system. It primarily has two jobs to accomplish- first to run containerd containers and secondly, be secure. Linuxkit provides the ability to build an extremely small distribution (~50m) stripped of everything but that which is required to run containers.The init system and all other userland system components (e.g. dhcp, sshd) are containers themselves and as such can be swapped out, or others be plugged in. The whole OS is immutable (unless data volumes are configured), and will run anywhere one finds silicon or a hypervisor: Windows, MacOS, IoT devices, the Cloud etc.The system does not contain extraneous packages or drivers by default. Because LinuxKit is customizable, it is up to individual operators to include any additional bits they may require.
Let us try building LinuxKit ISO out of the minimal.yaml (shown below) which is also available under LinuxKit repository and verify its size.
https://gist.github.com/ajeetraina/0d32eaca14df499fce883cebd8f6d559
[simterm]
$git clone https://github.com/linuxkit/linuxkit
$cd linuxkit && make
$cp -rf bin/linuxkit /usr/local/linuxkit
[/simterm]
Building LinuxKit ISO
[simterm]
Ajeets-MacBook-Air:examples ajeetraina$ linuxkit build -format iso-bios -name kitiso minimal.yml
[/simterm]
This will build up minimal LinuxKit ISO.
Let us verify its size as shown below:
[simterm]
Ajeets-MacBook-Air:examples ajeetraina$ du -sh * | grep linuxkit.iso
133M linuxkit.iso
Ajeets-MacBook-Air:examples ajeetraina$ file linuxkit.iso
linuxkit.iso: DOS/MBR boot sector; partition 1 : ID=0x17, active, start-CHS (0x0,0,1), end-CHS (0x84,63,32), startsector 0, 272384 sectors
Ajeets-MacBook-Air:examples ajeetraina$
[/simterm]
A Traditional OS, in the other hand, is a complete operating system and includes hundreds and thousands of application programs.
2. Minimal Provisioning/Boot Time(1.5 – 3 minutes)
LinuxKit hardly takes 1.5 to 3 minutes to get up and running on either local hypervisor/cloud or bare metal system. This is way better than 3 to 5 minutes of provisioning time as compared to traditional OS distribution. LinuxKit is an entirely immutable system, coming in at around 50MB with nothing extraneous to that which is needed to run containers. The root filesystem is read-only, making it stateless and tamper-proof. LinuxKit runs almost everything – onboot processes and continuous services- in a container which makes the boot time quite fast. Even the init phase – where the OS image will be booted – is configured by copying files from OCI images.
While booting up the minimal LinuxKit ISO, it just took 0.034515sec for the containerd to get booted up.(shown below)
3. Build-time Package Updates(LinuxKit) Vs Run-time Package Updates(OS)
LinuxKit uses Moby tool which assembles a set of containerized components into an image. It uses YAML configuration which specifies the components used to build up an image . All components are downloaded at build time to create an image. The image is self-contained and immutable, so it can be tested reliably for continuous delivery. Most importantly – The build itself takes a matter of seconds, and is eminently reproducible, making it an ideal candidate to pass through a CI system. This is definitely an advantage over traditional GNU operating system.
4. Built-in Security(LinuxKit) Vs Base Security(Traditional OS)
LinuxKit allows users to create very secure Linux subsystems because it is designed around containers. All of the processes, including system daemons, run in containers, enabling users to assemble a Linux subsystem with only the needed services. As a result, systems created with LinuxKit have a smaller attack surface than general purpose systems.
LinuxKit is architected to be secure by default. LinuxKit’s build process leverages Alpine Linux’s hardened userspace tools such as Musl libc, and compiler options that include -fstack-protector and position-independent executable output.
Most importantly, LinuxKit uses modern kernels, and updates frequently following new releases. LinuxKit tracks new kernel releases very closely, and also follows best practice settings for the kernel configuration from the Kernel Self Protection Project and elsewhere.
The core system components included in LinuxKit userspace are written in type safe languages, such as Rust, Go and OCaml, and run with maximum privilege separation and isolation. LinuxKit’s build process heavily leverages Docker images for packaging. Of note, all intermediate build images are referenced by digest to ensures reproducibility across LinuxKit builds. Tags are mutable, and thus subject to override (intentionally or maliciously) – referencing by digest mitigates classes of registry poisoning attacks in LinuxKit’s buildchain. Certain images, such as the kernel image, is usually signed by LinuxKit maintainers using Docker Content Trust, which guarantees authenticity, integrity, and freshness of the image.
If you compare it with the traditional OS, many kernel bugs usually lurk in the codebase for years. Therefore, it is imperative to not only patch the kernel to fix individual vulnerabilities but also benefit from the upstream security measures designed to prevent classes of kernel bugs.
5. Container & Cloud Native at the same time
It is important to note that Linuxkit is built with containers, for running containers. Linuxkit today is supported on local hypervisor, cloud and bare metal system. The same minimal LinuxKit ISO image which is container native runs well on Cloud platform like Google Cloud Platform, Amazon Web Services & Microsoft Azure flawlessly.
If we talk about the traditional OS available, the distribution is usually customized by Cloud vendors to fit into the Cloud which is quite different from the one which is available for bare metal system. For example, An Amazon Machine Image (AMI) or preemptive Google Cloud instances etc.
6. Batteries included but removable/swappable
LinuxKit is built on the philosophy of “batteries included but swappable”. Everything is replaceable and customizable under LinuxKit. That’s one of the unique feature in LinuxKit. The YAML format specifies the components used to build an ISO image. This is made possible via the moby tool which assembles a set of containerized components into an image. Let us look at the various YAML file which talks about how this philosophy really works:
- Minimal YAML file to build LinuxKit OS
https://gist.github.com/ajeetraina/0d32eaca14df499fce883cebd8f6d559
- YAML file which builds LinuxKit OS with SSH Enabled
https://gist.github.com/ajeetraina/9c46fc410e48f322ec72535d5db38078
If you compare the above YAML file with the minimal YAML, just SSHD service containers has been added and hence new LinuxKit ISO can be booted up. You can find various other YAML files under this link.
7. Immutable in Production
Lots of companies – small or very large – have used immutability as a core concept of their infrastructure.Immutable infrastructure basically consists of immutable components which are actually replaced for every deployment, rather than being updated. These components are usually started from a common image that is built once per deployment and can be tested and validated. The common image can be built through automation, but doesn’t have to be. Important to note that immutability is independent of any tool or workflow for building the images. Its best use case is in a cloud or virtualized environment.
Why is it important?
Immutable components as part of your infrastructure are a way to reduce inconsistency in your infrastructure and improve the trust into your deployment process.
Toolkit like LinuxKit have made building immutable components very easy. Together with existing cloud infrastructure, it is a powerful concept to help you build better and safer infrastructure.
8. Designed to be managed by external tooling
LinuxKit today can be managed flawlessly with popular tooling like Infrakit, Terraform or CloudFormation. It uses external tooling such as Infrakit or CloudFormation templates to manage the update process externally from LinuxKit, including doing rolling cluster upgrades to make sure distributed applications stay up and responsive.
Updates may preserve the state disk used by applications if needed, either on the same physical node, or by reattaching a virtual cloud volume to a new node.
Soon after Dockercon 2017 Austin, I wrote a blog post on how Infrakit & LinuxKit work together for building immutable infrastructure.
Why Infrakit & LinuxKit are better together for Building Immutable Infrastructure?
If you want to know how LinuxKit & Terraform can work together, you should look at this link too.
9. Enough to bootstrap distributed applications
LinuxKit is designed for building and running clustered applications, including but not limited to container orchestration such as Docker or Kubernetes. It is obvious that in the production environment, most of the users will use a cluster of instances, usually running distributed applications, such as etcd, Docker, Kubernetes or distributed databases. LinuxKit provides examples of how to run these effectively, largely using the tooling likeinfrakit, although other machine orchestration systems can equally well be used, for example Terraform or VMWare.
LinuxKit is gaining momentum in terms of a toolkit for building custom minimal, immutable Linux distributions. Integration of Infrakit with LinuxKit help users to build and deploy custom OS images to a variety of targets – from a single vm instance on the mac (via xhyve / hyperkit, no virtualbox) to a cluster of them, as well as booting a remote ARM host on Packet.net from the local laptop via a ngrok tunnel.
10. Eliminates Cloud-provides specific Base Image variance
If we talk about Cloud, one serious challenge is that each vendor has differentiated IP invested in how they handle certain aspects of their cloud infrastructure. It is not enough to just provision n-number of servers;what IT ops teams need is a simple and consistent way to declare the number of servers, what size they should be, and what sort of base software configuration is required. Linuxkit solves this problem of base software configuration issue by allowing users to build a stripped down version of traditional OS based on their own requirement.
As said earlier, LinuxKit is not here to replace Alpine OS. LinuxKit uses the minimalist Alpine Linux distribution by default as the foundation of its official container images, and will continue to do so in future. Alpine is seen as an ideal all-purpose generic base for running heavy-lifting software like Redis and Nginx within containers. You can also switch Alpine for Debian/ubuntu/centos, if you wish.
Interested to learn more about LinuxKit, head over to these series of articles.
Did you find this blog helpful? Feel free to share your experience. Get in touch @ajeetsraina.
If you are looking out for contribution/discussion, join me at Docker Community Slack Channel.
