Creating Docker images that can run on different platforms, including Raspberry Pi
Building a multi-architecture Docker images
As each generation of the Raspberry Pi single-board computer emerges, its processing power continues to advance. The current fourth generation boasts a processor with four Cortex-A72 cores and up to 8GB of RAM. These impressive capabilities, enclosed within a printed circuit board measuring 85✕55, make the Raspberry Pi an excellent choice for use as a home server, offering a wide range of possibilities from media serving to automation of the software development process (CI/CD) or home automation. Given our commitment to containerization through Docker, it makes sense to consider deploying Docker images on the Raspberry Pi.
It's worth noting that the Raspberry Pi's processor architecture and instruction set differ from those of typical desktop computers. As a result, the Docker image must be built separately, targeting specific architecture. Fortunately, the increasing popularity of other processor families has led to many Docker Hub images already being prepared for use on the Raspberry Pi. For example, let's examine the Debian stable version:
We are most interested in the linux/arm/v7 and linux/arm64 platforms, because these instruction sets have the last 3 generations of RPi. These platforms are targeting various operating systems. In this case the linux/arm/v7 is aimed for 32-bit and linux/arm64/v8 for 64-bit operating systems.
The question arises as to what to do when a specific Raspberry Pi version requires an image that has not been provided. While it is possible to build the image directly on a single-board computer (SBC), this process can be complicated. In this case, we recommend using the Docker extension Buildx to build the image on a local machine. This approach offers greater flexibility and can help to streamline the development process.
Docker Buildx
Buildx is a plugin for Docker's CLI that significantly expands Docker's image building and management capabilities. Introduced in version 19.03, Buildx is currently enabled by default. For more detailed information on how to use Buildx, please consult the official documentation.
Ok, Docker build capabilities are enhanced, so what’s next?
With Buildx, it is possible to prepare images for one or multiple platforms depending on the configuration. The building process can be performed using various drivers, each with different feature sets. These include the Docker server, an additional container capable of emulating other platforms, and the use of a Kubernetes cluster.
While Buildx offers a great deal of flexibility, its implementation has certain limitations. For instance, using the Docker server driver only allows for the creation of an image for a single platform. In such cases, the goal is often to build an image that can be executed on different platforms using only the local machine.
Once the appropriate driver has been selected, the next step is to identify the available instances of builders (build environments) that Buildx can locate on a given machine. This can be accomplished by issuing the docker buildx ls command.
➜ ~ docker buildx ls NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS default * docker default default running linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
The default builder is included by default in Buildx, and uses the local Docker server as its backend. While it supports cross-platform image building, it is important to note that it does not enable the creation of multi-platform images, as stated in the documentation. To build multi-platform images, a different builder configuration must be used.
error: multiple platforms feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")
Let’s start with creating a new builder called multi:
➜ ~ docker buildx create --driver=docker-container --name=multi --use multi ➜ ~ docker buildx ls NAME/NODE DRIVER/ENDPOINT STATUS PLATFORMS multi * docker-container multi0 unix:///var/run/docker.sock inactive default docker default default running linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
Note: in conjunction with the docker buildx create command, we utilized the following options:
- --driver - selects the driver that will be used,
- --name - gives a name to our builder instance,
- --use - instructs to use the newly created instance for subsequent operations.
Upon analyzing the output, it is evident that the builder is currently inactive. However, this will change once the first build is initiated.
In order to test our configuration, we can create a simple Dockerfile that displays information about the platform upon which it is executed at build time.
FROM debian:stable-slim RUN uname -m
Now it's build time! Buildx command syntax is modeled on a regular docker build command with additional parameters.
--progress- changes the way logs are displayed,--platform- allows providing a comma separated list of platforms for which the image will be created.
For this exercise, we will choose a PC as well as the two most popular platforms for Raspberry Pi.
➜ ~ docker buildx build -t dockerpro/multi-arch:latest --progress plain --platform linux/amd64,linux/arm/v7,linux/arm64 . WARNING: No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load #1 [internal] booting buildkit #1 pulling image moby/buildkit:buildx-stable-1 #1 pulling image moby/buildkit:buildx-stable-1 5.9s done #1 creating container buildx_buildkit_multi0 #1 creating container buildx_buildkit_multi0 0.6s done #1 DONE 6.5s #2 [internal] load .dockerignore #2 transferring context: 2B done #2 DONE 0.0s #3 [internal] load build definition from Dockerfile #3 transferring dockerfile: 74B done #3 DONE 0.0s *** image downloading cut out for readability *** #9 [linux/amd64 2/2] RUN uname -m #0 0.069 x86_64 #9 DONE 0.2s #11 [linux/arm64 2/2] RUN uname -m #0 0.050 aarch64 #11 DONE 0.1s #12 [linux/arm/v7 2/2] RUN uname -m #0 0.079 armv7l #12 DONE 0.2s
As observed in the output, a significant amount of logs are being generated. This can be attributed to BuildKit, an advanced Docker mechanism that enables parallel execution of tasks and offers improved cache management, among other capabilities. It is worth delving further into BuildKit, even if building Raspberry Pi images is not a priority. Additional information can be found here.
Going back to the log analysis.
Returning to the logs we can observe that in the first step the BuildKit image container is started, then the .dockerignore file is loaded and the build context is transferred. The next step is to load the Dockerfile and execute its instructions. For our image we selected 3 platforms, so the instructions are executed for each platform separately. At first, the Debian image is downloaded then the uname -m program is executed to print system information. Specifically on which "machine" it is currently running. You can see this in more details in steps #9, #11 and #12.
As mentioned earlier, regular computers and Raspberry Pi processors are not compatible. However, Buildx and BuildKit use QEMU to emulate other hardware, allowing the uname -m program to report other platforms during the build process. It should be noted that emulation using QEMU is slower than direct code execution, but it provides the ability to build images in one place. Alternatively, we can use other Raspberry Pi boards and add them to Buildx as external builders, which could be a more efficient solution depending on the requirements. Of course, this approach would require additional hardware.
The image building process was completed successfully. However, it is worth noting that a warning was displayed at the beginning of the build logs.
This one exactly:
WARNING: No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load
When we use a regular docker build command, the image defaults to a local collection and we can run a container from it or send it to an external registry.
In the Buildx context, a driver using an additional container to build images requires configuring what should happen with the image after it is built. The suggested --load option (which is a shorthand for --output=type=docker) will unfortunately not work in our case.
Using --load option will cause a following error:
error: docker exporter does not currently support exporting manifest lists
This is because the "docker exporter" used by the --load option cannot save the image manifest, which contains metadata for multiple platforms, at once. Therefore, we need to use a different output option to save the image in a way that supports multi-platform images.
Thinking of modernizing existing architecture?
Regardless of the profile of the solution, the introduction of a microservices architecture may positively affect the efficiency of the solution. We are happy to provide consultation in this area.
Custom image registry
Even though the image is built on the local machine, it ultimately needs to be transferred to the Raspberry Pi. In the Buildx context, the only available option to save the image requires a registry where images can be transferred and stored. Thus, a registry is a necessary solution to our problem.
To transfer the built multi-platform image to the Raspberry Pi, we need to save it in a registry. The Docker registry can run on any machine with sufficient disk space. The official implementation, written in Golang, requires minimal CPU and memory resources. Setting up the registry is straightforward and can be accomplished by:
➜ ~ docker volume create registry-data registry-data ➜ ~ docker run -d -p 5000:5000 -v registry-data:/var/lib/registry --restart always --name registry registry:2docker run -d -p 5000:5000 -v registry-data:/var/lib/registry --restart always --name registry registry:2 216f1553784bd45d2fbaf43906ac5f55182c1475ad1d0e94d2e41175293a48c7
This will start the registry on port 5000. Additionally, the registry will automatically start after a system reboot and will save data in a volume named registry-data. Note that if you plan to use the registry for anything beyond testing purposes, it is recommended to use docker-compose to keep the entire configuration as code.
To fully utilize the benefits of Buildx, we need to recreate the builder with a configuration file named builder.toml, which will serve as a blueprint for our builder. It is important to note that the IP address used in the file (192.168.1.2) must be replaced with the IP address of your computer. To find your IP address, you can use a command such as hostname -I:
[registry."192.168.1.2:5000"] http = true insecure = true
This file is required to connect to the registry without encryption. While we have omitted encryption for the purpose of this blog entry, it's important to note that encryption is a must(!) in a production environment to ensure the security of the data. When deploying a registry, it's recommended to use SSL certificates to encrypt the communication between the registry and the clients. This can be achieved by either obtaining a trusted SSL certificate or generating a self-signed one. It's important to note that using a self-signed certificate may result in warnings from the clients about the untrusted certificate.
Now, we can use the builder.toml configuration file to create a builder in Buildx. To use the configuration file, we need to add the --config option to the docker buildx create command. After that, we can build and upload the image to the registry.
➜ ~ docker buildx rm multi ➜ ~ docker buildx create --driver=docker-container --name=multi --use --config=builder.toml multi ➜ ~ docker buildx build -t 192.168.1.2:5000/multi-arch:latest --progress plain --platform linux/amd64,linux/arm/v7,linux/arm64 --push . *** cut off *** #13 exporting to image #13 exporting layers 0.1s done #13 exporting manifest sha256:c6623eda280c8ee76b2bd63f9a73f429204ad4a260d60edd44de92a9401d276e #13 exporting manifest sha256:c6623eda280c8ee76b2bd63f9a73f429204ad4a260d60edd44de92a9401d276e done #13 exporting config sha256:279c30a6d250e68984e909a414149aa21a62138523b0f59c41d0beaa5e65f27d done #13 exporting manifest sha256:ba2339518538ebd6e178db4b3f604fbe82d0167f25b453bd75dbd6bde5203732 done #13 exporting config sha256:a25a708b066296c7c952fa48bd59dda84ec3a0a7206e086302fd402f0a6d4d4c done #13 exporting manifest sha256:7625f06a22638df4e5749709851816a83222ca8e8ab35c1979eb4d206d22c72b done #13 exporting config sha256:1b27246fc54aaa67e4d09b8161e1068dc8527982c4c0173a31f025bde9ebbff2 done #13 exporting manifest list sha256:95a7f6cd1a6e7d5b2f4a5dd0d936d56f767166dedc13669cbd6c627b25a43cf4 done #13 pushing layers #13 pushing layers 2.5s done #13 pushing manifest for 192.168.1.2:5000/multi-arch:latest@sha256:95a7f6cd1a6e7d5b2f4a5dd0d936d56f767166dedc13669cbd6c627b25a43cf4 0.0s done #13 DONE 2.6s
The private Docker Hub has received and stored its first image. I would like to write that the next steps are super easy and it is only the download to Raspberry Pi operation that is left. Unfortunately, it is still not the case. As previously in the case of Buildx, we must first enable Docker to use an unencrypted connection to the registry.
To enable Docker to use an unencrypted connection to the registry, we need to create a file called /etc/docker/daemon.json with the following content, replacing the IP address used here (192.168.1.2) with the IP address of your computer:
{ "insecure-registries" : ["192.168.1.2:5000"] }
Now let's restart Docker and see if we can successfully run the container with the image previously prepared:
pi@raspberry:~ $ sudo systemctl restart docker pi@raspberry:~ $ sudo docker run --rm 192.168.1.2:5000/multi-arch:latest uname -m Unable to find image '192.168.1.2:5000/multi-arch:latest' locally latest: Pulling from multi-arch f83522bf96d7: Pull complete 329a37630ca9: Pull complete Digest: sha256:f994065ab94a50f09de56873529bcdb4660f50fa3c1b1fbb9797da96725c5766 Status: Downloaded newer image for 192.168.1.2:5000/multi-arch:latest armv7l
Congratulations, you have successfully completed the process of building and deploying a Docker image to a private registry for your Raspberry Pi. While the process may have required some effort, the end result is a highly efficient and customizable system that you can easily manage from your home network.
It is highly recommended to configure encryption and authentication to ensure the security of your registry. Additionally, you can enhance your registry with a user-friendly web interface, such as the one provided by docker-registry-ui.
Conclusion
The blog post describes the process of building and deploying a Docker image on a Raspberry Pi using a private Docker registry. It covers the use of Buildx, a Docker CLI plugin, for building multi-arch images, and creating a private Docker registry to store and transfer the images to the Raspberry Pi.
I hope that this post has been informative and useful in simplifying your multi-arch builds. Now you should know the process of building and deploying a Docker image on a Raspberry Pi using a private Docker registry. It covers the use of Buildx, a Docker CLI plugin, for building multi-arch images, and creating a private Docker registry to store and transfer the images to the Raspberry Pi.
If you require additional assistance beyond what has been provided, please feel free to reach out to us for professional support. We are always available and eager to assist you.
Useful links
Buildx:
BuildKit:
Registry:
- https://docs.docker.com/registry/deploying/
- https://docs.docker.com/registry/insecure/
- https://github.com/Joxit/docker-registry-ui
Raspberry Pi:
