Invalid Exec Format
They gave me a MacBook Pro at work. I built a docker image locally. I pushed it to Amazon cloud to run. I got “invalid exec format” error. What gives?
Compatibility problem! The Mac has Apple M3 chip based on the ARM technology, while the computer in the Amazon cloud has a chip with x64 instruction set. Oops… Docker images are processor-architecture and OS specific. An image built for ARM cannot run on x64 and vice versa. Fortunately, Docker provides tools for building cross-platform and multi-platform images. Unfortunately, this is still an area under development, so the situation changes quickly and the information about it is often outdated or confusing.
Terminology
“OS” is an operating system, like Windows or Linux.
“Architecture” is a family of compatible processors. Intel and AMD compatible processors are labeled amd64 or x86_64. 64-bit ARM processors are labeled arm64, aarch64, or armv8. Here amd64 and arm64” are ‘standard’ names, and others are aliases. E.g. if you ask docker to build an aarch64 image, it obliges, but then reports image architecture as arm64‘.
“Platform” is a combination of OS and architecture, e.g. “linux/amd64“.
Inspecting Image Platform
Each “regular” image has an OS and an architecture associated with it. These can be viewed using the following command
docker inspect --format='{{.Os}}/{{.Architecture}}' image_tag
The output would be something like this: 'linux/arm64'
Local Images are Single Platform
At the time of writing (April 2024) local Docker image storage only supports single-platform images. I.e. each local image will have a specific architecture and platform. However, it is possible to build an image for another platform, not the same as the local computer.
docker build . --platform linux/amd64 -t ikriv/testimage:1.0-intel
docker build . --platform linux/arm64 -t ikriv/testimage:1.0-mac
Images built in this way can be pushed to DockerHub.
The trouble is that when pulling an image, one has to know which image to pick for the current platform, which may be inconvenient and error-prone.
Multi-Platform Solutions
It would be nice to have a single image tag that produces an image for the right platform when pulled.
E.g. I want to be able to do
docker pull httpd:latest
and pull the latest Apache image that works on my machine, without worrying which platform it is.
Docker offers two solutions to this problem: manifests and multi-platform images.
Manifests
Manifests video: https://www.youtube.com/watch?v=AQeGdMuJWIM&ab_channel=AntonPutra
Manifest is a special entity that acts as a collection of images. Note that at the time of writing manifests cannot be created locally, they always live on a remote server. Thus, manifests can only contain images If I built two platform specific images ikriv/testimage:1.0-intel and ikriv/testimage:1.0-mac, I can create a multi-platform manifest as follows:
docker push ikriv/testimage:1.0-intel
docker push ikriv/testimage:1.0-mac
docker manifest create ikriv/testimage:1.0 ikriv/testimage:1.0-intel ikriv/testimage:1.0-mac
docker manifest annotate ikriv/testimage:1.0 ikriv/testimage:1.0-intel --os linux --arch amd64
docker manifest annotate ikriv/testimage:1.0 ikriv/testimage:1.0-mac --os linux --arch arm64
docker manifest push ikriv/testimage:1.0
This way, when someone does docker pull ikriv/testimage:1.0, they will get either “intel” or “mac” version, depending on their platform. All three entities, testimage:1.0, testimage:1.0-intael, and testimage:1.0-mac will be visible on Dockerhub.
Multi-Platform Images
Multi-platform images video: https://www.youtube.com/watch?v=ZxsAXNCiSZw
It is possible to build a single image that contains multiple platform-specific sub-images using docker buildx command. To do so, one needs to create a named docker builder:
docker buildx create --name mybuilder
Then we make this builder default:
docker buildx use mybuilder
Then we build our multi-platform image and push it to the remote server in the same command. Keep in mind, a multi-platform image cannot be stored in the local image storage:
docker buildx build --platform linux/amd64,linux/arm64 -t ikriv/testbuildx --push .
This would create a single entity ikriv/testbuildx:latest on the DockerHub server. It contain images for two platforms. docker pull will download the image for the current platform.
For an example, inspect httpd image from Apache.
What’s the difference?
Images create using docker build --platform and using docker buildx produce slightly different results. In my test, buildx produced smaller images, but took longer time. The nature of this difference is subject of further research. However, functionally these images behave identically when pulled.
Conclusion
Docker multi-platform support is a new area that is still developing. You need to worry about platforms if your dev platform is different from your hosting platform. Local storage currently does not support multi-platform images (but see https://docs.docker.com/storage/containerd/). Docker offers two ways to create multi-platform images on the server: manifests and docker buildx command. They produce slightly different results, but seem to be functionally equivalent. Most popular images on DockerHub support multiple platforms, when you pull, you get a version that is compatible with the local machine.