What and Why Cross-Compiling
Sometimes you want a program compiled for an architecture which is not the one
you are using. Specifically, I sometimes want to build Crystal binaries for
ARM so I can run them in my home server, but the computer I normally use is
an x86 one, with an AMD CPU.
There are at least two solutions for this:
1) Build it in the server, or in a machine similar to the server.
This may be tricky because of many factors:
- Maybe the server is too busy
- Maybe it doesn't have development tools
- Maybe I don't have another similar machine
- Maybe it's a heck of a lot slower
2) Build a binary for the server's architecture
on my machine even though it's a different architecture. That's crosscompiling.
This tutorial explains one of the possible ways to do that.
For other ways you can see this tutorial
or the official crystal docs
I think this solution is simpler than both of them :-)
The Magic of qemu-static
If you don't know QEmu, it's an awesome open source
emulator. It lets you run virtual machines for almost any architecture in
almost any other.
One offshoot of this project is qemu-static which enables you to build and run containers for other
architectures via transparent emulation.
You first need to run this command so everything else will work.
$ docker run --rm --privileged \
multiarch/qemu-user-static \
--reset -p yes
What that does is configure binfmt handlers for binaries in a number of platforms:
Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
...
You can read about this in more detail
but the short version is: binaries for any platform now work, and since in Linux
containers are just a way to run isolated binaries ... well, container images for
other platforms work too.
Building Crystal code using Docker
Let's create a simple docker image that can compile crystal code:
# This makes it use the platform we specify in the docker commandline
# rather than the one of the system you are on. So we just use alpine
# as a base
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine AS base
# And then we install crystal in it
RUN apk add crystal shards
We can build an image, let's call it crystal using that:
$ docker build . -t crystal
[+] Building 1.5s (7/7) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 120B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 1.4s
=> [auth] library/alpine:pull token for registry-1.docker.io 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/2] FROM docker.io/library/alpine:latest@sha256:...8a8bbb5cb7188438 0.0s
=> CACHED [2/2] RUN apk add crystal 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:...7f6abb5fe6d393a94689834bef88 0.0s
=> => naming to docker.io/library/crystal
So if we have some crystal code, like hello.cr:
puts "Hello!"
And we can use that image to build a statically linked binary (don't be scared by the long command):
$ docker run -ti -u $(id -u):$(id -g) \
-v .:/src -w /src crystal \
crystal build hello.cr -o hello --static
This tells docker to run in an interactive terminal (-ti) as the current user (-u $(id -u):$(id -g)) with the current folder visible as /src (-v .:/src) inside the folder /src (-w /src), using the container crystal the command crystal build hello.cr -o hello
After a second or so, a new hello file appears in your folder. It's the compiled version of hello.cr and is a regular binary:
$ ll hello
-rwxr-xr-x 1 ralsina users 3.6M Jun 24 16:18 hello*
$ file hello
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=5c7de9ae7321754c7c53c8ea60670c19f3424fbe, with debug_info, not stripped
Well, regular up to a point. It's statically linked. And it was not built in my arch linux system, it was built in the alpine continer! If it wasn't static, it would depend on musl
instead of glibc, and in fact, I don't even need to have a crystal compiler in my system at all!
Bringing It All Together
So, if we know how to build crystal code using Docker, and we have a system
that can run Docker images for other architectures ... why not build our code
using Crystal in a container for other architectures?
First: we build a ARM version of our crystal container:
$ docker build . --platform=aarch64 -t crystal
[+] Building 1.4s (7/7) FINISHED docker:default
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 120B 0.0s
=> [internal] load metadata for docker.io/library/alpine:latest 1.3s
=> [auth] library/alpine:pull token for registry-1.docker.io 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [1/2] FROM docker.io/library/alpine:latest@sha256:b89d9c93e9ed3597455c90a0b88a8bbb5cb7188438 0.0s
=> CACHED [2/2] RUN apk add crystal 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:95feb8f2b9773f6946bd39b07e4dab7fb974012db58f81375772e88d417a323e 0.0s
=> => naming to docker.io/library/crystal
The only thing different from before is the --platform=aarch64 argument, which makes Docker build an ARM image.
And we can use the same argument to build an ARM version of hello:
$ docker run --platform=aarch64 -ti -u $(id -u):$(id -g) -v .:/src -w /src crystal crystal build hello.
cr -o hello --static
$ ll hello
-rwxr-xr-x 1 ralsina users 3.6M Jun 24 16:24 hello*
$ file hello
hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, BuildID[sha1]=56183bdedb28fd383643fcbd234fdcee6aae2b4f, with debug_info, not stripped
As you can see it's an aarch64 binary now (which is ARM) not a x86_64 one.
You can even create a couple of shell aliases so that you have "crystal-arm" and "shards-arm" commands:
$ alias crystal-arm="docker run --platform=aarch64 -ti -u $(id -u):$(id -g) -v .:/src -w /src crystal crystal"
$ alias shards-arm="docker run --platform=aarch64 -ti -u $(id -u):$(id -g) -v .:/src -w /src crystal shards"
And then you just build things as always, but using the alias:
$ crystal-arm build hello.cr -o hello
Caveats and Conclusions
- There is a performance penalty, the ARM version of crystal running in emulation will be slower than the x86 version.
- If you are building more complex things using
shards then you may have to change the Dockerfile and add dependencies such as libraries or C compilers in the crystal image.
- qemu-static itself only works on X86 so you cannot use this to cross-compile to x86 from ARM.
I think this is not much documented elsewhere and similar approaches should work for any language where you don't want to bother setting up a cross-compiling toolchain or if the tooling doesn't allow it.
