What and Why Cross-Compiling

Some­times you want a pro­gram com­piled for an ar­chi­tec­ture which is not the one you are us­ing. Specif­i­cal­ly, I some­times want to build Crys­tal bi­na­ries for ARM so I can run them in my home server, but the com­put­er I nor­mal­ly use is an x86 one, with an AMD CPU.

There are at least two so­lu­tions for this:

1) Build it in the server, or in a ma­chine sim­i­lar to the serv­er.

This may be tricky be­cause of many fac­tors:

• Maybe the serv­er is too busy
• Maybe it does­n't have de­vel­op­ment tools
• Maybe I don't have an­oth­er sim­i­lar ma­chine
• Maybe it's a heck of a lot slow­er

2) Build a bi­na­ry for the server's ar­chi­tec­ture on my ma­chine even though it's a dif­fer­ent ar­chi­tec­ture. That's cross­com­pil­ing.

This tu­to­ri­al ex­plains one of the pos­si­ble ways to do that.

For oth­er ways you can see this tu­to­ri­al or the of­fi­cial crys­tal docs

I think this so­lu­tion is sim­pler than both of them :-)

The Magic of qemu-static

If you don't know QE­mu, it's an awe­some open source em­u­la­tor. It lets you run vir­tu­al ma­chines for al­most any ar­chi­tec­ture in al­most any oth­er.

One off­shoot of this project is qe­mu-stat­ic which en­ables you to build and run con­tain­ers for oth­er ar­chi­tec­tures via trans­par­ent em­u­la­tion.

You first need to run this com­mand so ev­ery­thing else will work.

$ docker run --rm --privileged \
        multiarch/qemu-user-static \
        --reset -p yes

What that does is configure binfmt handlers for binaries in a number of platforms:

Setting /usr/bin/qemu-alpha-static as binfmt interpreter for alpha
Setting /usr/bin/qemu-arm-static as binfmt interpreter for arm
Setting /usr/bin/qemu-armeb-static as binfmt interpreter for armeb
Setting /usr/bin/qemu-sparc-static as binfmt interpreter for sparc
Setting /usr/bin/qemu-sparc32plus-static as binfmt interpreter for sparc32plus
Setting /usr/bin/qemu-sparc64-static as binfmt interpreter for sparc64
Setting /usr/bin/qemu-ppc-static as binfmt interpreter for ppc
Setting /usr/bin/qemu-ppc64-static as binfmt interpreter for ppc64
...

You can read about this in more de­tail but the short ver­sion is: bi­na­ries for any plat­form now work, and since in Lin­ux con­tain­ers are just a way to run iso­lat­ed bi­na­ries ... well, con­tain­er im­ages for oth­er plat­forms work too.

Building Crystal code using Docker

Let's cre­ate a sim­ple dock­er im­age that can com­pile crys­tal code:

# This makes it use the platform we specify in the docker commandline
# rather than the one of the system you are on. So we just use alpine
# as a base
FROM --platform=${TARGETPLATFORM:-linux/amd64} alpine AS base

# And then we install crystal in it
RUN apk add crystal shards

We can build an image, let's call it crystal using that:

$ docker build . -t crystal
[+] Building 1.5s (7/7) FINISHED                                                        docker:default
 => [internal] load build definition from Dockerfile                                              0.0s
 => => transferring dockerfile: 120B                                                              0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                  1.4s
 => [auth] library/alpine:pull token for registry-1.docker.io                                     0.0s
 => [internal] load .dockerignore                                                                 0.0s
 => => transferring context: 2B                                                                   0.0s
 => [1/2] FROM docker.io/library/alpine:latest@sha256:...8a8bbb5cb7188438  0.0s
 => CACHED [2/2] RUN apk add crystal                                                              0.0s
 => exporting to image                                                                            0.0s
 => => exporting layers                                                                           0.0s
 => => writing image sha256:...7f6abb5fe6d393a94689834bef88      0.0s
 => => naming to docker.io/library/crystal   

So if we have some crystal code, like hello.cr:

puts "Hello!"

And we can use that im­age to build a stat­i­cal­ly linked bi­na­ry (don't be scared by the long com­mand):

 $ docker run -ti -u $(id -u):$(id -g) \
    -v .:/src -w /src crystal \
    crystal build hello.cr -o hello --static

This tells docker to run in an interactive terminal (-ti) as the current user (-u $(id -u):$(id -g)) with the current folder visible as /src (-v .:/src) inside the folder /src (-w /src), using the container crystal the command crystal build hello.cr -o hello

After a second or so, a new hello file appears in your folder. It's the compiled version of hello.cr and is a regular binary:

$ ll hello
-rwxr-xr-x 1 ralsina users 3.6M Jun 24 16:18 hello*

$ file hello
hello: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, BuildID[sha1]=5c7de9ae7321754c7c53c8ea60670c19f3424fbe, with debug_info, not stripped

Well, reg­u­lar up to a point. It's stat­i­cal­ly linked. And it was not built in my arch lin­ux sys­tem, it was built in the alpine con­tin­er! If it was­n't stat­ic, it would de­pend on musl in­stead of glibc, and in fac­t, I don't even need to have a crys­tal com­pil­er in my sys­tem at al­l!

Bringing It All Together

So, if we know how to build crys­tal code us­ing Dock­er, and we have a sys­tem that can run Dock­er im­ages for oth­er ar­chi­tec­tures ... why not build our code us­ing Crys­tal in a con­tain­er for oth­er ar­chi­tec­tures?

First: we build a ARM ver­sion of our crys­tal con­tain­er:

$ docker build . --platform=aarch64 -t crystal
[+] Building 1.4s (7/7) FINISHED                                                        docker:default
 => [internal] load build definition from Dockerfile                                              0.0s
 => => transferring dockerfile: 120B                                                              0.0s
 => [internal] load metadata for docker.io/library/alpine:latest                                  1.3s
 => [auth] library/alpine:pull token for registry-1.docker.io                                     0.0s
 => [internal] load .dockerignore                                                                 0.0s
 => => transferring context: 2B                                                                   0.0s
 => [1/2] FROM docker.io/library/alpine:latest@sha256:b89d9c93e9ed3597455c90a0b88a8bbb5cb7188438  0.0s
 => CACHED [2/2] RUN apk add crystal                                                              0.0s
 => exporting to image                                                                            0.0s
 => => exporting layers                                                                           0.0s
 => => writing image sha256:95feb8f2b9773f6946bd39b07e4dab7fb974012db58f81375772e88d417a323e      0.0s
 => => naming to docker.io/library/crystal 

The only thing different from before is the --platform=aarch64 argument, which makes Docker build an ARM image.

And we can use the same argument to build an ARM version of hello:

$ docker run --platform=aarch64 -ti -u $(id -u):$(id -g) -v .:/src -w /src crystal crystal build hello.
cr -o hello --static

$ ll hello
-rwxr-xr-x 1 ralsina users 3.6M Jun 24 16:24 hello*

$ file hello
hello: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), statically linked, BuildID[sha1]=56183bdedb28fd383643fcbd234fdcee6aae2b4f, with debug_info, not stripped

As you can see it's an aarch64 binary now (which is ARM) not a x86_64 one.

You can even cre­ate a cou­ple of shell alias­es so that you have "crys­tal-ar­m" and "shard­s-ar­m" com­mand­s:

$ alias crystal-arm="docker run --platform=aarch64 -ti -u $(id -u):$(id -g) -v .:/src -w /src crystal crystal"

$ alias shards-arm="docker run --platform=aarch64 -ti -u $(id -u):$(id -g) -v .:/src -w /src crystal shards"

And then you just build things as al­ways, but us­ing the alias:

$ crystal-arm build hello.cr -o hello

Caveats and Conclusions

• There is a per­for­mance penal­ty, the ARM ver­sion of crys­tal run­ning in em­u­la­tion will be slow­er than the x86 ver­sion.
• If you are building more complex things using shards then you may have to change the Dockerfile and add dependencies such as libraries or C compilers in the crystal image.
• qe­mu-stat­ic it­self on­ly works on X86 so you can­not use this to cross-­com­pile to x86 from AR­M.

I think this is not much doc­u­ment­ed else­where and sim­i­lar ap­proach­es should work for any lan­guage where you don't want to both­er set­ting up a cross-­com­pil­ing toolchain or if the tool­ing does­n't al­low it.