Installing typos in Ubuntu Docker image

[oalders]

I have to install my dot files in a bunch of slightly different scenarios, so I occasionally test installing them in a Docker container. I'm having an issue getting ubi to install typos. Not urgent or even important, but I figured I'd open an issue.

My docker-compose.yml includes

services:
  app:
    image: ubuntu:latest
    platform: linux/arm64/v8

Inside the container:

/root/local/bin/ubi --project crate-ci/typos --in /root/local/bin --debug
[ubi][DEBUG] Parsed --project crate-ci/typos = crate-ci / typos
[ubi][DEBUG] exe name = typos
[ubi][DEBUG] install path = /root/local/bin/typos
[reqwest::connect][DEBUG] starting new connection: https://api.github.com/
[ubi::picker][DEBUG] filtering out assets that do not have a valid extension
[ubi::picker][DEBUG] current OS = linux
[ubi::picker][DEBUG] matching assets against OS using (?i:(?:\b|_)linux(?:\b|_|32|64))
[ubi::picker][DEBUG] matching OS against asset name = typos-v1.24.1-aarch64-apple-darwin.tar.gz
[ubi::picker][DEBUG] does not match our OS
[ubi::picker][DEBUG] matching OS against asset name = typos-v1.24.1-x86_64-apple-darwin.tar.gz
[ubi::picker][DEBUG] does not match our OS
[ubi::picker][DEBUG] matching OS against asset name = typos-v1.24.1-x86_64-pc-windows-msvc.zip
[ubi::picker][DEBUG] does not match our OS
[ubi::picker][DEBUG] matching OS against asset name = typos-v1.24.1-x86_64-unknown-linux-musl.tar.gz
[ubi::picker][DEBUG] matches our OS
[ubi::picker][DEBUG] current CPU architecture = aarch64
[ubi::picker][DEBUG] matching assets against CPU architecture using (?ix)
        (?:
            \b
            |
            _
        )
        (?:
            aarch_?64
            |
            arm_?64
        )
        (?:
            \b
            |
            _
        )

[ubi::picker][DEBUG] there is only one asset that matches our OS
[ubi::picker][DEBUG] it matches a CPU architecture which is not ours
[ubi][ERROR] could not find a release for this OS and architecture (aarch64-unknown-linux-musl) from typos-v1.24.1-aarch64-apple-darwin.tar.gz, typos-v1.24.1-x86_64-apple-darwin.tar.gz, typos-v1.24.1-x86_64-pc-windows-msvc.zip, typos-v1.24.1-x86_64-unknown-linu
x-musl.tar.gz

Would it be reasonable to fall back to typos-v1.24.1-x86_64-unknown-linux-musl.tar.gz in this case?

[autarch]

Does the x86_64 build work in an arm64 container?

[oalders]

I'm not a huge expert on Docker and how it handles different architecture, but I've got typos-v1.24.1-x86_64-unknown-linux-musl.tar.gz

root@b808aaa78716:~/dot-files# uname -a
Linux b808aaa78716 5.15.49-linuxkit #1 SMP PREEMPT Tue Sep 13 07:51:32 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux

root@b808aaa78716:~/dot-files# ./typos --version
typos-cli 1.24.1

[autarch]

I'm quite confused. How is an x86-64 binary working in an ARM container? I don't think that in general it would make sense to fall back to x86 binaries on normal ARM platforms, like if you were running Linux on an actual ARM-powered system.

I wonder what's going on in Docker to make this work.

[oalders]

ChatGPT says:

Running x86-64 Binaries in ARM Containers

Running an x86-64 binary in an ARM container involves a few key techniques:

1. Emulation with QEMU

QEMU is an open-source emulator that can translate x86-64 instructions to ARM instructions on the fly. Docker integrates with QEMU to provide this CPU emulation.

How it Works

• Container Setup: Start with an ARM container image containing x86-64 binaries.
• QEMU Integration: Docker uses QEMU to handle the execution of x86-64 binaries.
• Instruction Translation: QEMU translates x86-64 instructions to ARM instructions.

2. Multi-Architecture Docker Images

Docker supports multi-architecture images, allowing for both ARM and x86-64 binaries within a single image.

Tools

• Multi-Arch Images: An image can be built with binaries for multiple architectures.
• Docker Buildx: A tool that facilitates building images supporting multiple architectures.

3. Cross-Compiling

Cross-compiling the application for ARM directly avoids the need for emulation by creating ARM-native binaries.

How it Works

• Build Process: Use a cross-compiler to target the ARM architecture.
• Native Execution: The binaries run natively on ARM, improving performance.

4. Docker Desktop and Other Platforms

Platforms like Docker Desktop on macOS or Windows use QEMU for emulation to run ARM containers on x86-64 hosts.

Summary

Emulation with QEMU is commonly used for running x86-64 binaries in ARM containers. For better performance, native ARM compilation or multi-architecture Docker images are preferred.

[autarch]

Hmm, I'm still not sure how this worked. From my reading, it seems multi-arch images (option 2) are just a way to distribute an image that can run on different host architectures. But inside the container image you should still only be able to run the architecture provided by the host.

And for the other options, it doesn't seem like it produces a situation where you can run x86-64 binaries on an ARM host or vice versa. Here's me trying to run an arm64 image:

docker run -it --platform linux/arm64/v8 ubuntu:latest  /bin/sh
Unable to find image 'ubuntu:latest' locally
latest: Pulling from library/ubuntu
Digest: sha256:ab64a8382e935382638764d8719362bb50ee418d944c1f3d26e0c99fae49a345
Status: Downloaded newer image for ubuntu:latest
exec /bin/sh: exec format error

That's what I'd expect. I know that on macOS Docker uses VMs under the hood, but I still don't understand how you produce a VM that can run both architecture's binaries!

[autarch]

Just for reference - https://docs.docker.com/build/building/multi-platform/