Docker Desktop versus Colima on Apple M2 silicon

So, this is written from the perspective of someone that doesn’t really care too much which container engine that they’re using. I simply wanted to evaluate a vendor’s product and knew that containers would be the most efficient way to get up and running fast. I’m also undertaking this activity as a personal training exercise and that means that Docker Desktop licensing should not be an issue. However, if you are using containers in a work capacity but are not licensed for Docker Desktop then read on as this may provide you with viable open source solution for container management.

Also note that the challenges I faced here were specific to Apple MacOS users that are running on Apple’s proprietary M1/M2 silicon – the irony is not lost on me here running on such expensive tin and concerned about relatively trivial licensing costs 😁.

TLDR

If licensing is not an issue and you lack that engineering curiousity about the various container engines but simply want to get the job done Docker Desktop is the one for you!

Just don’t forget to install Apple’s Rosetta 2 software. This emulator/translator will help to seamlessly translate the x86/AMD (think Intel) containers or more correctly those containers’ low level CPU instructions to an Apple M2 recognised instruction set.

# softwareupdate --install-rosetta

And configure Docker Desktop to use Rosetta 2.

Then it’s just business as usual with your Docker commands.

Colima is a dropin replacement, kinda!

Colima is another superb community project that is helping to improve the MacOS users’ experience of managing containers on linux, MacOS and Nix!?! (don’t ask, I haven’t played with Nix yet either).

It’s as simple as

# Homebrew
brew install colima
...

followed by

colima start
...

and then you’re good to go, right? Hmmm, yes, I suppose so, provided you’re running some self contained (pun intented) images that don’t require external mounts.

The challenge that I had was that I was running a relatively complex vendor application that required multiple containers and physical volume mounts on the underlying OS. Also I had not yet optimised colima for the M2 silicon.

First things first, to optimise for M2 silicon and increase some of the basic resource configurations beyond their defaults let’s kill this engine and make a few tweaks as follows.

colima stop
...
colima delete
...
colima start --arch aarch64 --vm-type=vz --vz-rosetta --cpu 6 --memory 12 --disk 64

This should give you an optimal setup provided that you have similar resources available, if not then adjust to taste.

However, I still couldn’t get the vendor application to successfully run. Remember, it worked first time on Docker Desktop. Hunting through the docker container logs I could see the main container was stuck in an initialisation boot loop – it was complaining about file permissions on the mounted drives.

A quick search through the Colima Issues revealed the challenge along with the workaround – at least this worked for me.

So the final working Colima configuration for my M2 MBP

• create or put the following in /Users/(username)/.lima/_config/override.yaml

mountType: 9p
mounts:
  - location: "/Users/(username)"
    writable: true
    9p:
      securityModel: mapped-xattr
      cache: mmap
  - location: "~"
    writable: true
    9p:
      securityModel: mapped-xattr
      cache: mmap
  - location: /tmp/colima
    writable: true
    9p:
      securityModel: mapped-xattr
      cache: mmap

Please don’t forget to swap out (username) for your own username above (if you do you’re in good company but it was late in the evening for me, what’s your excuse?)

• delete the existing engine and start again – this time swapping back to qemu emulation and the older mountType

colima stop
...
colima delete
...
colima start --arch aarch64 --vm-type=qemu --vz-rosetta --cpu 6 --memory 12 --disk 64 --mount-type 9p

Finally we can successfully launch the same vendor application on the M2 silicon using Colima! Your standard Docker commands and docker-compose.yml files should all work as normal.

Docker’s context switch is useful if you do decide to run multiple container engines. It tells the docker binary which container engine to talk to. Also note, when you start Docker Desktop it swaps the context over automatically (yes that does make sense, I guess).

# docker context ls
NAME              DESCRIPTION                               DOCKER ENDPOINT                                  ERROR
colima            colima                                    unix:///Users/graz/.colima/default/docker.sock   
default           Current DOCKER_HOST based configuration   unix:///var/run/docker.sock                      
desktop-linux *   Docker Desktop                            unix:///Users/graz/.docker/run/docker.sock
#

Conclusion

In theory, the performace of Colima should be reduced as we are no longer using Apple’s native virtualisation by swapping back to QEMU in this instance. I did feel that there was a slight reduction in the performance compared with Docker Desktop.

Scientific Analysis – It’s a beefy (technical term) application and consistently took 2 minutes from docker compose up -d to login on when running on Docker Desktop whereas it was taking 3 minutes plus to get the login prompt when running on Colima.

If you’re in a rush, are a business, and can afford the license then Docker Desktop is possibly the best business focused option. However, Engineers and Developers will definitely play with Colima and please do contribute towards the project – it works and is continually improving.

MBP Details -> Apple M2 Max, 32GB, OS 13.5.2
Docker Desktop -> version 4.22.0
Colima -> version 0.5.5

Happy Shipping!

Graz

Below is the docker-compose.yml file used for the deployment.

version: '3.6'
services:
  web:
    image: 'gitlab/gitlab-ce:16.2.6-ce.0'
    depends_on:
       - redis
       - postgresql
    hostname: 'gitlab.demo'
    networks:
      vpcbr:
        ipv4_address: 10.5.0.4
    environment:
       GITLAB_OMNIBUS_CONFIG: |
         postgresql['enable'] = false
         gitlab_rails['db_username'] = "gitlab"
         gitlab_rails['db_password'] = "gitlab"
         gitlab_rails['db_host'] = "postgresql"
         gitlab_rails['db_database'] = "gitlabDB"
         gitlab_rails['db_adapter'] = 'postgresql'
         gitlab_rails['db_encoding'] = 'utf8'
         redis['enable'] = false
         gitlab_rails['redis_host'] = 'redis'
         gitlab_rails['redis_port'] = '6379'
         prometheus['enable'] = false
         external_url "http://gitlab.demo"
         gitlab_rails['gitlab_shell_ssh_port'] = 23
    container_name: gitlabce
    extra_hosts:
      - "gitlab.demo:127.0.0.1"
    restart:  always
    ports:
      - '80:80'
      - '443:443'
      - '23:22'
    volumes:
      - '$GITLAB_HOME/config:/etc/gitlab'
      - '$GITLAB_HOME/logs:/var/log/gitlab'
      - '$GITLAB_HOME/data:/var/opt/gitlab'
    shm_size: '4GB'
  postgresql:
     container_name: postgress
     image: postgres:15.4
     networks:
      vpcbr:
        ipv4_address: 10.5.0.5
     environment:
       - POSTGRES_USER=gitlab
       - POSTGRES_PASSWORD=gitlab
       - POSTGRES_DB=gitlabDB
  redis:
     image: redis:7.2.1
     container_name: redis
     networks:
      vpcbr:
        ipv4_address: 10.5.0.6

  runner:
    image: gitlab/gitlab-runner:v16.3.0
    container_name: gitlab-runner
    restart: always
    extra_hosts:
      - "gitlab.demo:10.5.0.4"
    volumes:
     - '$GITLAB_HOME/gitlab-runner/config:/etc/gitlab-runner'
     - '/var/run/docker.sock:/var/run/docker.sock'
    networks:
      vpcbr:
        ipv4_address: 10.5.0.7

networks:
  vpcbr:
    driver: bridge
    ipam:
     config:
       - subnet: 10.5.0.0/24
         gateway: 10.5.0.1