Docker for Mac: the Missing Manual

Under the hood, Docker for Mac is running an Alpine Linux virtual machine. This guide helps with issues related to communication between OS X/macOS and this VM, and running up against limits on the size of the disk allocated to the VM.

Speeding things up

Disable sync on flush

This speeds up write operations involving containers. The tradeoff is increased risk of data loss: pending writes will be lost if your computer, Docker, or a container crashes. Since Docker for Mac is used for development, not production, this may be a good tradeoff to make. Here's how:

References:

overlay2 storage engine

If you installed Docker for Mac a while ago, it's probably using the aufs storage engine. overlay2 is a newer, more performant storage engine. From https://docs.docker.com/engine/userguide/storagedriver/selectadriver/#docker-ce:

When possible, overlay2 is the recommended storage driver. When installing Docker for the first time, overlay2 is used by default. Previously, aufs was used by default when available, but this is no longer the case.

On existing installations using aufs, it will continue to be used.

Elsewhere, this page says:

Docker for Mac and Docker for Windows are intended for development, rather than production. Modifying the storage driver on these platforms is not possible.

But this is not true: you can use overlay2 with Docker for Mac.

Switching storage engines changes where Docker looks for containers and images, so none of the ones you had with the old storage engine will be found. When you build images, Docker will download new ones to be stored using the new storage engine. Once you're happy with the new storage engine, you can temporarily switch back, delete all the images and containers associated with the old one, and then return to the new storage engine.

Not sure if you need to switch? Check first:

docker info | grep "Storage Driver"
# If it's overlay2, you're good. If it's aufs, continue with the next steps.

To switch, go to Docker, Preferences, Daemon, Advanced, and add the following key/value pair to the json in the box:

1
2
3
{
  "storage-driver": "overlay2"
}

Click Apply & Restart.

References:

Docker-sync

Some people use Docker-sync for better performance, especially when using PHP projects like Symfony or Drupal. I haven't used it myself, so here's where to learn more:

Freeing disk space

The VM disk has a limit on how big it can grow, so it can run out of space, resulting in errors when trying to build images.

Additionally, you may want to reclaim space from the VM disk image for your own use, particularly if you know that you are storing images and containers that you no longer need.

Diagnosis

What size is your VM's disk?

$ /Applications/Docker.app/Contents/MacOS/qemu-img info ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
image: /Users/william/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
file format: qcow2
virtual size: 64G (68719476736 bytes)
disk size: 53G
cluster_size: 65536
Format specific information:
    compat: 1.1
    lazy refcounts: true
    refcount bits: 16
    corrupt: false

64G for the VM. How much space is left on it?

$ docker run --rm --privileged debian:jessie df -h
Filesystem      Size  Used Avail Use% Mounted on
overlay          60G   51G  5.2G  91% /
tmpfs            64M     0   64M   0% /dev
tmpfs           3.0G     0  3.0G   0% /sys/fs/cgroup
/dev/sda2        60G   51G  5.2G  91% /etc/hosts
shm              64M     0   64M   0% /dev/shm

Only 5.2G left on /! Pull a few more big images, or build a large project inside a container, and this VM could run out of space.

Subproblem 1: removing unneeded images and containers

  1. Start containers that you know you want to keep.

  2. Using Spotify's docker-gc, do a dry run to see what would be deleted:

    $ docker run --rm -e DRY_RUN=1 -v /var/run/docker.sock:/var/run/docker.sock -v /etc:/etc:ro spotify/docker-gc

  3. If that looks good, do the real thing:

    $ docker run --rm -v /var/run/docker.sock:/var/run/docker.sock -v /etc:/etc:ro spotify/docker-gc

References:

Pruning volumes

More about this to come.

docker volume ls -f dangling=true
docker system prune --volumes

Alternative

docker-clean provides a nice wrapper around the actual docker commands.

1
2
3
4
5
# Install:
$ brew install docker-clean

# Dry run:
$ docker-clean -n

Subproblem 2: reclaiming space for VM and host

Now that unneeded images and containers are gone, we want to reclaim the disk space they occupied for use by the host and/or the VM.

Docker for Mac (after version 1.12) is supposed to run TRIM every 15 minutes using a cron job. If you want to manually trigger a TRIM:

$ docker run --rm -it --privileged --pid=host walkerlee/nsenter -t 1 -m -u -i -n fstrim /var

The first time I tried this, nothing happened until I restarted Docker. After restarting, hyperkit went to max cpu for hours. While it was running, docker commands hung. Once it finished, I had to restart Docker again. It had freed up 30 GB!

Since then, it has worked (freeing space on the VM and on the host) without a restart.

References:

Getting a shell in the VM

1
docker run --rm -it --privileged --pid=host walkerlee/nsenter -t 1 -m -u -i -n sh
References:

Alternative: attach to tty

1
2
$ brew install screen
$ screen ~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/tty

This isn't really getting a shell, but rather using GNU screen to connect to a tty device. Don't exit the shell; instead, tell screen to detach by typing Control-a d. To re-attach:

1
2
3
4
5
$ screen -ls
# lists screen session that is still open

$ screen -dr
# re-attaches to open session

If you simply attach screen again, the terminal text will be garbled.

References:

Problems with tar

macOS uses GNU tar. Inside a Linux-based image, the tar that's available is likely GNU tar. If you encounter errors like "Directory renamed before its status could be extracted", they might stem from running GNU tar inside an image on a tarball created by BSD tar (outside of the image).

Try either:

  1. Install GNU on your host and use it:
1
2
brew install gnu-tar
gtar ...
  1. Or use BSD tar inside the image by installing it in your Dockerfile:
1
2
3
4
5
6
FROM ...

apt-get update -qq
apt-get install -qqy --no-install-recommends bsdtar

RUN bsdtar ...

More!

Got more tips? This is a living document. Please leave a comment below.