Fixing 'Failed to ping backend API' Docker error on macOS

I recently upgraded to an M1 Macbook Pro running macOS Monterey, and at first, Docker seemed to run fine. I was able to build my containers and do a lengthy database import.

A day later, when I started Docker, nothing happened for a minute, then I saw an error modal saying "Failed to ping backend API". The modal offered a few buttons such as resetting to factory defaults, but 1) I really didn't want to reset and lose my containers, including that lengthy database import, and 2) the buttons did not work anyway -- I got a beachball cursor if I tried to click them.

I quickly found this issue with lots of folks reporting the same problem. It was still open, with no progress in sight. There were several reported fixes, and I tried the ones that seemed possibly applicable to my case, mostly around giving Docker additional privileges through System Settings -> Privacy, including Full Disk Access, Automation control of Terminal.app, and Developer Tools. I continued to get the same error. Nothing was left except the brute-force approach of uninstalling Docker via command line, which again, I really didn't want to do.

I opened Console.app just to have a look at what happened when I launched Docker. As usual, a flood of output scrolled past, plenty of it from Docker. One snippet that caught my eye was this:

1
2
3
unlocking leaked directory locks: 2 errors occurred:
	* modifying existing entry for /Users/william/scm/sd/sd-playground/config: getting qualifier in ACL entry &{0x13700ba10} for /Users/william/scm/sd/sd-playground/config: mbr_uuid_to_id: No such file or directory
	* modifying existing entry for /Users/william/scm/sd/sd-playground/src: getting qualifier in ACL entry &{0x13800d810} for /Users/william/scm/sd/sd-playground/src: mbr_uuid_to_id: No such file or directory

sd-playground is a source-controlled directory containing an application I work on for my job at SpanishDict. It contains a Dockerfile and a container is from it by our application stack. I was curious why these subdirectories were singled out with this error, so I searched on "mbr_uuid_to_id" and found...nothing very useful. The other part that grabbed my eye was the mention of an ACL -- I didn't see any reason these directories should have ACLs, as I've never used that feature. I noticed something strange when I did ls -l on them:

1
2
$ ls -ld config
drwxr-xr-x+ 10 william  staff   320B Oct  6 13:57 config/

The plus sign in the rightmost column stood out, so I searched on that and found that it signifies the presence of extended permissions called -- wait for it -- ACLs. Some more searching, and I found that I could see the actual ACL with the -e flag:

1
2
3
$ ls -lde config
drwxr-xr-x+ 10 william  staff   320B Oct  6 13:57 config/
 0: 247BA062-6ABD-446F-80B7-6BE861CFCA42 deny delete

Mostly, I wanted to remove the ACL since I saw no use for it, and figured it might get rid of one problem bothering Docker. chmod 755 config didn't do it, but chmod -N did the trick:

1
2
3
$ chmod -N config
$ ls -led config
drwxr-xr-x  10 william  staff   320B Oct  6 13:57 config/

I figured this was unlikely to fix Docker, but when I restarted Docker.app, to my surprise, it came up! Obviously, it would be helpful if Docker for Mac could surface errors like this for the end user, so they don't have to go spelunking in the system logs.

How to use prettier with Emacs js2-mode 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
(require 'nvm)

(defun do-nvm-use (version)
  (interactive "sVersion: ")
  (nvm-use version)
  ;; exec-path-from-shell made a new login shell at startup and imported values,
  ;; including PATH to exec-path. But nvm-use does setenv "PATH". So we need to
  ;; update exec-path to the current PATH in the Emacs process.
  (exec-path-from-shell-copy-env "PATH"))

(defun my/use-prettier-if-in-node-modules ()
  "Use prettier-js-mode if prettier is found in this file's
project's node_modules. Use the prettier binary from this
project."
  (let* ((root (locate-dominating-file
                (or (buffer-file-name) default-directory)
                "node_modules"))
         (prettier (and root
                      (expand-file-name "node_modules/prettier/bin/prettier.js"
                                        root))))
    (when (and prettier (file-executable-p prettier))
      (setq prettier-js-command prettier)
      (prettier-js-mode))))

(when (require 'prettier-js nil t)
  (make-variable-buffer-local 'prettier-js-command)
  (add-hook 'js2-mode-hook #'my/use-prettier-if-in-node-modules))

Docker for Mac: the Missing Manual

Under the hood, Docker for Mac is running an Alpine Linux virtual machine. This guide helps with issues related to communication between OS X/macOS and this VM, and running up against limits on the size of the disk allocated to the VM.

Speeding things up

Disable sync on flush

This speeds up write operations involving containers. The tradeoff is increased risk of data loss: pending writes will be lost if your computer, Docker, or a container crashes. Since Docker for Mac is used for development, not production, this may be a good tradeoff to make. Here's how:

References:

overlay2 storage engine

If you installed Docker for Mac a while ago, it's probably using the aufs storage engine. overlay2 is a newer, more performant storage engine. From https://docs.docker.com/engine/userguide/storagedriver/selectadriver/#docker-ce:

When possible, overlay2 is the recommended storage driver. When installing Docker for the first time, overlay2 is used by default. Previously, aufs was used by default when available, but this is no longer the case.

On existing installations using aufs, it will continue to be used.

Elsewhere, this page says:

Docker for Mac and Docker for Windows are intended for development, rather than production. Modifying the storage driver on these platforms is not possible.

But this is not true: you can use overlay2 with Docker for Mac.

Read More

How I made Magit fast again

TLDR: here are the settings that made the biggest difference for me. YMMV.

1
2
(setq magit-commit-show-diff nil
      magit-revert-buffers 1)

Magit is awesome. And it's getting better with regular releases, a more consistent interface, and much more. But since the release of 2.1, it's generally been slower for me. I'm not the only one. In particular, the status buffer would take multiple seconds to refresh after almost any action such as committing, checking out a branch, stashing/popping, deleting a file, etc.

Tarsius, Magit's maintainer, is clearly aware of the performance issues and working to fix them. It can't be easy to diagnose performance problems given the multitude of ways Magit can be configured, plus the huge variety of characteristics among all the git repos out there. Nonetheless, I'm sure performance will improve in future versions.

But I needed to do something about it in the near term. I searched online and while Magit has a page dedicated to perf settings, none of them helped me much. So I grepped the Magit source for defcustom and read all the docstrings in search of things to try. Here's what I found.

Read More

How HTML5 sandboxes could be so much more useful

I love the idea of HTML5 iframe sandboxes but I'm unable to apply them the way they are currently implemented.

Why? Because the iframes I want to target are created programmatically. I'm not writing the iframe tags myself, or the code that writes them, so I can't specify the sandbox attribute.

www.spanishdict.com is an ad-supported site, like many other sites out there. We use Google Publisher Tags (GPT) which creates a cross-origin iframe for each ad slot on the page -- automatically.

unsandbox iframes in unsandboxed iframes

We have relationships with several different ad companies. On every pageview, we load a script from each company so they can evaluate the impression and make their bid (a technique known as "header bidding").

Then the winner of the bid gets to load their creative, which they typically do by creating another cross-origin iframe within the iframe that GPT created and has given them access to. Sometimes you'll see several layers of iframe nesting.

These iframes are all created programmatically. Because I don't create the iframes (or know anything about them ahead of time such as what domains they're going to come from), I can't add the sandbox attribute, which makes it useless for me.

In my ideal world

Ideally, I could set sandboxing to apply to the cross-origin iframes I know are going to be created on my page.

Read More

Why I'd like Node and io.js to merge

I made nodegovernance.io — which encourages Node users to express support for using io.js's open governance model as the basis for the Node foundation's technical committee — because from where I sit, one Node with a technical committee composed of the best technical people is the ideal outcome of this situation.

My life would be easier if there were just one Node. My team wouldn't have to spend time discussing and deciding which to use. There wouldn't be any confusion in the future about whether npm install would give me a module I could use with whatever runtime I happened to be using.

The recent Medium post said that io.js met with Joyent CEO Scott Hammond last week, so I'm sure he knows exactly where they stand.

And the big companies that are already on the foundation board are certainly going to get input into the rules for the technical committtee.

What about the rest of us — engineers who use Node / io.js every day but aren't inner core, who don't work for an IBM or a PayPal, who want stability and also language improvements that will make our jobs easier?

I hated the idea of looking back a few months from now, in the midst of heated discussions about whether to use Node or io.js, with npm installs failing all around me, and wondering if there was anything I could have done to prevent this situation. A tweet doesn't count for all that much, but it's more than nothing.

Using a Node repl in Emacs with nvm and npm

Running a repl inside Emacs is often convenient for evaluating code, checking syntax, and myriad other tasks. When I wanted to run a Node REPL, I found that I needed to do a little set up to get everything working the way I wanted.

My first question was: which Node? With nvm, I've installed multiple version on my machine. So I needed a way to specify one to execute.

Another question was: where to run Node? Since npm looks inside node_modules directories starting with the current directory and working up the file system hierarchy, the current working directory is important. If I want access to the npm modules installed for project A, I need to start my repl's Node process from path/to/projectA.

But that raises another question: what happens when I want to switch to project B? Do I need to use process.chdir() to switch the Node repl's current working directory to path/to/projectB? That's clumsy and annoying.

Here's how I answered these questions:

Read More

How legit HTTP (with an async io assist) massacred my Node workers

An uncaught exception in our Node app was causing not only one, but two and then three workers to die. (Fortunately, we hardly ever encounter uncaught exceptions. Really, just this one since launch a few months ago. We're Node studs! Right?)

The funny thing is that we're using Express, which (via Connect) wraps each request / response in a try / catch. And we use Express's error handler, which returns 500 on unhandled errors.

Another funny thing is we use cluster, which isolates workers from each other. They live in separate, solipsistic processes.

But instead of returning 500, our worker simply died. And, as if in sympathy, the rest immediately followed.

Time to get to the bottom of this. A Node stud like me can figure it out. No sweat. Right?

Read More