Anyone who runs GitLab Runner on Kubernetes eventually meets this problem. A project wants docker build in its pipeline. The usual answer is Docker-in-Docker (docker:dind), and DinD wants privileged: true. A privileged container can load kernel modules, mount host devices and, on most setups, trivially become root on the node. Every job from every project that can reach that runner then has that power.
The previous post was about Windows jobs on throw-away VMs. macOS is the same wish with harder constraints:
macOS may only legally run on Apple hardware, so you cannot just launch a VM from a cloud image on any host. On AWS, that hardware is an EC2 Mac, which is a dedicated host. You pay for the host, not for the instance, and a host cannot be released before 24 hours after it was allocated. A shell executor on a Mac is the easy approach, and it is how most people start. Every job then runs as the same user, in the same filesystem, with the same keychain. State leaks between jobs, and so do secrets. The setup in this post gets one clean VM per job on a small number of Macs, without giving up much of the simplicity.
Most of my GitLab Runner experience is Linux containers on Kubernetes (see the GKE as Gitlab Runner post). Windows jobs are a different story, and the first thing to learn is that “Windows runner” can mean two quite different things:
In the last post I argued that you should not split a model that fits on one machine: the 1B model I benchmarked was faster, steadier and cheaper in a single pod. The flip side of that argument is the case where splitting is not a choice. When the weights do not fit, pipeline parallelism is how you run the model at all.
In the last post I split a 1B-parameter Gemma model across two CPU nodes on purpose, to learn how LeaderWorkerSet and vLLM’s multi-node mode behave. I said then that splitting a model that fits on one machine makes it slower, and I listed “compare the split against a single replica” among the things I had not done.
I wanted to try two things at once: Kubernetes’ LeaderWorkerSet (LWS), the API for running a group of pods as one unit, and vLLM’s multi-node serving. The cheapest way to exercise both is not a GPU cluster. It is two small CPU nodes, one small model, and a deliberate decision to split that model across the nodes even though it fits on one.
Worklayer has workspaces, and each workspace has multiple panels including terminals. The naive implementation of switching workspaces would be: destroy the current workspace’s DOM, render the new workspace’s DOM. This works for web panels and file editors which can reload their state from a URL or file path. But terminals are different.
CLI tools like gh auth login, gcloud auth login, and aws sso login all share a pattern: they print a URL or open the system browser for OAuth authentication, wait for the callback, and then continue in the terminal. When running these commands inside Worklayer’s terminal panels, the browser opens externally and the user has to leave the app to complete the flow.
When engineers discuss storage and file systems, the conversation often starts and stops with the classics: ext4 on Linux, NTFS on Windows, and perhaps XFS for large database servers. For decades, the dominant answer to filesystem reliability was straightforward: journaling.
Whenever I board a flight, one of the first things I do after reaching cruising altitude is open my laptop or tablet to read through saved technical articles, documentation, or past notes. In-flight Wi-Fi is either non-existent, prohibitively expensive, or throttled to the point where simple page requests time out.