Status: runbook (actionable) · Date: 2026-06-04 · Author: docs/scribe agent
Owner action required: yes (registration token, launchd install, token scoping)
Scope: turn a spare Apple-Silicon MacBook into a dedicated, always-on org
self-hosted GitHub Actions runner for the mahmoudholding org, ending the
single-runner SPOF.
CI for all six repos currently leans on one self-hosted runner that lives
on the dev laptop (laptop 1) — see ci-alternatives-proposal.md §3.
That is a single point of failure, and it bit us: when the dev laptop slept, or
its swarm workers wedged, the runner went offline and CI died fleet-wide for
~2 days — every repo's PRs and CD stalled because no runner could pick up jobs.
The fix: a second Apple-Silicon MacBook that does nothing but run the runner, always on, never sleeping, isolated from the dev laptop's user/tokens/work dirs. The dev-laptop runner can stay registered as a fallback, but the dedicated box becomes the primary so day-to-day dev activity (sleep, reboots, wedged swarm) no longer takes CI down.
Complementary to, not the same as,
openclaw-headless-laptop2-proposal.md. That proposal evaluates running an OpenClaw agent gateway (a worker host the swarm dispatches to, phone-controllable) on the same 2nd Mac. This runbook is about the CI runner role only. The same machine can do double duty, but the two roles MUST run as different macOS users with different tokens — a CI runner has repo-write + GHCR push + the VPS SSH key (a large blast radius); the agent gateway must never share those credentials. See §3 and that proposal's §4 cross-reference. This runbook assumes the dedicated-runner user is isolated from any future OpenClaw user.
This is separate from any Hetzner / cloud migration discussion — it uses hardware already owned, at $0 metered compute (self-hosted minutes are not metered).
mahmoudholding)Org-level, not repo-level — one runner serves all six repos (mahmoud-consultancy, europeLogin, claimio, auditPic, valideerleeftijd, developer-portal). Registering per-repo would re-create the same fan-out problem.
ci-runner — that is not the
account anyone logs into to do dev work (see §3).osx-arm64 exist; we
use them to run the agent, and run the builds inside Linux containers.# as the ci-runner user
mkdir -p ~/actions-runner && cd ~/actions-runner
# grab the latest osx-arm64 runner (check github.com/actions/runner/releases for current ver)
curl -o actions-runner-osx-arm64.tar.gz -L \
https://github.com/actions/runner/releases/download/v2.XXX.X/actions-runner-osx-arm64-2.XXX.X.tar.gz
tar xzf actions-runner-osx-arm64.tar.gz
The owner (or anyone with admin:org) mints a short-lived registration token:
gh api -X POST /orgs/mahmoudholding/actions/runners/registration-token --jq .token
(Token expires in ~1h — register promptly.)
cd ~/actions-runner
./config.sh \
--url https://github.com/mahmoudholding \
--token <REGISTRATION_TOKEN_FROM_1.2> \
--name laptop2-ci-arm64 \
--runnergroup Default \
--labels self-hosted,linux,x64,laptop2,dedicated \
--work _work \
--unattended \
--replace
Label rationale (important):
runs-on: [self-hosted, linux, x64] (per
ci-alternatives-proposal.md §3.2). The agent
itself runs natively on arm64 macOS, but the build steps run inside a
linux/x64 container (§4), so advertising linux,x64 is correct for the job
environment the steps see — only if you keep the Linux-container build model.
Do not advertise linux,x64 from a runner that executes steps directly on
bare-metal macOS — the apt-get/Linux-Docker steps would break.laptop2 and dedicated are extra discriminators so you can later pin specific
jobs to this box (or away from the dev laptop) without ambiguity.self-hosted, linux,x64) so jobs fall through to whichever runner is online — that redundancy is
the whole point. Use laptop2 only when you deliberately want this box.No
runs-onedits are needed in the workflows if the dedicated runner advertises the sameself-hosted, linux, x64set the 108 existing jobs already use. The job YAML stays byte-for-byte identical (steps, secrets, GHCR, SSH, Helm).
gh api /orgs/mahmoudholding/actions/runners --jq '.runners[] | {name, status, labels: [.labels[].name]}'
You should see laptop2-ci-arm64 with status: online once the service (§2) is up.
A login-shell ./run.sh dies on logout and does not survive reboot — exactly the
fragility that caused the outage. Run it as a launchd service that auto-starts on
boot and login, wrapped so the machine never sleeps (a past outage was caused
by the dev laptop sleeping).
The Actions runner ships a svc.sh helper that generates and loads a LaunchDaemon:
cd ~/actions-runner
sudo ./svc.sh install ci-runner # installs as the ci-runner user
sudo ./svc.sh start
sudo ./svc.sh status
This creates ~/Library/LaunchAgents/actions.runner.mahmoudholding.laptop2-ci-arm64.plist
(or a system /Library/LaunchDaemons/ entry depending on svc.sh mode) with
KeepAlive + RunAtLoad so the agent restarts if it crashes and comes back after a
reboot.
The runner staying alive is not enough — if the Mac sleeps, the runner is offline. Belt and braces:
a) pmset power policy (run as admin once):
sudo pmset -a sleep 0 # never system-sleep
sudo pmset -a disablesleep 1 # hard-disable sleep (incl. clamshell if on power)
sudo pmset -a displaysleep 10 # screen can sleep; system stays awake
sudo pmset -a autorestart 1 # auto-reboot after power loss
sudo pmset -a womp 1 # wake on network (optional)
sudo pmset -a powernap 0
# verify:
pmset -g
Also: System Settings → Battery / Energy → "Prevent automatic sleeping when the
display is off" ON, and Login Options → Automatic login = the ci-runner user
so an unattended reboot lands back in a working session.
b) A dedicated caffeinate LaunchDaemon as a second layer, so even if a macOS
update resets pmset, the box stays awake. Install
/Library/LaunchDaemons/com.mahmoudholding.ci-caffeinate.plist:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
"http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>com.mahmoudholding.ci-caffeinate</string>
<key>ProgramArguments</key>
<array>
<string>/usr/bin/caffeinate</string>
<string>-dimsu</string> <!-- d:display i:idle m:disk s:system u:user-active -->
</array>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
<key>StandardOutPath</key>
<string>/var/log/ci-caffeinate.log</string>
<key>StandardErrorPath</key>
<string>/var/log/ci-caffeinate.log</string>
</dict>
</plist>
Load it:
sudo chown root:wheel /Library/LaunchDaemons/com.mahmoudholding.ci-caffeinate.plist
sudo chmod 644 /Library/LaunchDaemons/com.mahmoudholding.ci-caffeinate.plist
sudo launchctl bootstrap system /Library/LaunchDaemons/com.mahmoudholding.ci-caffeinate.plist
sudo launchctl enable system/com.mahmoudholding.ci-caffeinate
# verify it's running:
sudo launchctl print system/com.mahmoudholding.ci-caffeinate | grep -i state
pgrep -fl caffeinate
Why both
pmsetandcaffeinate?pmsetis the policy;caffeinateis an active assertion that survives policy resets (OS updates love to re-enable sleep). The prior fleet-wide outage was a sleep event — this redundancy is deliberate.
Mirror the swarm's watchdog/keepalive pattern: a small launchd timer that checks
gh api /orgs/mahmoudholding/actions/runners for this runner's status: online and
sudo ./svc.sh starts it if it has gone offline. Keep it simple; the runner's own
KeepAlive already covers process crashes — this covers the rarer "registered but
wedged" state.
The whole point is that dev-laptop activity can't take CI down, and that a compromise of one box doesn't hand over the other's credentials. Enforce hard separation:
| Dimension | Dev laptop (laptop 1) | Dedicated runner (laptop 2) |
|---|---|---|
| macOS user | the owner's interactive account | dedicated ci-runner user, no interactive dev use |
| Work dir | swarm + project checkouts under ~/projects/prive | runner's own ~/actions-runner/_work only |
| Runner token | its own registration | its own registration (don't copy the dev laptop's .runner/.credentials) |
| VPS SSH key | present (dev convenience) | a separate ~/.ssh/id_ed25519 minted for CI, added as a distinct authorized key on the VPS, so it can be revoked independently |
| GHCR push | GHCR_TOKEN org secret (used by jobs) | same org secret via Actions — but the host never stores a long-lived PAT outside the runner's job env |
| 1Password | owner session | no prod op session on the box; sealing reads secrets inside the job via OP_SERVICE_ACCOUNT_TOKEN (org secret), never a host-resident token |
| OpenClaw (if added later) | n/a | a third, separate macOS user + its own scoped GH token — never the ci-runner user (see openclaw proposal §4) |
Concrete isolation steps:
ci-runner as a Standard (non-admin) user; grant admin only for the
one-time pmset/launchd install, then remove it from admins.sarkoutmahmoud@136.144.174.219) as a separate authorized_keys
line — so revoking laptop 2 doesn't disturb the dev laptop's access.~/projects/prive or the swarm .swarm/ onto laptop 2. The runner
checks out only what each job's actions/checkout pulls.GHCR_TOKEN,
OP_SERVICE_ACCOUNT_TOKEN, VPS_SSH_KEY) injected at job time — never as files
the host persists.This is the one genuinely new wrinkle versus the laptop-1 runner. The dev-laptop
runner runs builds in a linux/x64 container on an x64-capable host; the 2nd
Mac is arm64, so any step that builds a linux/amd64 image natively must be
QEMU-emulated — and M-series QEMU amd64 image builds TIME OUT (observed
~47 min for a frontend image, vs a few minutes native). If we naively keep
forcing linux/amd64 frontend images on this box, frontend CD will fail.
Backends are JVM (arch-independent bytecode in a JRE base image) — those build fine. The pain is the Angular frontend Dockerfiles, whose Node tooling stage is the slow part under emulation.
Good news — the Node stage is already arch-pinned to the builder. Issue #240
already pinned the ng build stage to $BUILDPLATFORM so the Node tooling runs
on the builder's native arch rather than under emulation. Confirmed in-tree, e.g.
mahmoud-consultancy/frontend/recruitment-portal/Dockerfile:
# Pinned to $BUILDPLATFORM so the Node tooling always runs on the builder's
# native architecture ...
FROM --platform=$BUILDPLATFORM node:20-alpine AS build
...
RUN npm run build # ng build emits arch-independent static JS/CSS/HTML
Because ng build emits arch-independent static assets, the build stage no
longer needs emulation — it runs native on whatever the builder is. The remaining
question is the final/runtime image platform (the nginx serving stage): if CD
insists on publishing a linux/amd64 runtime image, the final stage and any
docker buildx --platform linux/amd64 still emulate on this arm64 Mac.
Option A — publish NATIVE-arch (arm64) images and run them on an arm64 deploy
target. Build linux/arm64 images natively on the 2nd Mac (fast, no QEMU). This
is the cleanest if/when the deploy target can run arm64. Caveat: the production
target is the TransIP VPS, which is x86_64 — so arm64 runtime images won't run
there today. Use Option A only for arm64-capable targets; for the current VPS,
prefer Option B or C.
Option B — keep amd64 runtime, but offload amd64 to a REMOTE buildx builder
(RECOMMENDED for the current x86_64 VPS). Register a remote docker buildx builder
backed by an x86_64 host and let this Mac drive the build over the network — the
amd64 layers build natively on the remote, no QEMU on the Mac. Candidates for the
remote amd64 builder:
docker build there
would re-trigger it; see node-capacity-guardrail-proposal.md.)# one-time: create a remote amd64 builder over an SSH docker context
docker context create amd64-builder --docker "host=ssh://user@<amd64-host>"
docker buildx create --name remote-amd64 --driver docker-container \
--platform linux/amd64 amd64-builder --use
# CD build then targets it:
docker buildx build --builder remote-amd64 --platform linux/amd64 \
-t ghcr.io/mahmoudholding/<project>/<service>:sha-<sha> --push .
Option C — multi-arch manifest, build each arch on its native runner. Keep both the dev-laptop (x64-capable) runner and this arm64 Mac, and let buildx assemble a multi-arch manifest where each platform builds on a matching native runner. Heaviest to wire; only worth it if you genuinely need both arches published.
$BUILDPLATFORM pin already removed the
build-stage emulation cost; B removes the runtime-stage one.--platform linux/amd64 to build
natively on this Mac will QEMU-emulate and time out (~47 min) → frontend CD red.Runner online:
gh api /orgs/mahmoudholding/actions/runners --jq \
'.runners[] | select(.name=="laptop2-ci-arm64") | {status, busy, labels:[.labels[].name]}'
Expect status: "online".
Survives reboot: reboot the 2nd Mac; without logging in interactively, confirm
the runner returns to online (auto-login + launchd) within a couple of minutes.
Survives sleep pressure: close the lid / leave idle past the old sleep window;
confirm pmset -g assertions shows a PreventUserIdleSystemSleep assertion
(from caffeinate) and the runner stays online.
A real job goes green here: push a trivial commit on a feature branch in one
repo, watch the job get picked up by laptop2-ci-arm64, run, and pass. Confirm
GHCR push + VPS SSH steps succeed (proves the CI SSH key + GHCR token path).
Frontend CD specifically: trigger a frontend build/deploy and confirm it completes in minutes, not ~47 min (proves the §4 amd64 strategy works and QEMU is not in the hot path).
Fallback intact: stop this runner (sudo ./svc.sh stop) and confirm a job
still runs on the dev-laptop runner — i.e. you've added redundancy, not a new SPOF.
The change is purely additive — registering a runner doesn't modify any workflow
YAML (labels match the existing self-hosted, linux, x64 set), so rollback is clean:
Drain & remove the runner from the org:
cd ~/actions-runner
sudo ./svc.sh stop && sudo ./svc.sh uninstall
gh api -X POST /orgs/mahmoudholding/actions/runners/remove-token --jq .token # get a removal token
./config.sh remove --token <REMOVAL_TOKEN>
(Or remove from the GitHub UI: Org → Settings → Actions → Runners.)
Restore sleep policy (if repurposing the Mac):
sudo launchctl bootout system/com.mahmoudholding.ci-caffeinate
sudo rm /Library/LaunchDaemons/com.mahmoudholding.ci-caffeinate.plist
sudo pmset -a disablesleep 0 && sudo pmset -a sleep 1
Revoke the CI SSH key on the VPS (remove its authorized_keys line) and let
any CI-only PAT/token expire.
CI immediately falls back to the dev-laptop runner — the exact pre-change state. Because nothing in the repos changed, no PR/revert is needed.
| Action | Command |
|---|---|
| Org registration token | gh api -X POST /orgs/mahmoudholding/actions/runners/registration-token --jq .token |
| List org runners | gh api /orgs/mahmoudholding/actions/runners --jq '.runners[]\|{name,status}' |
| Install service | sudo ./svc.sh install ci-runner && sudo ./svc.sh start |
| Service status | sudo ./svc.sh status |
| No-sleep policy | sudo pmset -a sleep 0 disablesleep 1 autorestart 1 |
| Check sleep assertions | pmset -g assertions |
| Remote amd64 builder | docker buildx create --driver docker-container --platform linux/amd64 <amd64-ctx> --use |
| Removal token | gh api -X POST /orgs/mahmoudholding/actions/runners/remove-token --jq .token |
Cross-references:
ci-alternatives-proposal.md (why self-hosted, the
108-job runs-on landscape) ·
openclaw-headless-laptop2-proposal.md
(the other, separate role the same 2nd Mac can host — keep users/tokens isolated) ·
node-capacity-guardrail-proposal.md (why
builds must never land on the TransIP VPS).
Reacties