Name. The product is OpenAB Computer (repository
openabdev/openab-computer, tracked in #63). Since 0.12.0 the executable, bundle id (dev.openab.computer), LaunchAgent, systemd units, directories and release assets are all namedoab-computer. It wasoab-instance-mcpup to 0.11; see Upgrading from 0.11 or earlier. Commands in this README use the new names.
The "hands and feet" half of openabdev/openab#1544: a thin daemon that lives in a Mac's logged-in desktop session and exposes the machine to a coding CLI or agent elsewhere on the tailnet as MCP servers. Nothing here is smart; all the intelligence stays in the caller.
Read before deploying.
execis an unrestricted shell as the logged-in user, andmouse,keyandosascriptdrive the desktop. There are two profiles:observe(sys_infoandscreenshot) andowner(every local tool and browser tool). Every/mcpand/attachrequest, and every switchboard caller, must present a bearer token. A switchboard can also keep a per-caller allow-list.owneris not a security boundary (#45). Run this only on a Mac you own, for callers you would trust with your keyboard.Installing the
.pkg(orinstall-prebuilt.sh) runstailscale serve --bg --https=8444, so the Mac is reachable on your tailnet at:8444right after the install. A caller still needs the bearer token and an allowed Tailscale login. A plain app launch without the installer creates no mapping until you turn tailnet access on, but it leaves an existing:8444mapping in place. See Download and install.
flowchart LR
subgraph callers["Callers, where the model runs"]
cli["Coding CLI or agent with an MCP client"]
connect["OpenAB Connect"]
end
sb["Switchboard openab-sb, one bearer token per caller"]
rt["Agent runtime that was lent this Mac, reverse attach"]
subgraph mac["The Mac, with a desktop user logged in"]
ts["tailscale serve, https port 8444, TLS and the Tailscale login header"]
subgraph app["oab-computer, LaunchAgent dev.openab.computer"]
ep["HTTP endpoint 127.0.0.1 port 8795, paths mcp and attach and healthz. Callers are checked as described under Auth"]
mcp["MCP server, shared by the HTTP endpoint and both dial-out clients"]
tools["Local tools, sys_info exec screenshot ui_snapshot ui_action mouse key osascript"]
router["BrowserRouter, one Playwright slot, 32 browser tools, owner profile only"]
dial["Dial-out clients, switchboard and reverse attach"]
end
managed["Managed Playwright when installed, a node child over stdio with no port. Used when --upstream browser is not given"]
ext["External Playwright MCP, 127.0.0.1 port 8794, a separate process. Used instead when --upstream browser is given"]
ae["Apple Events tier, planned and not built"]
os["Shell, display, input and apps of the logged-in user, gated by TCC grants"]
end
cli -- "HTTPS, MCP over HTTP" --> ts
connect --> ts
ts --> ep
cli -- "HTTPS, MCP through the switchboard" --> sb
dial -- "WSS dialed out to vm attach" --> sb
sb -- "tool calls down that socket" --> dial
dial -- "WS dialed out after POST attach" --> rt
rt -- "tool calls back down the same socket" --> dial
ep --> mcp
dial --> mcp
mcp --> tools
mcp --> router
router --> managed
router --> ext
router -.-> ae
tools --> os
Read it left to right: the model and its reasoning live in the caller; the machine only receives
tool calls, and the daemon can do nothing a logged-in user sitting at it could not do. Without a
switchboard, everything on the right is one machine and no cloud hop: tailscale serve terminates
TLS and stamps each request with the caller's Tailscale identity, which the daemon checks together
with a bearer token. The switchboard path (--switchboard) is for a Mac that can only dial out.
The daemon dials the switchboard, and the switchboard routes caller requests down that socket. On
macOS the tools that touch screen or input are gated by TCC, granted once to the stable bundle id.
The loop the tools are built for: screenshot → decide →
mouse / key / osascript / exec → screenshot to confirm. SSH already gives you a shell;
this exists for what SSH cannot reach.
The browser part of the diagram is covered in Browser tools. The
Apple Events tier is drawn with a dotted arrow because it is planned and not built. The diagram shows
the macOS app. The Linux node is described in docs/linux-setup.md.
- Goal · Features · Status and known limits · Quick start · Platforms
- Tools · Auth · Network interfaces and ports
- Reverse attach · Switchboard · Browser tools
- Deploy & operate: Download and install · Upgrading from 0.11 · Build & test · Deploy · Menu bar · Operate
- Operator notes: TCC & signing · Gotchas
- Detail: Docs index · Upgrade guide · Browser tools guide · UI tools · Operator notes and gotchas
Components: oab-computer is this package (Swift on macOS, zero deps; Rust on Linux). Browser
tools come from @playwright/mcp, which is not our code. On macOS the app installs, runs and
removes it for you (see Browser tools); an
external one is the advanced option (poc/pw-mcp/README.md).
Design notes: docs/requirements/connect-closed-loop.md.
Let agents on your tailnet use your own computers, a Mac mini, a Linux mini PC or a Raspberry Pi,
once you have authorized or paired them with a token and a tool profile. An agent can drive the
browser and everything else you would do on that machine: build, compile and test, use the desktop,
and run everyday tasks. Each grant is per agent and can be revoked. The observe profile gives
read-only access (screenshot and system info), and owner gives full control. On macOS the full
tool set is available, including the accessibility-tree tools. On Linux the browser, shell,
screenshot, mouse and key tools are available, but the accessibility-tree tools are not yet
(#102).
- Browser: a Playwright-driven Chromium on the machine.
browser_snapshotreturns the page as an accessibility tree. Thebrowser_*tools need theownerprofile. - Shell, build and test:
bashon Linux,execon macOS. Background jobs (exec_start,exec_poll,exec_list,exec_cancel) run long builds, and they keep running if the connection drops. - Desktop control: screenshots, mouse and keyboard. AppleScript (
osascript) is macOS only. - Accessibility tree (macOS):
ui_snapshotreads a window's controls with refs.ui_actionpresses or sets them by ref without moving the pointer. - Access control: a bearer token per caller. A caller gets the
observeprofile (read-only:sys_infoandscreenshot) orowner(full control). A switchboard can also keep a per-caller tool allowlist. - Reverse attach: lend a machine to a sandboxed agent session with a time-limited grant that can be revoked.
- Switchboard: the machine dials out to a relay, so callers reach it without an inbound port on the machine.
- Platforms: macOS (signed, notarized
.pkg) and Linux nodes such as a Raspberry Pi or a mini PC, installed with an installer script.
The accessibility-tree tools are macOS only for now; Linux is deferred (#102).
- macOS has
ui_snapshot(0.9.0) andui_action(0.10.0). Linux has neither yet; the AT-SPI port is deferred (#102). owneris not a security boundary (#45). Onlyobserveis.- Switchboard callers are authorized by bearer token only. Binding a caller to its tailnet identity is open (openabdev/openab-sb#20).
- Switchboard audit logs record
ui_actionvalues andkeytext in full unless openab-sb 0.2.2 or later is in use (#14, #16). - Browser tools use one persistent browser profile per computer, with its cookies and logins.
- The browser router in 0.12.0 has only the Playwright backend. The Apple Events tier is not built, and neither is the "use my installed Chrome" fallback if macOS refuses the downloaded Chromium (#128).
- Linux has no managed browser tier (designed in #119,
not built; ADR). Playwright there is an external unit on
127.0.0.1:8794. - The Linux installer still requires
wlrctl, which Ubuntu 24.04 packages only for amd64 (universe). On Ubuntu arm64 the installer exits 78, so install the units by hand (Linux setup, steps 3-5) (#130).
macOS — download oab-computer-VERSION-universal.pkg from
Releases, double-click, and follow the
permissions wizard (details). Then point a caller at it:
kiro-cli mcp add --name mac-mcp --url https://<host>.<tailnet>.ts.net:8444/mcp \
--header "Authorization: Bearer $(ssh <host> cat ~/.config/oab-computer/token)" \
--scope global --timeout 30000Through a switchboard (the usual path for a Mac that dials out): the caller uses the
switchboard's per-computer route and its own bearer token. The first call is initialize. If the
response sets Mcp-Session-Id, send that header on later calls.
curl -s https://<switchboard>.<tailnet>.ts.net/mcp/<computer> \
-H "Authorization: Bearer <caller-token>" \
-H "Content-Type: application/json" \
-H "Accept: application/json, text/event-stream" \
-d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-06-18","capabilities":{},"clientInfo":{"name":"curl","version":"0"}}}'Linux (Raspberry Pi, Intel mini PC) — one installer does the whole node (full guide):
V=0.12.0 # the first release with oab-computer-* assets (v0.11.0 and older use the old names)
A=$(dpkg --print-architecture) # arm64 | amd64
curl -fsSLO "https://github.com/openabdev/openab-computer/releases/download/v$V/oab-computer-$V-linux-$A.tar.gz"
tar xzf oab-computer-$V-linux-$A.tar.gz && cd oab-computer-$V-linux-$A
./install-linux.sh # a box with no desktop: add --headless-seatThe URL is https://<host>.<tailnet>.ts.net:8444/mcp and the bearer token is in
~/.config/oab-computer/token on the node.
| macOS | Linux | |
|---|---|---|
| Implementation | Swift (Network.framework + ScreenCaptureKit) | Rust (poc/reverse-attach-linux/) |
| Screen / input | ScreenCaptureKit · CGEvent, gated by TCC | grim (JPEG encoded in-process) · wlroots virtual pointer · wtype on a wlroots seat (labwc, sway) |
| Shell | exec / exec_start* (zsh -f) |
bash / exec_start* (bash -c) |
| Apps | osascript / JXA |
— (use bash: gdbus, xdg-open) |
Accessibility tree (ui_snapshot / ui_action) |
✅ (0.9.0 / 0.10.0) | deferred (#102) |
| Browser tools | managed Playwright (installed from the wizard, stdio child, no port) or external via --upstream |
external Playwright unit on loopback 8794; a managed tier is designed (#119) and not built |
| Reverse attach | ✅ | ✅ |
| Install | signed/notarized .pkg |
-linux-{arm64,amd64}.tar.gz + install-linux.sh |
| Guide | this README | docs/linux-setup.md |
The rest of this README is the macOS reference; the two platforms share the wire contract, auth
and reverse-attach shape, so callers configure them identically. On Linux the job tools
(exec_start, exec_poll, exec_list, exec_cancel) take the same arguments and return the
same structuredContent; their logs are under $XDG_STATE_HOME/oab-computer/jobs/
(default ~/.local/state/…), and they are owner-only there too
(docs/linux-setup.md).
| tool | what | notes |
|---|---|---|
sys_info |
host, OS, chip, displays, tailnet IPs, console user, TCC status | call first; tells the model what will work |
exec |
zsh -f -c <command> as the desktop user |
cwd (a leading ~ is expanded and validated — a missing dir or unreadable external volume returns a clear error, not a hang), env, timeout_secs (≤600), max_output_bytes (≤1 MiB/stream). Spawned with POSIX_SPAWN_SETSID; timeout → killpg → exit 137, timed_out=true. structuredContent carries exit/stdout/stderr/duration. Use for commands that finish in seconds |
exec_start |
start a background job, return a job_id immediately |
for work that outlives one request (release builds). Same shell/session/TCC as exec; cwd, env, timeout_secs (0 = no timeout, stop via exec_cancel). stdout/stderr are tee'd to ~/Library/Logs/oab-computer/jobs/<job_id>.out/.err (never truncated; survive after the job is forgotten; tail -f-able) |
exec_poll |
fetch a job's state + incremental output | job_id, optional stdout_since/stderr_since byte offsets (from the prior poll) for only-new output; reads from the log files by seek. Terminal state (exited/killed) carries exit_code; out_path/err_path point at the files |
exec_list |
all running jobs + the 10 most recently finished | recover a forgotten job_id or see what is running; each entry has state, pid, exit_code, command, cwd, timestamps, stdout/stderr byte sizes |
exec_cancel |
stop a running job (or drop a finished one) | job_id, signal KILL (default) / TERM — signals the whole process group; poll once more for final output. Finished jobs are dropped from the registry (log files stay on disk) |
screenshot |
ScreenCaptureKit → JPEG/PNG as MCP image content |
display, scale (px per point, default 1.0), region {x,y,w,h} crop in points, format, quality. Needs Screen Recording TCC. Read small UI text with region + scale: 2 |
ui_snapshot |
Accessibility tree of a native app's window, read-only | Read-only. owner only. Targets the frontmost window by default; arguments, bounds and masking are in docs/ui-tools.md. Needs Accessibility TCC. macOS only |
ui_action |
Act on a ui_snapshot node by ref, without moving the pointer |
owner only. Actions: press, set_value, focus, show_menu, confirm, cancel, increment, decrement. Refs are tied to a snapshot; stale refs are refused. No pointer movement; activate: true is opt-in. Secure fields are refused over a switchboard by default. Full detail: docs/ui-tools.md. Needs Accessibility TCC. macOS only |
mouse |
CGEvent: move click double_click right_click drag scroll |
coordinates in display points = screenshot pixels at scale 1. modifiers. Needs Accessibility TCC |
key |
CGEvent: type (unicode, layout-independent) / press combos (cmd+shift+4) |
Needs Accessibility TCC |
osascript |
AppleScript / JXA via /usr/bin/osascript in the GUI session |
timeout_secs (default 15). First script against an app raises an Automation consent dialog on the Mac — screenshot, then mouse click Allow |
browser_* (32) |
Playwright browser tools, owner only (observe sees none) |
On macOS they go through the browser router to the managed or external Playwright; on Linux they come from the external Playwright (MCP_UPSTREAM). Names are listed below the table. browser_evaluate and browser_run_code_unsafe run arbitrary JavaScript in the computer's persistent profile. See Browser tools and docs/tool-profiles.md |
The 32 browser_* names, grouped by how well the planned Apple Events tier could serve them
(BrowserToolCatalog). That tier is not built in 0.12.0, so today every call goes to Playwright and the
grouping only shows what a later tier could cover:
| Planned Apple Events support | Tools |
|---|---|
| full | browser_navigate, browser_navigate_back, browser_fill_form, browser_select_option, browser_wait_for, browser_close |
| limited | browser_snapshot, browser_find, browser_click, browser_type, browser_press_key, browser_hover, browser_tabs, browser_resize, browser_evaluate, browser_mouse_wheel |
| none | browser_take_screenshot, browser_console_messages, browser_run_code_unsafe, browser_file_upload, browser_drop, browser_pdf_save, browser_network_requests, browser_network_request, browser_handle_dialog, browser_emulate_media, browser_drag, browser_mouse_move_xy, browser_mouse_click_xy, browser_mouse_drag_xy, browser_mouse_down, browser_mouse_up |
Which tool reads the screen: browser_snapshot for web pages, ui_snapshot for native apps
(exact roles, titles and frames, cheaper than reading an image), and screenshot + mouse for
custom-drawn or canvas content that has no accessibility tree. A ui_snapshot contains field
values and on-screen text (secure text fields are masked, other fields are not), so a caller
that stores snapshots must mask them first. An element whose role cannot be read is kept as a
leaf with no value and no children, since it cannot be ruled out as a secure field.
To act on a native app without taking the pointer: ui_snapshot → ui_action {snapshot_id, ref, action} → ui_snapshot again to confirm. set_value replaces the whole value, and some apps
commit a field only on confirm or when focus moves. Drags, gestures and custom-drawn content
that has no accessibility node stay on screenshot + mouse. Ref containment and stale-ref
rules are in docs/ui-tools.md.
The loop the tools are designed for: screenshot → decide → mouse/key/osascript → screenshot to confirm.
Verified 2026-09-22 from the laptop: open TextEdit, type a line, read it back, close via the save
sheet's Delete button located from a screenshot, quit — 0.15–0.8 s per call.
Planned: a streaming capture sink for OpenAB Connect (see requirement doc).
Evaluated per request in AuthPolicy; the server refuses to start with nothing configured.
Configured checks are AND-combined: a request must pass every check that is set.
--allow-login <email>— matchesTailscale-User-Logincase-insensitively (both sides are lowercased; the principal logged is the lowercased login), whichtailscale serveinjects and overwrites (verified: a client-supplied header is replaced). This is the primary control.--token <s>/--token-file <p>— additionally requireAuthorization: Bearer, constant-time compared. The token after theBearerscheme (case-insensitive) must match byte for byte; whitespace around the whole header value is stripped by the HTTP parser, nothing inside it is.deploy.shnow generates one at~/.config/oab-computer/token(mode 600) and passes--token-file, so the deployed agent requires both an allow-listed Tailscale login and the bearer token. This is defence-in-depth: a leaked tailnet auth key that enrols a node as your Tailscale login passes the login check but still gets401without the token (verified 2026-09-23: no token → 401, correct token → 200, wrong token → 401). The token is stable across re-deploys; rotate by deleting~/.config/oab-computer/token(macOS) and re-deploying. The first 0.12 run copies a 0.11 token from~/.config/oab-instance-mcp/tokenonce, when~/.config/oab-computeris created; after that the old file is never read again, so a deleted token is not restored from it. The menu bar shows a masked form and copies the full token (see below); it is never written toagent.log.--insecure-local— allow bare loopback requests with no Tailscale headers. Debugging only./healthzis unauthenticated and says onlyok.
Trust model: exec is a full shell as the logged-in user. /mcp is "my CLI on my Mac". A
sandboxed agent never gets /mcp; it gets a reverse attach under the observe or owner
profile (below).
This section describes v0.12.0 (and main at the time of writing). Checked by reading the code and by a read-only look at a running agent
(an older 0.10.1 install) and a bare 0.12.0 launch. The menu bar item's Network section shows the
same facts (see What the menu shows).
The app runs tailscale serve in three cases, all for port 8444 only: (a) serve --bg --https=8444 ...
after you turn tailnet access on, from the wizard's tailnet page or the menu; (b) the same command at
launch, to re-apply the mapping when you had already turned access on and the mapping is missing; (c)
serve --https=8444 off after you turn access off, and as an automatic safety removal when no
Tailscale login is detected and no --allow-login is configured. Every other Tailscale call is
read-only.
| Address:port | Protocol | Who can reach it | What it is for | How it is enabled | Auth |
|---|---|---|---|---|---|
127.0.0.1:8795 (--host, --port; --path for the /mcp route) |
TCP, plain HTTP/1.1 | Processes on this Mac only | /mcp (POST, DELETE), /attach (reverse attach; off with --no-attach), /healthz. Anything else is 404 |
Always, on IPv4 loopback by default. No-flag launch from Finder, the installer's LaunchAgent, or a manual launch. A no-flag launch also briefly binds the port to test it is free, without listening | /mcp and /attach: Auth. /healthz: none |
https://<host>.<tailnet>.ts.net:8444 → 127.0.0.1:8795 |
HTTPS, terminated by Tailscale | Your tailnet only | The way other machines reach the agent | scripts/install-prebuilt.sh (and the .pkg) run tailscale serve --bg --https=8444. The app runs the same command only after you turn tailnet access on, from the wizard's tailnet page or the menu's "Allow tailnet access…", and removes it with "Turn Off Tailnet Access". A launch with no stored opt-in does not create it. A launch with a stored opt-in re-applies it when it is missing. A mapping that already exists is shown as "On (existing mapping, not managed by this app)" and left alone until you choose "Manage with this app" |
Every configured check must pass. The installer configures both: a Tailscale login on the allow list and the bearer token |
127.0.0.1:8794 |
TCP, HTTP | Processes on this Mac only. Other machines reach the browser tools only through the app's --upstream |
Playwright MCP, a separate process the app only connects to (--upstream). It has no :8443 or other tailnet access path |
Its own LaunchAgent (poc/pw-mcp/install.sh), which binds --host 127.0.0.1 and creates no tailscale serve mapping. The app never starts this one. The managed Playwright (see Browser tools) is a different thing: a child over stdio that listens on nothing |
None in Playwright. The app's Auth applies to callers before it relays browser tools |
The macOS app opens one listening socket. It uses no UDP and no Unix domain sockets. The :8444
mapping is made by the installer, by the app after you turn tailnet access on, or by the app's launch
re-apply of that choice. Nothing in this repo maps :8443 any more. Older versions of
poc/pw-mcp/install.sh did (tailscale serve --bg --https=8443 http://127.0.0.1:8794), which put
the unauthenticated Playwright MCP on your whole tailnet. Re-running the new installer does not
remove it. Check for it and remove it on any Mac that ran the old installer:
tailscale serve status # look for :8443 pointing at http://127.0.0.1:8794
tailscale serve --https=8443 off # remove only that mappingBrowser tools keep working afterwards, because the app reaches Playwright over loopback. The new
installer runs the read-only serve status at the end and prints a warning if it finds the mapping.
| What | To where | When | What it is for |
|---|---|---|---|
| On-demand Playwright install, HTTPS | The app's own downloads (Node, Chromium) go to nodejs.org and storage.googleapis.com and refuse redirects to any other host. The downloaded Node's npm ci then connects to registry.npmjs.org; the app does not bound or inspect npm's connections or redirects, and the lockfile pins every package to that host and to a sha512 digest. The install's final health check also starts the installed Chromium once, and its own connections are Chromium's (see below) |
Only when you choose to install or update Playwright (the install step is offered, pre-checked, and you can uncheck it). Never at launch, never in the background | Download Node, the pinned @playwright/mcp and the pinned Chromium. See What the app downloads |
| Upstream MCP, HTTP(S) | The URL given to --upstream. The installer uses http://127.0.0.1:8794/mcp. The app does not require loopback |
Only with --upstream. The installer adds it when the Playwright agent is loaded |
Browser tools |
| Switchboard, WebSocket | wss://<switchboard-host>.<tailnet>.ts.net/vm/attach. ws:// only to loopback |
Only with --switchboard or a saved switchboard config. Redials for the life of the process |
Serving a switchboard |
| Reverse attach, WebSocket | ws(s)://<runtime>/tools/attach/<session>. The runtime and port come from the grant |
Only after a POST /attach. Saved grants are redialled at launch |
Reverse attach |
| Mint call, HTTP(S) | http(s)://<runtime>/admin/sessions/<session>/tools-attach |
Only when the /attach body carries an admin_credential |
Get a one-time attach secret |
tailscale status --self --peers=false --json |
Local child process (read-only) | On the no-flag first-run launch, on every menu-bar launch with an allow list (to read the login), when the app reads the existing mapping at launch, when you turn tailnet access on, and in the recheck while no Tailscale login is detected | Find your Tailscale login and tailnet name, and tell whether tailscaled is running |
tailscale serve status --json |
Local child process (read-only) | When the app checks the :8444 mapping: at launch, when you turn tailnet access on or off, and on the recheck while no Tailscale login is detected |
See whether :8444 is mapped, and to what |
tailscale serve --bg --https=8444 http://127.0.0.1:8795 and tailscale serve --https=8444 off |
Local child process (changes the mapping) | When you turn tailnet access on or off in the wizard or the menu. If you had turned it on, a launch re-applies it once when the mapping is missing. With no Tailscale login detected and no --allow-login configured, launch and the recheck run serve --https=8444 off if :8444 still maps to this app |
Create or remove the tailnet HTTPS mapping on :8444 |
The menu bar and the permission wizard also start local processes (open, launchctl, a
--permission-check re-run of the app). These make no network connections.
Nothing is bundled in the app and nothing is fetched until you ask. The on-demand Playwright install
downloads three things into ~/Library/Application Support/oab-computer/playwright/ (never inside
the app bundle), for the architecture of this Mac:
| What | From | Version |
|---|---|---|
Node.js (includes npm) |
https://nodejs.org/dist/v24.22.3/ |
24.22.3 |
@playwright/mcp and its dependencies (3 npm packages) |
https://registry.npmjs.org/, fetched by the downloaded Node's npm ci --ignore-scripts |
0.0.82 (with playwright-core 1.64.0-alpha-1789764292000) |
| Chromium (Chrome for Testing) | https://storage.googleapis.com/chrome-for-testing-public/154.0.8037.0/ |
154.0.8037.0 |
Download: about 249 MB (Apple silicon) or 260 MB (Intel). On disk once installed: about 593 MB (Apple silicon) or 606 MB (Intel). While it runs, the install needs the downloads plus the unpacked files, so about 842 MB of free space on Apple silicon (about 867 MB on Intel), and an internet connection. The check is the larger of three times the download and the download plus the unpacked size. It is not the 593 MB figure: that is what remains afterwards.
- Every file is checked against sizes and SHA-256 digests built into the signed app, before it is
extracted. The npm packages are checked against a lockfile built into the app (
sha512integrity for each package, all fromregistry.npmjs.org). The downloaded Node must also pass a code-signature check against the Node.js release team. A mismatch installs nothing. - Archives are checked before anything is written (no absolute paths,
.., links that leave the folder, devices, or oversize entries) and are extracted by the app itself, not bytarorunzip. npmmakes its own connections toregistry.npmjs.org; the app's host list does not boundnpm, which is why the lockfile allows no other host.- The app never adds or removes
com.apple.quarantine. If macOS refuses to run the downloaded browser, the install stops with that message. - The Chromium it installs is a browser: the connections it makes while running (the pages it opens, and any background service Chrome for Testing contacts) are Chromium's, not the installer's, and the app does not bound them. On the pinned build it opened no listening TCP or UDP socket, and it exits when its stdin closes.
- The install adds no LaunchAgent and no listener. An update is only ever offered, never applied
silently. Removing Playwright deletes the version folders that carry the installer's marker (even when
install.jsonis unreadable, invalid or hand-made), and also the browser profile and its cookies when the installer's own, readableinstall.jsonconfirms it created that profile (a profile or output folder that already existed is never used: install stops and leaves it alone). A hand-madeinstall.jsonis never replaced or removed: install refuses and remove leaves it alone (a version folder carrying the installer's marker is still removed). Aninstall.jsonthat is unreadable or invalid proves nothing, so remove leaves it and the profile and output folder too, even if this app created them. - The numbers above come from the manifest in
release/playwright-pins/; the release check (scripts/playwright-pins.py, run in CI) fails if they drift.
This is about the macOS app process. Tools such as exec start child processes that can do anything
your user can.
- It opens no UDP socket and no Unix domain socket.
- It does not bind a non-loopback address unless you pass
--host. - It serves no TLS. Tailscale terminates TLS in front of it.
- It never runs
tailscale funnel,up,setorcert. It runstailscale serve(andserve … off) only for port 8444. It does so after you turn tailnet access on or off, for the one launch re-apply of a choice you already made, and for one safety removal: when no Tailscale login is detected and no--allow-loginis configured, at launch and on its recheck, it runstailscale serve --https=8444 offif:8444still maps to this app, because the bearer token alone would then guard that path. - Nothing in the menu's Network section probes a port or changes a setting, except the explicit tailnet actions the menu offers. The rows are read from the app's own state.
- On the switchboard path it dials out and opens no extra listener.
- It downloads nothing at launch and nothing in the background. The only download is the on-demand Playwright install, which runs when you ask for it and is listed under What the app downloads.
- It never adds or removes
com.apple.quarantineon anything it downloads.
The tailnet mapping belongs to tailscaled. The installer creates it, or the app does when you turn
tailnet access on (and re-applies it at launch if you had turned it on and it is missing). Removing it (tailscale serve --https=8444 off, or "Turn Off Tailnet Access" in
the menu) closes the tailnet path and leaves the loopback listener.
A caller must pass every configured check. The installer always sets both, so a tailnet caller needs
an allow-listed Tailscale login and the bearer token. A no-flag launch falls back to the token
alone if it cannot find your login, and so does a manual launch without --allow-login. Playwright MCP
has no auth of its own, so it stays on loopback and is reached only through the app. The app does not
guard --host: a non-loopback value binds in plain HTTP. The menu's Local MCP row then says
"reachable from other devices", but the app does not block it. The app
trusts a Tailscale-User-Login header from any peer, so on a non-loopback bind only the bearer
token stops a forged header. Keep the default, and always set a token.
The menu bar item has a Network section with one row for each of the app's own listener and connections described above. The rows come
from the app's own state (its --host, --port and --upstream options, its tailnet record and its
attach manager), not from a port scan. The menu connects to nothing to fill them in. When it cannot
tell, it says so. The Tailnet row reads "checking…" from launch until the first serve status read
finishes, then "Off", "On" or "unknown (Tailscale status unreadable)"; it never shows "Off" before that read.
Example:
Network
Local MCP: http://127.0.0.1:8795/mcp (this Mac only)
Tailnet: On, https://<host>.<tailnet>.ts.net:8444/mcp (your tailnet only)
Browser tools: 127.0.0.1:8794 (separate process, this Mac only)
Switchboard: not configured
Reverse attach: 0 live grants
The Tailnet row comes only from Tailscale state (the read-only serve status result and your stored
choice), never from --public-url. A --public-url that is not the local URL is shown only on the top URL line. When the app has no
Tailscale state of its own (an installer owns another port), the Tailnet row reads "unknown (not
checked by this app)", whatever the public URL is. The Browser tools row says "this Mac only" about
the app's own --upstream address. Playwright has no tailnet mapping; the note under the table above covers older installs.
Clicking the Local MCP row or the Tailnet URL copies it. Tailnet can also read "Off", "On (existing
mapping, not managed by this app), " or "opted in, not applied (see the log)". Browser tools
shows the external --upstream browser address (and says "(remote)" for a
non-loopback host), or, with no such upstream, the managed Playwright state ("Playwright ready", "Playwright not installed", "Playwright install not usable" or "Playwright unavailable"),
for example Browser tools: Playwright ready. Reverse attach reads "off" with --no-attach. The tailnet actions (Allow, Turn
Off, Manage with this app) and the Switchboard actions (Configure, Reconnect, Disconnect & forget)
stay in the menu: the tailnet actions right after the Tailnet row, the Switchboard actions after the
permissions block.
The Linux node (poc/reverse-attach-linux/) has one daemon listener (127.0.0.1:8795 when installed
by install-linux.sh) and, unless you pass --no-browser, a second loopback listener for
Playwright (8794, no authentication). The differences from macOS:
- It reads
BINDfrom the environment, not flags. A bare binary listens on127.0.0.1:8790.scripts/install-linux.shsets127.0.0.1:8795. /attachis always on. There is no--no-attach.- There is no switchboard and no menu bar.
install-linux.shrunssudo tailscale servefor:8444only with passwordless sudo. Otherwise it prints the command.install-linux.shalso starts Playwright MCP on127.0.0.1:8794(user unitoab-computer-pw-mcp.service) whennodeandnpmare present, and setsMCP_UPSTREAM=browser=http://127.0.0.1:8794/mcp. Skip it with--no-browser. It makes no tailnet mapping for 8794.MCP_UPSTREAMtakes plainhttp://URLs only. Port 8794 is unauthenticated: any local user on this node can call every browser tool there, includingbrowser_run_code_unsafe(arbitrary Node code). On a shared host use--no-browser. A managed Linux tier that removes the listener is designed (ADR) and not built.- The Linux node does use a Unix domain socket: it connects, as a client, to the Wayland socket for display and pointer control. It does not listen on one.
- The mint call is plain
http://only. Awss://runtime withadmin_credentialis refused.
An agent in an openab-pty session cannot reach this Mac — the pod has no tailnet egress by design
(it does reach the internet, for its model API, git and package registries). So
this Mac dials the pod and serves MCP over that socket with a narrowed tool list. Design:
docs/adr/reverse-attach.md; wire contract: openab-pty
CLIENT-CONTRACT.md §9.
Connect / Remote ──POST /attach (your credential)──► this Mac ──WS dials in──► openab-pty pod
└─► CLI in the session
# "lend my Mac to session laptop for an hour, look only"; the Mac mints the attach secret at
# the runtime with its admin credential (used once, not stored) and dials in. Use "owner" for
# full control (shell, input and browser).
curl -s -X POST -H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d '{"runtime":"ws://<pod>.<tailnet>.ts.net:8090","session":"laptop","profile":"observe",
"ttl_secs":3600,"admin_credential":"<openab-pty admin credential>"}' \
https://<host>.<tailnet>.ts.net:8444/attach
# → 202 {"id":"…","profile":"observe","requested_profile":"observe","state":"idle"|"attached",…}
# GET /attach lists · DELETE /attach/{id} revokes/attachuses the sameAuthPolicyas/mcp— the human's tailnet login + bearer. The grant lives on this Mac; the phone can be put away after the tap.- Profiles are per grant and fixed for its life. There are two
(#59):
owner= every local tool and every browser tool;observe=sys_info+screenshotonly. A request withoutprofilegetsobserve. Any other value is a 400profile must be one of [owner, observe], including the removed namesandbox. The retireddesktopruns asobserveand the grant says so (requested_profile,warning); see Upgrading from 0.8.x. Hidden tools are absent fromtools/list, and a forcedtools/callis an unknown tool error. Widening is a new grant. Full per-profile tool lists:docs/tool-profiles.md. ⚠️ owneris not a security boundary; onlyobserveis (#45).ownerincludes a shell in several forms (exec,osascript,key,mouse), and the browser tools run with this computer's logged-in browser profile. Filtering text cannot fix this. When an agent needs full control, lend a dedicated computer (a Linux hands node or a throwaway VM), not the Mac you work on.ProfileBoundaryTestsfails any profile that claims to be narrower while still allowing a shell-capable tool.- Either
secret(already minted by the operator at the runtime) oradmin_credential(the Mac mints, TTL is the runtime's) — exactly one. A new grant for the same runtime+session replaces the old one; that is renewal.ttl_secsis forwarded all the way to the runtime (not just held by the Mac): Connect/Remote offer exactly 1/2/4/12/24 hours; OpenAB Computer accepts up to 24h, and a runtime with a lower operator ceiling rejects the request rather than silently shortening it. - Redial: close codes and backoff follow openab-pty
CLIENT-CONTRACT §9.2.
Tool lists per profile are in
docs/tool-profiles.md. - Liveness and exec: the Mac keeps a receive outstanding during tool calls, so it answers the
runtime's pings, and pings the runtime every 15 s. On these attaches
execrefuses atimeout_secsover 100 s, because the runtime fails any call still running after 120 s; useexec_start+exec_pollfor long work. Nothing on the attached socket is trusted as identity: the grant is the identity, and aTailscale-User-Loginheader there would be pod-supplied. --no-attachdisables the plane (/attach→ 404).
An openab-sb switchboard lets callers (Connect,
agents) reach a computer that can only dial out. This daemon can be that computer: it dials the
switchboard's GET /vm/attach and serves its tools there, under one profile, for as long as it
runs.
openab-sb gen-secret # on the switchboard: verifier → [vm].secret_sha256
oab-computer --token-file ~/.config/oab-computer/token \
--switchboard wss://<switchboard>.<tailnet>.ts.net/vm/attach \
--switchboard-secret-file ~/.config/oab-computer/switchboard.secret \
--switchboard-profile observe # observe (default) | owner- Profile: the profile here is this computer's ceiling. The switchboard's per-caller
allowlist applies on top, so a caller gets the intersection. With
ownerthat allowlist is the only limit. The retireddesktopservesobservewith a warning in the agent log, and the installer writesobservefor aswitchboard.profilefile that still saysdesktop(#59). - Audit exposure: the switchboard writes its audit line before it forwards a call, so this
daemon cannot keep argument values out of that log. openab-sb 0.2.2 or later
(openabdev/openab-sb#16) redacts
ui_actionvalueandkeytextto a redacted length. An older switchboard withaudit_args = truerecords them in full, so do not send secrets throughset_valueorkeyover one. Upgrading does not clean an existing log, so rotate audit files written by an older release. - Secure fields:
ui_actionset_valueon a password field is refused through a switchboard by default, and nothing is typed into the app. To allow it, writeallowto~/.config/oab-computer/switchboard.secure-set-valueand re-run the installer (flag--switchboard-allow-secure-set-value). Do this only when the switchboard hasaudit_args = falseor runs openab-sb 0.2.2 or later. Otherwise, fill secure fields over a direct/mcpconnection. - Transport:
ws://is accepted for loopback only. Anything else needswss://. - Redial: stops on
4002(replaced by another daemon) and4003(secret revoked). Redials with backoff (1 s → 60 s, ±20 %) on4005,1001, drops and proxy errors. Retries every ~5 minutes on401/403instead of stopping. The contract is openab-sbSOUTHBOUND-CONTRACT.md. - Rotation: the secret file is re-read on every dial. Update the file, and the next retry picks it up.
- Installed with the package: put the URL in
~/.config/oab-computer/switchboard.url, the secret inswitchboard.secret(mode 600) and, optionally, the profile inswitchboard.profileandallowinswitchboard.secure-set-value. Then re-run the installer: it writes the flags into the LaunchAgent plist whenever both files exist, so upgrades keep them. - Saved files (menu bar): Configure writes
switchboard.url,switchboard.secretandswitchboard.profiletogether with a digest marker over the URL and profile. The daemon dials the saved files only when the marker matches, so a URL or profile edited by hand is ignored until the menu bar saves again, and the daemon logs one line saying so. Rotating the secret by hand keeps the saved config. Saved files apply in two cases only: the flags name the installer'sswitchboard.secret, or the process is a--menu-barprocess given no switchboard flags. A headless process with no switchboard flags never dials from saved files. A process given explicit flags for another secret file dials those flags, and its menu cannot edit the saved files. When saved files apply, the daemon logs one line saying so. - Unreadable secret: if the flagged secret file is missing and nothing is saved, a
--menu-barprocess starts with the switchboard off and logs why. It keeps serving/mcpand/attach, and it does not exit. A headless process (no--menu-bar) exits with status 66, as before. Configure the switchboard from the menu bar or re-run the installer. - Exiting: the HTTP endpoint keeps serving after the switchboard attach stops. To dial again
after a
4002or4003, choose Reconnect in the menu bar, or restart the process.
On macOS the app installs, runs and removes its own Playwright (the managed tier; see
What the app downloads), and serves the 32 browser_* tools to every
caller with the owner profile: direct /mcp, a switchboard, or a lent session. An external
Playwright is the other option (--upstream, below). Both feed the same router.
The router reads an OAB-Browser-Backend request header (direct /mcp requests only; switchboard
and reverse-attach callers use the stored preference, or auto) with the values auto, playwright or
appleEvents. Only the Playwright backend exists in 0.12.0, so appleEvents answers "not set up".
The Apple Events tier is not built. Design: ADR.
flowchart TB
caller["Caller with the owner profile, observe sees no browser tools"]
pref{"Which backend. The OAB-Browser-Backend header first, else the stored preference (no menu control sets it in 0.12.0), else auto"}
pready{"Playwright ready"}
aready{"Apple Events ready"}
noBackend["No backend. No browser tools are listed and a call says browser tools are not set up"]
subgraph tierp["Playwright tier, shipped in 0.12.0"]
which{"Was --upstream browser given"}
ext["External Playwright, 127.0.0.1 port 8794, a separate process you started. The managed child is not used"]
man["Managed Playwright, a stdio child of the app with no listener"]
end
tiera["Apple Events tier, NOT BUILT, so it is never ready in 0.12.0"]
caller --> pref
pref -- "auto or playwright" --> pready
pref -- "appleEvents" --> aready
pready -- "yes" --> which
pready -- "no, and auto" --> aready
pready -- "no, and pinned to playwright" --> noBackend
aready -- "yes" --> tiera
aready -- "no" --> noBackend
which -- "yes" --> ext
which -- "no" --> man
Under auto Playwright wins whenever it is ready. A named backend that is not ready is never
replaced silently. Every result from a selected backend carries structuredContent.backend; the
"not set up" and guarded-selection errors come before a backend is chosen and carry none.
User guide for both platforms: docs/browser-tools.md.
The sandbox has no path to any browser, so browser control is served from this Mac: with
--upstream browser=http://127.0.0.1:8794/mcp (the installer adds it when the external Playwright MCP
LaunchAgent from poc/pw-mcp is installed; most Macs use the managed Playwright from the
wizard instead and need no --upstream) the daemon re-serves
Playwright's tools under its own tools/list, filtered by the connection's profile:
ownersees the 32browser_*tools the catalog lists;observesees none (#59). On macOS the tools go through a router (ADR); a tool Playwright adds later is filtered out until the catalog, the vector file and the tests are updated together. Port 8794 is loopback, not private: any process on this Mac can reach it, and the daemon uses it only when you pass--upstream browser=…(literal127.0.0.1or::1, no redirects) and the listener runs as your user. The browser is this computer's persistent profile, with its cookies, so browser access is owner-level:browser_evaluateandbrowser_run_code_unsaferun arbitrary JavaScript there.browser_navigate+browser_snapshotreturns the page as an accessibility tree — the first video title on a channel page is one text line, no screenshot, no OCR. The browser is a real window on this Mac's desktop, so Connect's Screens pane shows what the agent is doing.- Upstream down → its tools are absent from
tools/list; everything else works. The upstream'sMcp-Session-Idis re-established automatically. Local tool names win on collision. - Managed Playwright (no listener). Without
--upstream browser=…the app can run an installed Playwright MCP itself, as a child process over stdio: nothing listens on any port. It looks under~/Library/Application Support/oab-computer/playwright/(--playwright-root <dir>changes it;install.jsonthere names the install). The child starts on first use, is restarted with a backoff (1 s, 2 s, 4 s, up to 30 s) and is marked unavailable after 5 failures in 2 minutes. Its stderr is kept in memory only and is never written to a log. With nothing installed, the browser tools are not listed, a call says Playwright is not installed, andsys_infoand the menu show the same. - Installing, updating and removing the managed Playwright. The setup wizard has a Browser
tools page after Full Disk Access and before the optional tailnet page. Its Playwright box is
pre-checked; the text shows the download size, the three hosts, the install folder and the free
disk needed, all taken from the manifest built into the app, and says to uncheck it to skip.
Nothing is downloaded until you press Continue with the box checked; with the box unchecked,
Continue goes on and downloads nothing. The wizard always finishes: offline, or after a failed
download or check, the page shows what went wrong with Retry and Continue without, and
Playwright simply stays not installed. Closing the window is not completion. The same steps are in
the menu bar: Set up browser tools…, Install Playwright… (asks first, with the same text),
Update Playwright… (x → y) (shown only when this app carries a newer pinned version; it asks
before downloading, and a failed update leaves the installed version running), Remove
Playwright… (asks first; removes the marker-carrying version folders even when
install.jsonis bad, and the browser profile and its cookies when the app's own readableinstall.jsonconfirms it created that profile. A hand-made, unreadable or invalidinstall.jsonand what it points to stay, and the app then says it removed only part; Chrome, Safari and~/.local/oab-computer/are never touched), and Open Playwright window to sign in…. With--upstream browser=…the page and the menu items (except sign-in) are still offered, with a note that the external provider is used first in Automatic mode. A launch with every permission granted opens the Browser tools page until it has been completed once. Opening the menu readsinstall.jsonand checks the installed files, and starts nothing. The installer and what it fetches are described under What the app downloads; an install, update or removal takes effect on the nextbrowser_*call with no restart. - Signing in once. Open Playwright window to sign in… starts the managed browser if needed and opens a headed Chromium window on the managed browser's own persistent profile (not your Chrome or Safari). Sign in to the sites you want agents to use; the session stays in that profile, including after the app or the browser restarts. Removing Playwright deletes the profile only when this app created it and its install record is readable; otherwise the profile stays. The profile holds cookies, so browser tools are owner-level (see above). Some sites may refuse an automated browser, and the app cannot change that.
Tagged releases publish a universal, Developer-ID-signed and Apple-notarized installer:
- Download
oab-computer-VERSION-universal.pkgfrom GitHub Releases. - Sign into Tailscale and keep a desktop user logged in.
- Double-click the package. It auto-detects your Tailscale login/name, preserves or creates the
bearer token, installs the LaunchAgent, detects the Playwright upstream, and runs
tailscale serve --bg --https=8444. So the Mac is reachable on your tailnet at:8444as soon as the package finishes, before the wizard asks. A caller still needs the bearer token and an allowed Tailscale login. The "opt-in" for tailnet access applies to launching the app without the installer (for example a plain app launch): then the app creates no mapping until you turn tailnet access on, but it leaves an existing:8444mapping in place (for example one from an earlier install; the menu shows it as an existing mapping) and never overwrites a mapping it does not own. After an install, the wizard's tailnet page and the menu item Turn Off Tailnet Access control the mapping. - On first launch, the Set Up Mac Permissions window opens if a required permission is missing. It has three pages. Page 1 asks for Screen Recording and Accessibility, and you cannot finish it until both are on. Click Allow on each row, turn the switch on in System Settings, and return to the wizard, which re-tests on its own. Use Check Again if the row does not update. Page 2 is optional Full Disk Access, which has a Skip link. Page 3 offers to install Playwright (pre-checked; uncheck it to skip; see Browser tools), and the wizard completes with or without it. A fourth page, tailnet access, appears only when a Tailscale login is detected. Browser tools work without Full Disk Access. Future releases keep the same Developer ID + bundle id, so grants survive (the 0.12 rename changed the bundle id once; see Upgrading from 0.11 or earlier).
- The same wizard remains available from the menu bar as Set Up Permissions… (and Set up browser tools…, which opens it on the Browser tools page); use the menu item to copy the MCP URL and bearer token into OpenAB Connect/Remote.
Linux hands nodes: see Quick start and docs/linux-setup.md.
The .app.zip beside the package is an advanced/manual artifact. After unzipping:
/path/to/oab-computer.app/Contents/Resources/install-prebuilt.sh \
/path/to/oab-computer.app --allow-login autoThe installer never re-signs the app: doing so would change the identity TCC grants are bound to.
See docs/releasing.md for artifacts, signing/notarization, required secrets and
local packaging smoke.
A step-by-step guide with backup and rollback is in docs/upgrade-0.12.md. The summary follows.
0.12.0 renames the product's technical names from oab-instance-mcp to oab-computer. This is a
breaking change with no compatibility layer: nothing is read from or written to the old names, except
the token copy in step 3 (the old path is its source only) and the warn-only check for the old agent. The product name stays OpenAB Computer.
| 0.11 and earlier | 0.12.0 | |
|---|---|---|
| Executable, app file | oab-instance-mcp, oab-instance-mcp.app |
oab-computer, oab-computer.app |
| Bundle id, LaunchAgent label | dev.openab.instance-mcp |
dev.openab.computer |
| Playwright LaunchAgent, Linux unit | dev.openab.instance-mcp.pw-mcp, oab-pw-mcp.service |
dev.openab.computer.pw-mcp, oab-computer-pw-mcp.service |
| Linux unit | oab-instance-mcp.service |
oab-computer.service |
| Install, config, logs | ~/.local/oab-instance-mcp, ~/.config/oab-instance-mcp, ~/Library/Logs/oab-instance-mcp |
~/.local/oab-computer, ~/.config/oab-computer, ~/Library/Logs/oab-computer |
| Saved grants, audit log (macOS), state (Linux) | ~/Library/Application Support/oab-instance-mcp, $XDG_STATE_HOME/oab-instance-mcp |
~/Library/Application Support/oab-computer, $XDG_STATE_HOME/oab-computer |
| Keychain service (attach secrets) | dev.openab.instance-mcp.grant |
dev.openab.computer.grant |
| Environment variables | OAB_INSTANCE_MCP_* |
OAB_COMPUTER_* |
| Release assets | oab-instance-mcp-VERSION-* |
oab-computer-VERSION-* |
To upgrade:
-
Install 0.12.0 with the new installer (the
.pkg,install-prebuilt.sh, orinstall-linux.shfrom the new Linux tarball). The installer only warns when it finds the old agent. It never stops or removes it. -
macOS: grant Screen Recording and Accessibility again (and Full Disk Access if you use it). TCC grants are keyed to the bundle id, which changed. To drop the stale entries, run
tccutil reset ScreenCapture dev.openab.instance-mcpandtccutil reset Accessibility dev.openab.instance-mcp(andtccutil reset SystemPolicyAllFiles dev.openab.instance-mcpfor Full Disk Access). -
The bearer token is copied automatically from
~/.config/oab-instance-mcp/tokento~/.config/oab-computer/token, mode 600, and the old file stays in place. macOS: the copy happens when the new config directory is first created. Linux:install-linux.shcopies whenever~/.config/oab-computer/tokendoes not exist yet. If~/.config/oab-computeralready exists on macOS, nothing is copied and a new token is generated when the token file is missing. An existing new token is never overwritten. On Linux a new token is generated only when neither file exists. A user who never had an old token gets a new one and must update their clients. -
Not migrated, so redo them by hand, but only after the new installer or app has run once so that
~/.config/oab-computerexists: switchboard settings (switchboard.url,switchboard.secret,switchboard.profile,switchboard.secure-set-value), saved grants and their Keychain secrets (lend again), logs and audit files (on Linux, grants and audit under~/.local/state/oab-instance-mcpstay there too). You may copy the switchboard files, logs and audit files from the old directory if you want them. Do not copygrants.json: the grant secrets stay under the old Keychain service, so lend each grant again. -
Remove the old agent. It still holds port 8795 and fights the new one.
macOS:
launchctl bootout gui/$(id -u)/dev.openab.instance-mcp rm ~/Library/LaunchAgents/dev.openab.instance-mcp.plist # only if you used the Playwright agent: launchctl bootout gui/$(id -u)/dev.openab.instance-mcp.pw-mcp rm ~/Library/LaunchAgents/dev.openab.instance-mcp.pw-mcp.plist # then restart the new agent, which could not bind port 8795 while the old one held it: launchctl kickstart -k gui/$(id -u)/dev.openab.computer
If you used the Playwright agent, you can instead install the managed Playwright from the wizard (menu: Set up browser tools…) and not reinstall the external agent. To keep the external agent, install the new one from a clone of this repo (
poc/pw-mcp/install.sh), then run the main installer (the.pkgorinstall-prebuilt.sh) again. The installer adds the browser upstream to the agent only when the new Playwright agent is already loaded, and a restart alone does not.Linux:
systemctl --user disable --now oab-instance-mcp.service rm ~/.config/systemd/user/oab-instance-mcp.service # only if you used the Playwright unit (not on --no-browser nodes): systemctl --user disable --now oab-pw-mcp.service rm ~/.config/systemd/user/oab-pw-mcp.service systemctl --user daemon-reload # then restart the new unit, which could not bind port 8795 while the old one held it: systemctl --user restart oab-computer.service
After the new agent works you can delete the old directories.
rsync -a --delete --exclude .build --exclude .git ./ <mac>:~/src/oab-computer/
ssh <mac> 'cd ~/src/oab-computer && swift build -c release && swift test'
ssh <mac> 'cd ~/src/oab-computer && bash scripts/smoke.sh' # loopback, every step time-bounded~/src is on the internal disk on purpose: ~/build is a RAID symlink and LaunchAgents cannot
read the RAID (TCC on external volumes).
ssh <mac> 'cd ~/src/oab-computer && \
KEYCHAIN=~/Library/Keychains/signing.keychain-db KEYCHAIN_PASSWORD_FILE=~/.config/signing/keychain-password \
bash scripts/deploy.sh [email protected] <codesign-identity-sha1>'deploy.sh wraps the binary in ~/.local/oab-computer/oab-computer.app (bundle id
dev.openab.computer) so TCC grants bind to a stable identity, signs it with the
given Apple Development identity (any cert of yours; TCC grants are keyed on identity + bundle id, so keep using the same one), installs LaunchAgent dev.openab.computer in gui/501,
and runs tailscale serve --bg --https=8444 http://127.0.0.1:8795.
Then, once, on the Mac's own screen, System Settings → Privacy & Security:
- Screen & System Audio Recording → enable OpenAB Computer
- Accessibility → enable OpenAB Computer
The app shows as OpenAB Computer in System Settings, Finder and the menu bar. Its bundle id is
dev.openab.computer since 0.12.0 (earlier builds used dev.openab.instance-mcp), so grants made to
a build before 0.12.0 do not carry over.
then launchctl kickstart -k gui/501/dev.openab.computer. sys_info reports both as true when
done. Grants survive re-signing with the same identity + bundle id (verified across 0.1.0→0.2.0).
Also on the Mac: sudo pmset -a displaysleep 0. With display sleep on, an idle Mac returns black
screenshots and, once the lock engages, drops injected input.
With --menu-bar (deploy.sh sets it) the agent shows a status item: the name "OpenAB Computer" and version, the public MCP URL
(click to copy), a masked bearer token line (click to copy the full token), live ✓/✗/? rows for
Screen Recording, Accessibility, and Full Disk Access, Set Up Permissions… (the same window, which opens on launch when a permission is missing and
reopens while Screen Recording or Accessibility is still off), session / call counters with the
last tool call, the Playwright items (install, update, remove, sign in; see Browser tools), Open Log, Restart, and Quit. The setup window opens the exact System Settings pane
for each permission. Its Allow button calls the system permission request and re-tests, and it
opens the pane only if the permission is still off. The window re-tests when the app becomes
active or the user clicks Check Again. The menu bar checks Screen Recording and Accessibility every
30 seconds without prompting, and it calls the system permission request only when you click Allow.
It never captures a screenshot by itself. The icon fills briefly on each
tool call.
With --menu-bar, the menu also shows a switchboard section and the reverse-attach grant list. The
switchboard section appears in every menu bar process and reads "not configured" until a switchboard
is saved. The setup is described in Serving a switchboard,
and the profiles in docs/tool-profiles.md. The menu offers:
- Switchboard status: connected (with its profile), dialling, backing off (with the seconds until the next try), or stopped, plus the last close code and time and this computer's name. A startup note appears when saved files override the launch flags (their URL or profile differs), and a disabled state appears when the flagged secret is unreadable.
- Configure…: the URL, the secret and the profile. The secret field is never shown back; the
dialog only says whether one is saved. Leave it blank to keep the saved secret. The profile
offers
observe(the default) andowner. Choosingownerasks for confirmation first, becauseowneris shell-equivalent. Changing the URL or profile reconnects at once and drops the current attach. A new secret applies on the next dial; if the switchboard is not attached, saving dials at once. - Reconnect: drops the current attach and dials again. After a
4002it asks first, since another daemon with the same secret would be displaced again. - Disconnect & forget…: after confirmation, deletes
switchboard.url,switchboard.secretandswitchboard.profileand stops dialling. The LaunchAgent plist keeps its flags until the installer is re-run, so until then the daemon starts with the switchboard off. - Reverse-attach grants: each live grant, with its session, profile, state and expiry.
ownergrants are marked shell-equivalent. A grant whose request named a retired profile shows that name and the warning. Revoke asks for confirmation, then ends the grant on this Mac.
ssh <mac> 'launchctl print gui/501/dev.openab.computer | grep -E "state|pid"'
ssh <mac> 'tail -20 ~/Library/Logs/oab-computer/agent.log' # one line per session open / tools/call / deny
ssh <mac> 'launchctl kickstart -k gui/501/dev.openab.computer' # restart
ssh <mac> '/Applications/Tailscale.app/Contents/MacOS/Tailscale serve status'
curl -s https://<host>.<tailnet>.ts.net:8444/healthz
ssh <mac> 'ls -lt ~/Library/Logs/oab-computer/jobs/' # exec_start job logs (<id>.out/.err), GC'd after 7 days
ssh <mac> 'tail -f ~/Library/Logs/oab-computer/jobs/<job_id>.out' # follow a background build liveMeasured from the laptop: sys_info 0.75 s, exec 0.47 s, a 1 s exec timeout returns in 1.6 s.
Screen Recording, Accessibility and Full Disk Access are keyed on the code-signing identity +
bundle id, not the path. So a grant you make once in System Settings stays across upgrades
only if every build is signed by the same identity. An ad-hoc signature (codesign -s -)
has no stable identity — macOS treats each one as a new app and silently drops every grant, and
the symptom is the Screens pane freezing / screen_recording=false after a deploy.
Therefore deploy.sh refuses to install anything but a Team-signed bundle (team 6LPQNY95AQ).
Run it from a console session (the login keychain is locked over SSH, which is why ad-hoc kept
sneaking in). You grant each permission once; later versions keep it. Override for a throwaway
local build with ALLOW_ADHOC=1, accepting that you will have to re-grant.
The gotchas that each cost a deploy cycle, with their fixes, are in
docs/operators.md.