webkit

A pure-logic, humane web-automation toolkit — the reusable gems lifted out of the owner's browser-automation-platform (BAP, MIT) and repackaged as a single small Rust CLI. 0 API key. 0 infra. No database, no async runtime, no network calls inside webkit-core — every subcommand is a deterministic function over plain data (a string, a JSON doc, a byte slice) in, JSON envelope out.

Actual browser control (navigating a real page, clicking a real element) is delegated to the external browser-act CLI (installed and configured separately) for webkit robot play and webkit browse *, which shell out to browser-act --session <name> ...; extract, cookie convert|detect, humanize plan and robot validate|lint need no browser at all.

Browser gen-2 (platform phases PP4/PP6) adds three ways to work with the user's real Chrome — webkit still never launches it:

• webkit browser attach … — attach via DevToolsActivePort (one approved connection, shared through a loopback relay);
• webkit browser mirror-profile — copy session state into a separate user-data-dir for a second instance;
• webkit ext … + the MV3 extension in extension/ — borrow a tab with the user's consent, trusted CDP input, and HITL on CAPTCHA/2FA.

These live in webkit-cli (sync tungstenite, no async runtime); webkit-core stays pure.

Why

BAP is a full platform (Postgres, Redis, a render sidecar, an LLM-backed selector-recovery finder). Most of what makes it useful to an agent is a much smaller set of pure algorithms buried inside it — a Bézier mouse-curve planner, a Gaussian/bigram-aware typing planner, an HTML→Markdown cleaner, a CSS-schema scraper, a cookie-jar format converter, a portable "robot" replay DSL. webkit ports exactly those pieces, severs everything that needed a database or a live browser connection, and exposes them as one CLI any agent can shell out to with zero setup. See NOTICE for the exact BAP source file each module came from.

Install

Requires the Rust toolchain (cargo) — see rust-toolchain.toml for the pinned version; rustup will fetch it automatically on first build.

Windows (PowerShell):

.\install.ps1

Builds a release binary and copies it to %LOCALAPPDATA%\webkit\bin\webkit.exe.

Linux / macOS:

./install.sh

Builds a release binary and installs it to ~/.local/bin/webkit.

Both scripts are idempotent — re-run any time after pulling new commits. Neither touches Docker, a container runtime, or a browser download; browse/robot play assume browser-act is separately installed and on PATH when you need it.

Output envelope + exit codes

Every subcommand prints exactly one line of JSON to stdout and nothing else (anything human-readable — usage errors from clap — goes to stderr instead):

{"ok": true, "data": { ... }}
{"ok": false, "error": {"code": "some_code", "message": "..."}}

Exit codes:

Code	Meaning
0	success (ok: true)
1	user/input error — bad file, bad JSON, failed validation
2	internal error — the process itself misbehaved (e.g. browser-act spawn failure)
3	a human must act (help_requested / needs_human from webkit ext) — stop, do NOT retry

Error envelopes may carry extra fields next to code/message (e.g. helpId, reason, tabUrl, runId for exit 3).

Every subcommand that reads a document takes it as a positional path argument, -, or (when omitted) stdin — so everything below also works piped: cat file.json | webkit robot validate.

Commands

robot — RobotDsl scripts (validate / lint / play)

A RobotDsl is a versioned JSON document describing a sequence of browser steps (goto, click, type, press, scroll, wait_for, wait_ms, if_exists, extract). validate and lint never touch a browser; play only does when you drop --dry-run.

$ webkit robot validate good.robot.json
{"data":{"steps":2,"valid":true,"version":1},"ok":true}

$ webkit robot validate bad.robot.json   # empty step list
{"error":{"code":"dsl_empty","message":"dsl has no steps"},"ok":false}
$ echo $?
1

$ webkit robot lint good.robot.json
{"data":{"has_secret":false,"max_if_depth_seen":0,"start_url":"https://example.com","step_types":{"goto":1,"wait_ms":1},"steps":2,"valid":true,"version":1},"ok":true}

# Plan the replay without a browser (what CI/tests exercise):
$ webkit robot play --dry-run good.robot.json
{"data":{"dry_run":true,"plan":[{"detail":"goto https://example.com","index":0,"kind":"goto"},{"detail":"wait_ms 100","index":1,"kind":"wait_ms"}],"start_url":"https://example.com","steps":2,"version":1},"ok":true}

# Replay for real, driving a browser-act session (requires browser-act on PATH):
$ webkit robot play good.robot.json --session my-session --trace

cookie — convert / detect cookie jars

Converts between Chrome devtools JSON, Netscape cookies.txt, Playwright storage_state.json, and a canonical shape — or just sniffs which format a blob is, without parsing it.

$ webkit cookie detect cookies.txt
{"data":{"format":"netscape"},"ok":true}

$ webkit cookie convert cookies.txt --to canonical
{"data":{"cookies":1,"result":{"cookies":[{"domain":"x.com","expires":1894924800,"http_only":false,"name":"sid","path":"/","same_site":null,"secure":false,"value":"abc123"}],"local_storage":{},"session_storage":{}},"to":"canonical"},"ok":true}

$ webkit cookie convert cookies.txt --to playwright
$ webkit cookie convert chrome_export.json --to netscape

--to accepts canonical (default) | netscape | playwright.

extract — HTML → Markdown, or CSS-schema → JSON

$ cat page.html | webkit extract md --only-main
{"data":{"markdown":"# Hello World\n\nThis is the main content.\n"},"ok":true}

$ webkit extract css --schema schema.json card.html
{"data":{"data":[{"href":"/a","label":"Alpha"},{"href":"/b","label":"Beta"}]},"ok":true}

--only-main strips nav/footer/aside/form/header boilerplate before converting; without it the whole document is converted. --schema points to a JSON file describing the CSS schema, e.g.:

{
  "base_selector": ".item",
  "fields": [
    { "name": "label", "selector": "a", "source": "text" },
    { "name": "href", "selector": "a", "source": "attr", "attr": "href" }
  ]
}

humanize — plan mouse curves / typing event streams

Plan only — this never touches a browser or dispatches real input; it emits the deterministic event stream a driver would turn into real actions. Same --seed + inputs ⇒ byte-identical output (golden-tested).

$ webkit humanize plan --text "hi there" --seed 1 --mode simple
{"data":{"count":8,"events":[{"action":"char","ch":"h","time_ms":219}, ...]},"ok":true}

$ webkit humanize mouse --from 0,0 --to 100,50 --steps 4 --seed 1
{"data":{"count":5,"points":[{"x":0.0,"y":0.0}, ...,{"x":100.0,"y":50.0}]},"ok":true}

plan --mode is simple | realistic (default) | sloppy; --wpm overrides the default 70-WPM target. mouse --seed is optional — omit it for a one-off random curve.

browse — snapshot + act on numbered elements (needs browser-act)

Delegates entirely to an external browser-act --session <name> process. Index-only surface — no CSS selectors ever cross this boundary.

$ webkit browse snapshot https://example.com --session my-session
$ webkit browse click 3 --session my-session
$ webkit browse type 2 "hello" --session my-session

Without --session (and no existing session to reuse), these fail fast with need_session / exit 1 — before ever shelling out to browser-act:

$ webkit browse snapshot https://example.com
{"error":{"code":"need_session","message":"missing --session (open one first with `browser-act browser open`)"},"ok":false}

browser — attach to the real Chrome / mirror a profile / tab-governor

Attach (preferred). Chrome 136+ ignores --remote-debugging-port on the default profile, and webkit never uses it there. Instead the user switches on chrome://inspect/#remote-debugging once; Chrome then writes <user-data-dir>/DevToolsActivePort. Every new DevTools WebSocket makes Chrome ask the user, so attach serve connects exactly once, keeps it (TCP-probe liveness, re-discovers when the token rotates after a Chrome restart) and serves a loopback CDP relay that any number of clients share without new dialogs. Browser.close/Browser.crash* are refused.

$ webkit browser attach discover --wait 30          # just read the port file
{"data":{"port":9222,"port_open":true,"user_data_dir":"C:\\Users\\u\\AppData\\Local\\Google\\Chrome\\User Data","ws_url":"ws://127.0.0.1:9222/devtools/browser/<redacted>"},"ok":true}
$ webkit browser attach serve [--governor] &        # long-running; prints one ready line
$ webkit browser attach status                      # upstreamConnected / upstreamConnects / clients
$ webkit browser attach call Target.getTargets      # one CDP call through the relay
$ webkit browser attach call Webkit.relayStatus     # relay-local: relayStatus, leaseTab, governorPlan

No port file → attach_not_enabled (exit 1) with the chrome://inspect hint. CDP clients (e.g. Playwright connectOverCDP) use relay_ws_url from attach-relay.json in the state dir (--state-dir › $WEBKIT_STATE_DIR › %LOCALAPPDATA%\webkit\state / ~/.local/state/webkit).

Mirror (second instance). Copies ONLY Cookies, Local/Session Storage, Login Data, Device Bound Sessions, Trust Tokens (+ journals) and Local State reduced to os_crypt (so DPAPI/keychain still decrypts) into a separate dir; scrubs SingletonLock/DevToolsActivePort; sets exit_type=Normal; refreshes the copy on every run; never writes to the source; refuses while the source (or the mirror) is open in Chrome (mirror_source_locked / mirror_dest_locked).

$ webkit browser mirror-profile --from "%LOCALAPPDATA%\Google\Chrome\User Data" --to C:\hive-chrome-mirror --debug-port 9333
{"data":{"copied":["Local State","Default/Network/Cookies",...],"launch_args":["--user-data-dir=C:\\hive-chrome-mirror","--profile-directory=Default","--no-first-run","--no-default-browser-check","--remote-debugging-port=9333"],...},"ok":true}

Tab-governor. Agent-owned tabs idle 2 min → throttled, 10 min → frozen, 24 h → closed; RAM budget / tab cap sleep or close the longest-idle first. User tabs and tabs leased mid-run are never touched. attach serve --governor applies it to tabs created through the relay; governor plan is the pure planner over JSON:

$ webkit browser governor plan tabs.json   # {"tabs":[{"id","agent_owned","last_active_ms",...}],"policy":{...},"now_ms":N}

cookie backup — strict by default

$ webkit cookie backup --out fb-cookies.json            # Storage.getCookies via the attach relay
$ webkit cookie backup --out x.json --ws ws://127.0.0.1:9333/devtools/browser/<id>

A dead backend is cookie_backend_dead (exit 2) and nothing is written — never a fake empty backup. --lenient turns that into skipped (still no file). An empty jar needs --allow-empty, and no mode ever overwrites a non-empty backup with an empty one (cookie_backup_would_clobber).

ext — borrow a tab with consent (MV3 extension)

Channel + security design: docs/adr/0001-extension-channel.md (WebSocket on 127.0.0.1 + token; web origins refused).

One-time setup per machine (HITL): chrome://extensions → Developer mode → Load unpacked → webkit/extension; run webkit ext pair and paste the token into the extension's Options page. Permissions: debugger, tabs, storage, notifications, alarms; the overlay content script runs on *.facebook.com, 127.0.0.1, localhost only (no <all_urls>).

$ webkit ext serve &                         # bridge the extension dials into (prints one ready line)
$ webkit ext status                          # bridge + extension session (lease, runs, humanBusy)
$ webkit ext borrow url:facebook.com/stockandview   # page shows "Cho phép agent dùng tab này?"
$ webkit ext act steps.json                  # [{"op":"type","selector":"…","text":"…"},{"op":"click","selector":"…"}]
$ webkit ext help-wait <helpId>              # after exit 3 help_requested: 0 resumed | 3 needs_human
$ webkit ext act --resume <runId>            # continue at the paused step (idempotent)
$ webkit ext release                         # detach, overlay off, tab back to its place

Steps: goto, click, type (clear), press (named keys), wait, wait_for, help (explicit HITL). Input is CDP Input.* through chrome.debugger only (20+rand(40) ms per character) — no synthetic DOM events. While driving, a closed-shadow-root overlay shows "hive đang thao tác · [Dừng]" and shields the page; a user click/keypress makes the agent wait 15 s (--human-busy-ms). Dừng (or Chrome's debugging-bar Cancel) ends the lease → cancelled_by_user. A CAPTCHA/2FA/checkpoint pauses the run (help_requested, exit 3): the tab is focused, an OS notification shown, and the human presses Tiếp tục (or webkit ext resume <helpId>); after 300 s (--help-timeout) it becomes needs_human {reason, tabUrl} — never a retry loop. webkit never tries to solve a CAPTCHA.

Extension tests: cd extension && npm test (node --test, pure modules). E2E (real unpacked extension in a throwaway Chromium profile + real webkit ext serve + local fixture pages): cargo build -p webkit-cli && cd extension && npm run e2e — needs the playwright npm package and a Chromium that still honours --load-extension (CHROMIUM_PATH, default: newest local ms-playwright chromium); HEADED=1 to watch.

What's next

.rune/features.md tracks the shipped gems above plus the v2 backlog (affordance schema, crawl-control gates, identity-FSM, browse live-watch) that were deliberately deferred — read it before planning new work here.

License

MIT — see LICENSE. Per-module BAP provenance is in NOTICE.