Protocol

Message format

Every message is one JSON object. A request names a command in t; include an id when you want a correlated reply (reads), omit it for fire-and-forget (HID writes).

// request  → fire-and-forget (no id)
{ "t": "hid.move", "dx": 30, "dy": -5 }

// request  → with id, expects a reply
{ "t": "screen.pixel", "id": 7, "x": 100, "y": 50 }
// response →
{ "id": 7, "r": 139, "g": 0, "b": 0 }

// subscribe → the server then pushes frames as state changes
{ "t": "subscribe", "events": ["mouse", "input"] }
{ "t": "mouse", "x": 812, "y": 344 }

On connect, the server sends a hello frame with the protocol version and whether auth is required, so a client can verify compatibility with no round-trip.

Command surface

system.window reads the active (foreground) window; the window.* commands list, find, activate, and move windows.

screen.capture (protocol 1.1.0+) returns a display as a base64 PNG at native resolution plus its geometry and the cursor position, and screen.displays enumerates monitors with signed virtual-desktop origins — together the basis for screenshot-driven, computer-use-style automation. The hello frame advertises the server’s protocol version so a client can check for these before using them.

Screen and Window commands depend on platform support — both are limited or absent on Linux; Clipboard is available on Windows and macOS. See Platforms & limits.

Authentication

Auth is off by default (fine on localhost). To require a token, set AUTH_TOKEN at the top of remote_access.lua before installing, then have the client send { "t": "auth", "token": "..." } first. The lua.exec escape hatch (arbitrary Lua) is disabled unless you set ALLOW_LUA_EXEC = true — leave it off unless you trust every client.

Performance

Measured on localhost (Windows host, release build):

Round-trips are sub-millisecond over localhost and a wired LAN — the transport won’t be your bottleneck.