Native desktop automation for agents

Observe. Decide. Act across real desktop apps.

agent-desktop gives AI agents structured access to macOS, Windows, and Linux applications through accessibility trees, deterministic element refs, and machine-readable command results.

See the workflow Install from npm

agent-desktop snapshot --app Slack --skeleton --compact

snapshot_id: s8f3k2p9
refs: @e1 sidebar, @e3 message list

agent-desktop snapshot --root @e3 --snapshot s8f3k2p9
agent-desktop click @e12 --snapshot s8f3k2p9

Why agents need it

Desktop control should be structured, recoverable, and cheap in tokens.

Accessibility Trees

Work with semantic UI structure instead of brittle screenshots and pixel matching.

Deterministic Refs

Use stable element references and snapshot IDs so agents can act and then verify state.

Progressive Traversal

Inspect shallow maps first, then drill into regions to reduce context use in dense apps.

Best fit

For builders making agents that operate outside the browser.

AI coding assistants that need to inspect desktop apps
QA and operations workflows across Slack, Xcode, Finder, and system apps
Agent frameworks that need JSON output and recovery hints
Native automation loops that need observe-act-observe discipline