JohnXu22786/computer-control
dsh-computer-control
dsh plugin: desktop control - screen capture, pointer/keyboard injection, accessibility-tree semantic actions, with emergency stop, allow/deny rules, confirmation flow and idle standby
Install
npx @deepseek-ai/dsh plugin --profile demo add github:JohnXu22786/computer-controlRestart `dsh web` after install. Bundle APIs can change during the developer preview.
README badge
[](https://dshhub.dev/plugins/computer-control)Paste this into your README. The star count updates with every catalog sync.
From the README
Excerpt from JohnXu22786/computer-control, cleaned of badges and images.
computer-control — Desktop Control Plugin (for dsh)
Lets an agent operate the computer desktop directly: screenshot observation, mouse/keyboard injection, semantic actions via the accessibility tree (semantics first, pixel-coordinate fallback), with built-in safety guards — emergency stop, allow/deny rules, confirmation flow and idle standby.
The plugin is self-contained and loadable by dsh directly: it declares tools and events via manifest.json, speaks line-delimited JSON-RPC 2.0 (stdio) as its transport protocol, and python -m computer_control serve is the entry point.
Feature Overview
| Capability | Description |
|---|---|
Screenshot screen.capture | Full-screen or region capture, PNG/JPEG, scaling, grayscale — token costs under control |
Mouse pointer.* | Move, left/middle/right click (double/triple), drag, scroll (horizontal/vertical) |
Keyboard keyboard.* | Single keys, chords (scancode injection, layout-independent), arbitrary Unicode text input |
Wait wait.pause | Pause between actions so the UI can settle before the next screenshot |
Semantic actions a11y.* | Hierarchical accessibility-tree summaries (skeleton/standard/full), semantic activate/input — UIA mode first, automatic fallback to bounding-box pixel clicks |
Batch batch.execute | Run several actions in one call to reduce model round trips; confirmable as a whole, can continue on error |
| Capability broadcast | tools.list / system.status report per-backend availability; semantic tools are only exposed when UIA is available |
Safety guards (see the "Safety" section):
- Emergency stop: triple trigger — global hotkey (default
Ctrl+Alt+F12, configurable), protocol command, panic file; a STOP banner shows in the screen corner - Allow/deny rules: matched by tool name and arguments, deny always wins; whitelist mode switchable
- Confirmation flow: high-risk actions (including chords with
winorctrl+alt) wait for human approval, auto-denied on timeout - Idle standby: no activity past a threshold puts the session into standby, rejecting all actions until resumed
- Dry-run mode:
platform: "dry-run"logs actions without executing them, for safe rehearsals
Directory Structure
…
