dsh-client-vision

English | 中文

Give your DeepSeek Harness agent eyes. dsh-client-vision is a screen-capture + external image-recognition plugin for DeepSeek Harness: the agent takes a screenshot (or points at any image), hands it to a vision-capable model through a pluggable channel, and gets back plain text it can actually act on — no multimodal model required.

Why you want it

• DeepSeek can't see — now it can. The harness model has no image input. This plugin runs the whole "look" outside the model and returns text the agent can reason about, exactly like Codex's semantic vision tool.
• Capture anything, any way. fullscreen / window (with live window enumeration) / region / interactive — grab the browser, a game window, or one corner of the screen.
• Multi-channel by design. Tools are decoupled from recognition backends. The gpt channel ships ready to use; adding Claude, Gemini, or a local model is one analyze() implementation + one registry line — the three tools never change.
• Secret-safe. The API key lives in the harness credentials store (VISION_GPT_API_KEY) — never in settings files, logs, or the conversation transcript.
• Every preset, out of the box. Mounted on the host plane, so code, standard, cordis, minimal — every agent sees the tools. No preset switching.
• Ready to ship. Prebuilt bundles included; three install paths (drop into the monorepo / pnpm publish / tarball).
• Smart payloads. Large captures are auto-downscaled and re-encoded (≤1568px JPEG q80) before they leave the machine.

Capabilities

Tools

Tool	What it does
take_screenshot	Capture the screen: fullscreen (primary display), window (by id from list_windows), region (x, y, width, height), interactive (user selection), android (adb device/emulator), or ios (booted simulator). Returns the PNG path + dimensions.
list_windows	Enumerate on-screen windows (id, app, title) — macOS CGWindowList, Windows Get-Process main handles, Linux X11 (wmctrl/xprop) — pick the browser or game window to capture.
analyze_image	Submit an image (a path, or the most recent screenshot) to the configured vision channel and return a plain-text description.
view_image	One-shot "look at this": capture the screen (or use image_path) and recognize it through the active channel. The screenshot is rendered as an image card in the Web conversation, while the model context receives only the plain-text description — the image bytes never enter the model context.

Platforms

Platform	Capture backend	Window enumeration	Extra requirements
macOS	screencapture (system)	Swift CGWindowList	Screen Recording permission on first use
Windows	PowerShell System.Drawing (system)	Get-Process main window handles	PowerShell System.Drawing
Linux	ImageMagick import	wmctrl + xprop	ImageMagick (convert/identify), wmctrl, x11-utils

mode=interactive (system selection UI) is macOS-only; on Windows and Linux use mode=region with explicit coordinates.

Device capture

…