dsh-vision-fallback

English | 中文

Silent vision enhancement for DeepSeek Harness (dsh): keep your real text-only main model (e.g. deepseek-v4-flash), and let chat images "just work" — every image you drop, paste, or reference in the chat box is automatically sent to a fixed vision model, converted into a factual text observation, and handed to your main model as hidden context. The UI keeps showing your original image; no model groups, no model switching, no extra tools.

Why

• DeepSeek V4 Flash / Pro and other strong coding models are text-only: dropping an image into the chat box fails with "model does not support image input".
• Existing "vision tool" plugins require saving images as files and invoking a see_image(path) tool — clunky, and the main model still can't see chat attachments.
• This plugin bridges the gap at the request layer, so chat-box images work exactly like you expect, regardless of which main model you pick in the model picker.

How it works

You drop/paste an image ──► chat attachment (kept visible in UI)
        │
        ▼
agent/pre-step ──► image + current question + recent context
        │                    │
        │                    ▼
        │            fixed vision model (OpenAI-compatible /chat/completions)
        │                    │  factual text observation
        │                    ▼
        └──► model-only surface replacement ──► main model (text only)

1. The plugin overrides the pre-send capability check, so a text-only model can receive image-bearing messages.
2. agent/pre-step detects images in the incoming turn, and sends the image, the latest user question, and recent conversation context to the configured vision model.
3. Your original image stays in the UI as a normal chat attachment.
4. A model-only surface replacement swaps the image for the vision observation before the request reaches the main model.
5. Switching the main model (DeepSeek, Kimi, MiniMax, ...) never changes the fixed vision model.

Complete call paths

• Normal conversations: agent/pre-step processes images before the main-model request and projects observations into the model view.
• Context compaction: the llm/stream path with purpose: "compaction" reuses existing observations first and only describes genuinely unseen images.
• Tool-result images: images inside tool/result events are projected for the model while the original tool result remains visible and traceable in the UI.
• Reloads and restarts: observations are persisted to observations.json, so the same session/message position does not trigger another vision call after restart.
• Model capability detection: in auto mode the plugin reads the main model's real inputModalities; it stays out of the way for image-capable models and bridges only text-only routes.

Install

From npm / local checkout

# npm (if published) or a local checkout directory
dsh plugin --profile web add dsh-vision-fallback
# or: dsh plugin --profile web add /path/to/dsh-vision-fallback

From source

git clone https://github.com/1HelloMan1/dsh-vision-fallback.git
cd dsh-vision-fallback
pnpm install --config.minimumReleaseAge=0   # rc.6 peers need the release-age flag bypassed
pnpm test                                    # 22 unit tests
dsh plugin --profile web add "$PWD"

Then verify and restart:

…