dsh-llm-vision-bridge

English | 中文

Let text-only LLMs (DeepSeek) "see" images in the dsh web GUI: paste an image into the chat and the plugin automatically routes it to a vision model (Qwen3-VL via your existing pi-ai / llama.cpp route), then feeds the resulting text description to DeepSeek, which continues the conversation as if it were a native multimodal model.

Features

• Native LLM provider — registers deepseek-vision on the DSH LlmAdapter seam. Image admission, request routing, and session compaction all run through harness-native mechanisms; no UI changes, no front-end interception.
• Zero overhead without images — image-free requests pass straight through to the fallback provider (default deepseek-official).
• Vision-assisted replies — each image block is described by the vision model (attached user text is included in the prompt), then replaced with a [图片 N 描述] text block before the request reaches DeepSeek.
• LRU description cache — the same image + prompt is never re-described; history replay and compaction do not re-run the vision model.
• 503/429 auto-retry — tolerates the desktop GPU's single-card exclusive scheduling (vision gateway returns 503 while other tools occupy VRAM).
• Configurable failure policy — placeholder (insert a failure note and continue) or error (fail the turn).

How it works

The chat composer natively supports image attachments: images enter the model request as {type:"image", attachment} content blocks. The DeepSeek chat-completions adapter rejects image blocks with UNSUPPORTED_CONTENT, so a text-only model cannot process them directly.

This plugin's bridge provider (deepseek-vision) declares inputModalities: ["text", "image"], which satisfies the host's image-admission check (MODEL_DOES_NOT_SUPPORT_IMAGES is otherwise thrown before the message ever reaches the agent). Inside its stream():

1. No image → yield* ctx.llm.stream({ ...options, provider: fallbackProvider }) — passthrough, zero cost.
2. Has image → for each image block, call the vision model via a nested ctx.llm.stream() against the configured vision provider (e.g. pi-ai's llama route; image bytes are read automatically by the attachment service), then replace the image block with a [图片 N 描述]\n<description> text block and forward the rewritten messages to the fallback provider.

Session compaction reuses the provider of the most recent request, so image-bearing history is also bridged automatically. The optional autoRoute setting (default off) additionally rewrites deepseek-official agent requests to this provider, but it cannot bypass the host's image-admission check — it is only a fallback. To actually send images, set the main model to deepseek-vision.

Install

# From GitHub (plain JS, no build step, no allowBuilds needed)
dsh plugin --profile web add github:Einskyle/dsh-llm-vision-bridge

# Or from the npm registry
dsh plugin --profile web add dsh-llm-vision-bridge

# Restart the web service
pnpm dsh web

Manual install without pnpm (equivalent):

1. Copy this package into %USERPROFILE%\.dsh\profiles\web\node_modules\dsh-llm-vision-bridge\
2. Edit %USERPROFILE%\.dsh\profiles\web\package.json:
   • add "dsh-llm-vision-bridge": "file:<absolute path>" to dependencies
   • add "dsh-llm-vision-bridge" to dsh.profile.bundles
3. Restart the web service

Quick start

…