dsh-read-url

🌐 English | 中文

URL reader plugin for DeepSeek Harness: fetch any URL — webpages (HTML), JSON APIs, RSS/Atom feeds — auto-detect encoding (UTF-16 BOM / GBK/GB2312 / UTF-8 / Big5 / Shift-JIS), extract the clean main content (auto-joining paginated articles), and return token-efficient compact text or structured Markdown.

Zero runtime dependencies (Node 20+ built-ins handle fetch/decode/extract), no API key, no server side — install and use.

Supported content

• HTML technical documentation, forums, nested articles and custom elements, with visible code examples and metadata preserved.
• Plain text, Markdown, CSV, JSON and vendor +json APIs.
• RSS 1.0/2.0 and Atom, including namespace prefixes, relative article links and xml:base.
• UTF-16 BOM text without a Content-Type header.

Continue at charsStart + charsReturned; the returned charsStart may exceed the requested offset when paragraph separators are skipped. Markdown tables convert at most the first 25 rows and report omitted rows.

Why

This plugin keeps URL-reading output compact: cleaned body + essential metadata from HTML, paragraph-aligned truncation, continuation and a session cache. It also preserves plain text or Markdown returned by the configured DSH provider. DSH 0.2.1-rc.2 includes a DeepSeek Fetch API provider, so available source content and metadata depend on the selected provider.

Competitor comparison (measured from source/docs, 2026-08-15)

This is a historical snapshot; it does not describe the newer DeepSeek Fetch API provider.

Capability	Official tool-web web_fetch	dsh-webfetch	dsh-scrape-webpage	dsh-read-url
Body cleaning (container-level)	❌ whole page	⚠️ tag-level, nav/footer leak in	⚠️ custom, noisy	✅ article/main containers + noise stripping
Default output cap	200,000 chars	50,000 chars	30,000 chars	6,000 chars + paragraph-aligned truncation
Chinese GBK/GB2312	provider-dependent	⚠️ not normalized, GB2312 garbles	❌ not handled	✅ normalized + mojibake fallback + UTF-16 BOM + Shift-JIS
JSON / RSS / Atom URLs	❌ HTML only	❌	❌	✅ native compact rendering
Paginated articles	❌ manual per-page calls	❌	❌	✅ auto-joined (default 3 pages)
Session-level cache	❌	❌	❌	✅ 5-min TTL
ctx.web seam	✅ (official core)	❌ global fetch	❌	✅ seam-first, fallback included
ctx.effect unload cleanup	✅	❌	❌	✅
Cooperative timeout (hidden from model)	✅	⚠️ self-managed	⚠️ self-managed	✅ timeoutMs + exec.signal
Model-facing output	whole-page Markdown	compact text	15-field JSON	compact text (no JSON parsing)
Dependencies	official	TypeScript build	zero deps	zero deps (JS ESM, drop-in)
Anti-bot / degraded responses (UA & TLS fingerprint)	⚠️ Node default UA; measured: https intercepted by middlebox TLS fingerprinting, Baidu returns a degraded page without trending topics	❓ not disclosed	❓ not disclosed	✅ full browser UA; measured: full page fetched (Baidu trending topics intact)

…