dsh-media-guard

Readme

# dsh-media-guard

Request image optimization, intelligent retention, and observability for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (DSH).

`dsh-media-guard` optimizes the provider-bound image view before every agent-loop model call. It inventories attachment references, compresses oversized images, deduplicates by content address, preserves current-turn and user media first, and replaces media that still cannot fit with factual Evidence Notes. The durable session log and original attachments are never modified; compressed variants are added through DSH's immutable attachment store.

DSH 0.1.0-rc.8 already supplies the baseline safety layer this plugin originally filled: attachment admission rejects unsafe per-image/per-message inputs, and `offloadRequestImages()` lets the official Pi AI and DeepSeek adapters cap aggregate Base64 image payload at 20 MiB by replacing the oldest images. Media Guard is therefore an **optimization, retention-policy, and observability layer**, not a claim that DSH lacks request-size protection. Its remaining advantages are compression instead of immediate removal, content-address deduplication, current-turn/source-aware retention, tighter configurable budgets, Evidence Notes, reporting, and uniform coverage of third-party adapters that do not call DSH's helper.

This is the DSH port of [pi-media-guard](https://github.com/spyfree/pi-media-guard); the planner, fair-share math, evidence notes, and failure policy are shared line for line where the hosts allow.

> **Alpha:** this line implements the ledger, planner, sharp-backed compression, budget-driven externalization, the `llm/stream` projection, and a local report UI. The request path, attachment API, client bundle, optional `webServer` seam, and coexistence with rc.8 `offloadRequestImages()` are tested against `@deepseek-ai/dsh-*` 0.1.0-rc.8. DSH itself is a developer preview and its APIs may still move.

## How it works

- Registers on the `llm/stream` waterfall and acts **only on agent-loop-built requests** (`isAgentLoopRequest`). Hand-built calls — vision bridges' VLM traffic, compaction, replay — pass through untouched, and so does the plugin's own projected re-dispatch, which terminates recursion.
- Accounting is **zero-byte-cost**: DSH image blocks carry content-addressed attachment references (`sha256:…`) with exact byte sizes and dimensions, so the ledger, dedup, and budget math never read image data. Only images selected for compression are read through `ctx.attachments`.
- Oversized images are constrained with this package's own sharp dependency, encoded as WebP, and saved back through `ctx.attachments.saveImage`. Variants are cached by source content address and target, so duplicates and later matching requests reuse the same stored ref.
- When the projection changes anything, a fresh request is dispatched via `ctx.llm.stream(...)` — the frozen loop-built request and the stored session are untouched; only the provider-bound view changes.
- In `protect` mode, the serialized budget is capped to the mirrored adapter `maxRequestImageBytes`, so an official rc.8 adapter normally sees an already-safe request and does not perform a second, invisible oldest-first removal.
- If the projection itself fails, an intentionally simpler emergency path strips every image leaf. The guard never silently falls back to sending the oversized original.

## Install

```bash
dsh plugin --profile <name> add dsh-media-guard
```

Restart the profile afterwards. The bundle inserts a `media-guard` row into DSH's existing home composer status strip; no configuration is required for the defaults.

Installing straight from a git host (`dsh plugin --profile <name> add github:spyfree/dsh-media-guard`) builds `dist/` via the package's self-contained `prepare` script; pnpm ≥ 10 will ask you to allowlist that build in the profile's `pnpm-workspace.yaml` on first install, exactly as DSH's packaging guide describes.

## Default behavior

- Runs in `protect` mode before every agent-loop provider request.
- Applies a conservative default budget: 8 media blocks, 2 MiB serialized (Base64) / 1.5 MiB decoded aggregate, 512 KiB serialized per image.
- Preserves current-turn media before older media; user media before tool media.
- Deduplicates identical images by their content address; later copies become duplicate notes.
- Replaces overflow media leaves with deterministic Evidence Notes (hash, size, dimensions, origin, reason — no invented descriptions, no directives).
- Preserves message count, order, roles, ids, sources, and tool-call/result pairing.
- Emits a `media-guard/report` event after each guarded call and keeps a bounded, metadata-only in-memory history for the report UI; never logs or emits image bytes.

The default budget is a conservative optimization policy, not a claim about any provider's maximum body size. It sits well below DSH's official-adapter default of 20 MiB, so Media Guard acts first and the core helper is normally a no-op. Raise it per route only together with the route's actual adapter ceiling.

## Configuration

Override the plugin row from your profile's `cordis.patch.yml` with a flat
patch entry targeting the row id (the same form DSH's own bundles use):

```yaml
- id: media-guard
  config:
    mode: protect            # observe | optimize | protect
    log: false               # true prints a one-line summary per projection
    budget:
      maxMediaBlocks: 8
      maxSerializedMediaBytes: 2097152
      maxDecodedMediaBytes: 1572864
      maxSerializedBytesPerImage: 524288
    adapterMaxRequestImageBytes: 20971520 # mirror adapter setting; rc.8 default
    profiles:                  # keyed by the request's provider route id
      my-vision-gateway:
        maxSerializedMediaBytes: 12582912
        maxDecodedMediaBytes: 9437184
        adapterMaxRequestImageBytes: 12582912 # mirror this route's adapter
    statusPollIntervalMs: 2000 # home-row refresh cadence
```

A `profiles` entry is more specific than the top-level `budget`, so for the fields it declares it wins on its route even when both are set. `adapterMaxRequestImageBytes` does **not** configure DSH's adapter; mirror the value from that adapter's route configuration. Media Guard caps its resolved `maxSerializedMediaBytes` to the matching global or route value. The global default is 20 MiB, matching the official rc.8 Pi AI and DeepSeek adapters. If an adapter is configured below that, mirror its lower value here; if a third-party adapter has a different bound, declare that route explicitly.

Modes:

- `observe`: inventory and report only;
- `optimize`: compress but do not externalize;
- `protect`: compress, then externalize overflow media (default).

In `observe` and `optimize`, over-budget images deliberately remain after Media Guard, so an official adapter may still apply its own oldest-first offload. The report describes Media Guard's output, not that later adapter-only step. Use `protect` with a correctly mirrored ceiling when the UI must describe the final image selection. No plugin can introspect a deployment-defined adapter setting, which is why a non-default adapter ceiling must be mirrored in configuration.

`enabled: false` bypasses the guard entirely. Every config field is validated individually; an invalid value falls back to its protective default and is reported — a broken config can never widen a budget or switch the guard off.

Unlike the Pi original there are no built-in provider profiles: DSH provider routes are deployment-defined, so every route gets the conservative default until you declare a profile for it.

## Report UI

The main DSH home page now carries a compact, live Media Guard row under the
composer, in the same `conversation.composer.dock` strip as DSH's own stats
line. It is scoped to the active session, refreshes from that session's
metadata-only report endpoint, and shows the current pressure, images kept,
serialized-media budget, compression, deduplication or externalization counts,
and an `input over limit` marker when the original request crossed the budget.
For example:

```text
media near limit now · 4/4 images kept · 1.5M/2.0M · 2 compressed · input over limit
```

This is the primary at-a-glance surface; it does not open a new page.

When the profile includes DSH's Web server (the normal `web` profile), open:

```text
http://<your-dsh-host>:<port>/media-guard
```

The page is the detailed, optional view: it shows process-lifetime totals and
the 100 most recent guarded requests. It polls the same read-only JSON endpoint
at `/media-guard/api/reports` using the configured cadence. The home row adds the active
session's `sessionId` query parameter; the detailed page deliberately leaves it
off and therefore shows the process-wide history. Both surfaces contain
metadata only; report history is held in memory and resets when DSH restarts.
The home-row cadence is configured with `statusPollIntervalMs`; the API
advertises the validated value to the browser bundle.
Session-scoped latest summaries are retained for at most the most recent 100
session identities; process-wide totals remain lifetime totals.
In a headless profile the optional route is simply not registered, while the
guard and `media-guard/report` event continue to work.

## Marketplace discovery

The package declares its DSH bundle, server-plugin kind, optional Web-server peer, and privacy disclosure in `package.json`. The GitHub repository carries the `dsh-plugin` topic used by DSH marketplace scanners; after a push, topic-based marketplaces normally refresh it on their next indexing cycle. No install script is used—the official profile package flow builds and registers `cordis.patch.yml`.

## Anonymous regression

The test suite reproduces the original four-image failure distribution:

```text
1.82 MB + 0.74 MB + 1.21 MB + 2.80 MB = 6,567,972 Base64 bytes
```

Without a codec, `protect` brings the request under the default 2 MiB aggregate budget deterministically. The regression's deterministic codec keeps all four visual inputs available at 1,600,000 serialized bytes; the shipped sharp codec is separately tested against the same serialized-byte contract and always falls back to normal externalization when it cannot meet a target.

## Development

Requires Node.js 22.19 or newer, matching DSH's own engines requirement.

```bash
npm install
npm test
npm run typecheck
npm run lint
npm run build
```

## Security and privacy

Plugins run with the user's full local permissions. Review code before installation. This plugin reads only compression candidates through DSH's verified attachment API, stores compressed variants through the same API, sends no telemetry, and never logs attachment contents, payloads, or credentials. Reports and diagnostics carry metadata only. Stored variants follow the attachment store's retention policy; the in-memory report history resets on restart.

## License

MIT