dsh-bundle-vision

Readme

# dsh-bundle-vision

A **zero-core-change vision capability** for DeepSeek Harness, shipped as one installable npm package that is both a profile bundle and a plugin:

- the plugin registers the model-facing **`describe_image`** tool;
- the bundle patch mounts that plugin on any profile.

The tool reads a local PNG/JPEG/WebP/GIF file, commits the bytes through the shipped attachment service, and asks the **named** multimodal route about it in one direct LLM request (`provider` / `model` are tool arguments). The result is text only — no image block ever enters the calling session, so a text-only main model gains vision without any change to dsh itself.

## How it works against the shipped seams

Everything the tool uses already ships with every dsh profile:

- `ctx.fs` (bounded byte read, session-workspace resolution) — filesystem capability;
- `ctx.attachments` (`saveImage`, image limits, magic-byte validation) — durable image storage;
- `ctx.llm` (`resolveModelInfo` + `stream`) with the pi-ai multi-provider adapter — the multimodal request itself.

The one per-deployment prerequisite is the same as for any vision use of dsh: the multimodal model must declare image input in the `llm-pi-ai` settings section, e.g.:

```yaml
llm-pi-ai:
  providers:
    my-vision:
      apiKeyEnv: MY_VISION_API_KEY
      api: openai-completions
      baseURL: https://example.invalid/v1
      models:
        - id: my-vision-model
          input: [text, image]
```

(On releases whose Models page has the input-modality control, the same declaration is one dropdown.)

## Install (installed dsh, no source checkout)

From the npm registry:

```sh
dsh plugin --profile <name> add dsh-bundle-vision
```

Or from a packed tarball (e.g. before the first publish, or for a pinned version):

```sh
dsh plugin --profile <name> add ./dsh-bundle-vision-0.1.0.tgz
```

`dsh plugin` forwards to pnpm inside the profile directory and reconciles the profile's bundle layers automatically — a dependency declaring `dsh.bundle` joins the layer stack. Restart `dsh <name>`; the tool registers for every agent (profile-root registrations are visible to all preset scopes).

## Use

Ask the main model, naming the route:

> Use describe_image with file_path `/path/to/photo.jpg`, provider `my-vision`, model `my-vision-model`, and prompt "OCR the text in this image".

The main model supplies provider/model per call — configure several multimodal routes and switch per call, no profile edits. Refusals name the failing gate (unknown extension, deployment media types, a route whose model does not declare image input, a missing file, a type-mismatched file, an errored stream).

## Version floor

The package declares `>=0.1.0-rc.6` peer dependencies on the dsh seam packages. Bump the floor if a later release is required.

## Development

```sh
npm install          # dev deps resolve the seam packages from the npm registry
npm run typecheck    # tsc --noEmit over src + tests
npm test             # vitest
npm run build        # tsdown (lib/index.js) + tsc declarations (lib/types)
npm pack             # the installable tarball
```