dsh-openrouter-live

Readme

# dsh-openrouter-live

An OpenRouter provider for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`dsh`) that reads OpenRouter's live model list, so a model shows up in the picker as soon as OpenRouter serves it.

The harness's built-in `openrouter` route lists models from a catalog bundled with each release. A model released after that snapshot, such as `deepseek/deepseek-v4.1-flash`, does not appear in the picker until the next harness release. This plugin adds a second route, **OpenRouter (live)**. It reads OpenRouter's [`/models`](https://openrouter.ai/docs/api-reference/models/get-models) endpoint and takes each model's metadata from there.

## What it does

- **Live model list.** It lists every OpenRouter model that accepts tools, sorted by name, so models from one vendor stay together. Each name ends with the input/output price per million tokens. The list is refreshed every 10 minutes. If a refresh fails, the previous list stays in use.

  ![Model picker with prices](docs/picker.png)
- **Per-model metadata.** The context window and output cap come from OpenRouter. The reasoning levels come from the model's own `reasoning.supported_efforts`, so the composer offers only levels the model accepts, including *Off* when reasoning is optional.
- **Image input.** It inlines images for models whose `/models` entry lists `image` input. It follows the harness image budget: images are downscaled, and the oldest images are offloaded when a request grows too large.
- **Reasoning continuity.** It sends `reasoning_details` back on later requests, including signed and encrypted entries. This lets models such as Claude and DeepSeek keep their reasoning across tool calls.
- **Prompt caching.**
  - Anthropic models get [cache breakpoints](https://openrouter.ai/docs/features/prompt-caching) on the system prompt and the newest message. In a live check, a second request read 7209 of 7223 prompt tokens from cache.
  - Other providers cache automatically. The `x-session-id` header keeps each conversation on one upstream provider, so its cache stays warm.
- **Classified failures.** Errors map to the harness failure codes, so the harness retry plugin retries only what is worth retrying:
  - Retried: rate limits, with OpenRouter's `Retry-After` honored; upstream 5xx errors; idle streams; truncated streams; empty responses.
  - Not retried: bad keys, exhausted credits, and context overflow.
- **Actual spend, including subagents.** The plugin records the charge OpenRouter reports for every response (`usage.cost`), so these are real charges, not token estimates. Each charge is attributed to the session that made the request and to every session it was delegated from. Type `/cost` in the CLI or Web UI:

  ```text
  OpenRouter spend in this session: $0.0022 (5 requests)
    this agent: $0.0011 (3 requests)
    subagents: $0.0011 (2 requests in 1 session)
  All recorded sessions: $0.0022 (5 requests).
  ```

  In the Web UI the session total is always visible under the composer, next to the built-in statistics and context meter. It updates during a turn, including subagent charges, and its tooltip splits the total between the agent and its subagents. It is hidden in sessions with no OpenRouter charges.

  ![Spend under the composer](docs/web-ui.png)

  Charges are appended to `~/.dsh/openrouter-live/costs.jsonl`, one JSON line per response, so totals survive restarts. Each line carries the OpenRouter generation id, so any charge can be matched against your OpenRouter activity page.
- **Shared credential.** By default the route reads the same `OPENROUTER_API_KEY` credential as the built-in `openrouter` route, so a key you already stored works without extra setup.

## Requirements

- DeepSeek Harness `0.1.6-alpha.2` through `0.1.7`. It has been tested on `0.1.6-alpha.2` and `0.1.7-alpha.2`. The harness API is pre-stable, so later harness versions are added to the peer range after they are tested.
- Node.js `^22.19 || >=24`.
- An [OpenRouter API key](https://openrouter.ai/settings/keys).

## Install

**From npm:**

```sh
dsh plugin --profile web add dsh-openrouter-live
```

**From a local checkout:**

```sh
git clone https://github.com/Boomerangz/dsh-openrouter-live.git
cd dsh-openrouter-live
npm install
npm pack --pack-destination dist          # runs typecheck and tests first
dsh plugin --profile web add "$PWD/dist/dsh-openrouter-live-0.2.0.tgz"
```

> [!IMPORTANT]
> Install the packed `.tgz`, not the directory. When you add a directory, pnpm links it, Node resolves the link to its real path outside the profile, and the harness packages (`@deepseek-ai/*`) can't be found there. The whole profile then fails to boot.

Restart the profile after installing. **OpenRouter (live)** then appears in the model picker. If you haven't stored a key yet, choose one of these:

- Enter it on the built-in **OpenRouter** card under **Settings → Models**. It is stored as `OPENROUTER_API_KEY`, which both routes read.
- Export `OPENROUTER_API_KEY` before launching `dsh`.

To choose it as the default model, pick it in the composer's model picker. You can also set it in `~/.dsh/settings.yaml`:

```yaml
agent-default-model:
  provider: openrouter-live
  model: deepseek/deepseek-v4.1-flash
```

## Configuration

Every field is optional. Set fields in the profile's patch layer, `~/.dsh/profiles/<profile>/cordis.patch.yml`, under the plugin's entry id `openrouter-live`:

```yaml
- id: openrouter-live
  config:
    models: ['deepseek/*', 'anthropic/claude-*', 'openai/gpt-5*']
    extraBody:
      provider:
        sort: throughput
```

| Field | Default | Meaning |
| --- | --- | --- |
| `models` | `[]` | Glob patterns over model ids. An empty list lists every model. Ids outside the filter still work when typed. |
| `toolsOnly` | `true` | List only models whose OpenRouter entry accepts `tools`. Agent turns need tools. |
| `priceInName` | `true` | Append input/output prices per 1M tokens to model names, such as `DeepSeek: DeepSeek V4.1 Flash ($0.15/$0.60)`, because the pickers show names but not descriptions. The price then also appears in the composer's model button. |
| `apiKeyEnv` | `OPENROUTER_API_KEY` | Credential reference: a credential-store entry or environment variable. |
| `provider` | `openrouter-live` | Route id. Give each instance its own id to run several side by side, for example one with a `*:free` filter. |
| `displayName` | `OpenRouter (live)` | Provider name in the picker. |
| `baseURL` | `https://openrouter.ai/api/v1` | API root. |
| `modelListTtlMs` | `600000` | How long one fetched model list is reused. |
| `maxTokens` | `32768` | Output cap for requests that set none. A lower model limit wins. |
| `defaultContextWindow` | `262144` | Context size assumed for an id the list does not describe. |
| `streamIdleTimeoutMs` | `300000` | Longest silence inside a response stream before failing with `TIMEOUT`. |
| `maxRequestImageBytes` | `20971520` | Base64 image budget per request. Older images are offloaded beyond it. |
| `requestImagePixelBudget` | `4194304` | Pixel budget of one request image. |
| `requestImageMaxBytes` | `1048576` | Encoded-byte target of one request image. |
| `appTitle` / `appUrl` | `DeepSeek Harness` / unset | OpenRouter [app attribution](https://openrouter.ai/docs/app-attribution) headers (`X-OpenRouter-Title`, `HTTP-Referer`). |
| `extraBody` | `{}` | Fields merged into every request body, such as [provider routing](https://openrouter.ai/docs/features/provider-routing). Fields the plugin sets take precedence. |
| `costLedgerPath` | `$DSH_HOME/openrouter-live/costs.jsonl` | Spend ledger file. An empty string keeps spend in memory for the current process only. |
| `costCommand` | `cost` | Slash command that reports spend. Leave it empty on a second instance that shares the ledger, so the command is registered once. |
| `costIndicator` | `true` | Serve the Web UI spend indicator's data route (`/openrouter-live/spend`). Disable it on a second instance, which would otherwise claim the same route. |
| `retryPolicy` | harness default | The harness's provider retry policy for this route: `{mode: normal, maxRetries, retryableCodes, backoff}` or `{mode: always}`. |

## How it maps onto OpenRouter

| Harness | OpenRouter chat completions |
| --- | --- |
| System prompt, user and assistant text | `system`, `user` and `assistant` messages |
| Tool definitions and tool calls | `tools` (function type) and `tool_calls` |
| Tool results | `tool` messages. Images returned by a tool follow in a `user` message, because `tool` messages carry text only. |
| Images | `image_url` data URLs, each preceded by the harness's image handle line |
| Reasoning effort | `reasoning.effort` |
| Reasoning output | The `reasoning` stream becomes reasoning blocks. `reasoning_details` is kept for replay. |
| Usage | `prompt_tokens` is split into uncached input, cache reads and cache writes. Reasoning tokens are reported separately. |

## Development

```sh
npm install
npm run check                          # tsc --checkJs (strict) + node:test suite
OPENROUTER_API_KEY=... npm run smoke   # live checks against OpenRouter; costs a few cents
```

The plugin is plain ESM JavaScript with JSDoc types, so no build step is needed to run it. `npm pack` generates the `.d.ts` files in `types/`. The test suite includes a composition test that boots the real `LlmRuntime` from `@deepseek-ai/dsh-llm` in a Cordis context and streams through it.

## Known limitations

- Spend covers only requests through this plugin's routes. The built-in `openrouter` and `deepseek-official` routes are not counted.
- A charge is recorded when OpenRouter reports it at the end of a response. A request that is cancelled or fails mid-stream may still be billed by OpenRouter, but it does not appear in `/cost`.
- The Web UI indicator reads a route that answers only same-origin requests to a loopback address (`127.0.0.1`, `localhost`), because the harness web server does not authenticate plugin routes. When the Web UI is opened over the network, the indicator stays hidden; `/cost` still works.
- `/cost` needs the harness command service, which the CLI and Web profiles mount. Headless runs still write the ledger.

- Only OpenRouter's text output is used. Image generation and audio models are listed only when they also accept tools, and their non-text output is ignored.
- Prices in the picker are OpenRouter's base prices. Time-of-day price overrides are not shown.
- Explicit cache breakpoints are sent only for `anthropic/*` models. Every other model relies on its provider's automatic caching.
- A model that reasons but publishes no `supported_efforts` gets no reasoning selector, so OpenRouter's default applies.

## License

MIT