dsh-llm-finish-reason-tolerance

Readme

# dsh-llm-finish-reason-tolerance

A [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) host plugin that makes the agent tolerate OpenAI-compatible providers whose streaming responses end without a `finish_reason`.

The canonical case is the **Snowflake Cortex** gateway (`*.snowflakecomputing.com/api/v2/cortex/v1`): its Chat Completions SSE streams content deltas and a terminal `data: [DONE]` but never sends a `finish_reason` — not for plain text, not for tool calls (non-streaming responses return `finish_reason: ""`). The harness's model client (pi-ai) treats that as a truncated stream (`Stream ended without finish_reason`), and the harness maps it to a `TRANSPORT` error finish. Every request delivers content and then **fails the turn**.

This plugin rewrites only that specific terminal error into the successful finish the content already deserves.

## How it works

The plugin listens on the harness's `llm/stream` waterfall (registered globally and prepended, so its returned iterable is the one consumers iterate) and wraps every model stream:

- a **tool-call** block was delivered → the terminal finish becomes `{ kind: 'tool-calls' }`
- **text** content was delivered → the terminal finish becomes `{ kind: 'stop' }`
- nothing was delivered, or the error is anything else → passed through untouched

The rewrite is gated on both the exact pi-ai message (`Stream ended without finish_reason`) and on delivered content, so genuine mid-stream truncations (which surface as different pi-ai errors) are never masked.

No changes to pi-ai, `dsh-llm`, or `dsh-llm-pi-ai` are required.

## Requirements

- DeepSeek Harness (any recent deployment; `@deepseek-ai/cordis` ≥ 4 and `@deepseek-ai/schemastery` ≥ 3 are already present)
- Node ≥ 20
- The plugin must be installed into a **profile** (e.g. `web`) and mounted in the **host composition** — it is host-level and single-instance. Do not mount it inside an agent preset.

## Install

The package is shipped as a **profile bundle**: it carries its own `cordis.patch.yml` layer (declared via `dsh.bundle.patch` in `package.json`), so installing it mounts the plugin with no manual composition editing.

### Bundle install (recommended)

```sh
dsh plugin --profile web add /path/to/dsh-llm-finish-reason-tolerance-0.2.0.tgz
```

What this does:

1. `pnpm` installs the package into the profile's dependency tree.
2. The `dsh plugin` CLI detects `dsh.bundle.patch` and automatically appends the package to the profile's `dsh.profile.bundles` in `package.json`:

   ```json
   "dsh": { "profile": { "bundles": ["@deepseek-ai/dsh-base", "@deepseek-ai/dsh-web-app", "dsh-llm-finish-reason-tolerance"] } }
   ```

3. At the next boot, the bundle's own patch layer (`cordis.patch.yml` inside the package) inserts the `llm-finish-reason-tolerance` row into the host composition.

That's the whole install. Use `<profile>` instead of `web` for any other profile (e.g. `dsh plugin --profile tui add …`).

### Plain install (alternative)

If you prefer a plain dependency (e.g. the package is only published to a registry without bundle semantics), install it and add the row yourself. The profile patch layer (`~/.dsh/profiles/<profile>/cordis.patch.yml`) is a list of **patch operations**, so a new row must be wrapped in `insert:` — a bare `id:` row is an id-targeted override of an *existing* entry and is silently skipped when none matches:

```sh
# from the profile directory (e.g. ~/.dsh/profiles/web)
pnpm add /path/to/dsh-llm-finish-reason-tolerance-0.2.0.tgz
```

```yaml
# ~/.dsh/profiles/<profile>/cordis.patch.yml
- insert:
    - id: llm-finish-reason-tolerance
      name: dsh-llm-finish-reason-tolerance
      config:
        # Provider route keys from the `llm-pi-ai` settings section.
        # Empty (or omitted) applies to every provider.
        providers:
          - snowflake-cortex
```

### Restart

A **restart of the harness is required** after install — the plugin is a new module and is only mounted at boot. After restart, `llm-finish-reason-tolerance` appears in the plugins list.

Do not combine both install methods (that would insert the row twice), and do not also mount the row inside an agent preset.

## Configuration

| Field | Type | Default | Meaning |
| --- | --- | --- | --- |
| `providers` | `string[]` | `[]` | Provider route keys (as named in the `llm-pi-ai` settings section) this tolerance applies to. `[]` = all providers. |

### Scoping the tolerance to specific routes

The bundle ships with `providers: []` — the tolerance applies to **every** provider route. To restrict it to specific routes, override the row's config in your **own** profile patch layer. This works because bundle layers apply first and their inserted rows can be targeted by later layers (the `config` value replaces the bundle's wholesale, so state the full config):

```yaml
# ~/.dsh/profiles/<profile>/cordis.patch.yml
- id: llm-finish-reason-tolerance
  config:
    providers:
      - snowflake-cortex
```

With the plain install, scoping is simply the `providers` value inside the `insert:` row above.

## Using it with Snowflake Cortex

The Snowflake provider must be configured in the `llm-pi-ai` settings section with a route key that matches the plugin's `providers` list. With the default (empty) `providers` list the plugin applies to every route, so no matching is required; if you scope the plugin, the route key must match exactly (e.g. `snowflake-cortex`).

```yaml
# ~/.dsh/settings.yaml
llm-pi-ai:
  providers:
    snowflake-cortex:
      displayName: Snowflake Cortex (SG)
      apiKeyEnv: SNOWFLAKE_CORTEX_API_KEY
      api: openai-completions
      baseURL: https://<account>.snowflakecomputing.com/api/v2/cortex/v1
      models:
        - id: claude-sonnet-5
          name: Claude Sonnet 5
        # e.g. a DeepSeek model your account serves (deepseek-r1 is deprecated)
        # - id: deepseek-r1
        #   name: DeepSeek R1
```

Notes:

- Use `api: openai-completions` with the full `…/api/v2/cortex/v1` base URL. The OpenAI SDK authenticates with `Authorization: Bearer` from `apiKeyEnv` automatically — no headers needed.
- The plugin fixes the missing `finish_reason`; `deepseek-r1` is deprecated at the endpoint and cannot be used regardless of the plugin.

## Safety

- The rewrite fires **only** for pi-ai's `Stream ended without finish_reason` terminal error and **only** when content was delivered. All other errors — auth, rate limit, quota, genuine mid-stream truncation, empty responses — pass through unchanged.
- A response that ends without `finish_reason` and without content keeps its original error finish.
- Scope with `providers` to keep the behavior local to the routes that need it.

## Development

```sh
npm test        # node --test — exercises the wrapper with synthetic chunk streams
npm pack        # build the distributable tarball
```

Layout:

- `lib/index.js` — the plugin (plain ESM, no build step; the harness loads it directly)
- `lib/index.d.ts` — TypeScript declarations
- `test/finish-reason.test.mjs` — self-contained wrapper tests
- `cordis.patch.yml` — the bundle's own patch layer (auto-applied on bundle install)
- `cordis.example.yml` — the mount row for the plain-install path

## License
MIT