dsh-plugin-llmtr

Readme

# dsh-plugin-llmtr

English | [Türkçe](README.tr.md)

An [LLMTR](https://llmtr.com) provider route for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (`dsh`).

LLMTR is an OpenAI-compatible AI gateway. One API key reaches 170+ chat models from OpenAI, Anthropic, Google, Qwen, Mistral, xAI and others — plus the models LLMTR runs on its own infrastructure in Türkiye, for deployments that need their prompts to stay in-country.

This plugin registers those models as the `llmtr` provider route on `ctx.llm`, so they appear in the harness model picker and can be selected per session, per agent, and per subagent like any other route.

## Install

```sh
dsh plugin add dsh-plugin-llmtr
```

The package ships a bundle patch, so `plugin add` inserts the row for you. Then open **Settings → Models**, find the **LLMTR** card, paste a key from [llmtr.com](https://llmtr.com), and save. The key is stored in `$DSH_HOME/.credentials.yaml`; settings retain only its reference.

The model picker fills itself from the gateway's live listing — nothing to type, and models LLMTR adds later appear without an upgrade.

### Without the plugin CLI

Add the row to your own `cordis.yml` or `--patch` overlay:

```yaml
- insert:
    - id: llm-llmtr
      name: dsh-plugin-llmtr
```

and export the key instead of storing it:

```sh
export LLMTR_API_KEY=llmtr-...
```

## Model ids

Model ids are the gateway's own, `owner/model`, and are sent verbatim:

```
openai/gpt-5.4          anthropic/claude-...      google/gemini-3.7-flash
qwen/qwen3-8-...        mistral/...               deepseek/deepseek-v4-...
llmtr/muse-glimmer-30b-tr   llmtr/trendyol-asure-12b   llmtr/gemma-4
```

Ids starting with `llmtr/` are the ones LLMTR hosts in Türkiye; the picker labels them "Hosted in Türkiye on LLMTR infrastructure". Everything else is routed to its upstream provider and processed under that provider's own terms — the plugin does not claim otherwise for them.

## Data residency

To offer only the in-country models, narrow the catalog scope:

```yaml
llm-llmtr:
  catalogScope: turkey-hosted
```

The picker then lists nothing else, so a session cannot be started on a model whose prompts leave the country.

## Configuration

Every field is optional and lives in the `llm-llmtr` section of `$DSH_HOME/settings.yaml` (or in the plugin row's `config`). Changes reach the next request without a restart; an in-flight stream keeps the values it started with.

| Field | Default | What it decides |
| --- | --- | --- |
| `apiKeyEnv` | `LLMTR_API_KEY` | Credential reference resolved per request |
| `baseURL` | `https://llmtr.com/v1` | Endpoint; `$LLMTR_BASE_URL` from a trusted launch layer overrides the default |
| `headers` | `{}` | Extra request headers; authentication and attribution names always win |
| `catalogScope` | `chat` | `chat`, `turkey-hosted`, or `all` |
| `catalogTtlMs` | `900000` | How long a fetched listing is reused |
| `models` | `[]` | A pinned list; non-empty replaces the live listing entirely |
| `modelOverrides` | `{}` | Per-model corrections applied over the live listing |
| `reasoningEfforts` | `[]` | Effort ids the gateway accepts for your models |
| `maxTokens` | `32768` | Default per-request output cap |
| `defaultContextWindow` | `131072` | Capacity assumed for a model you have not sized |
| `streamIdleTimeoutMs` | `300000` | Idle ceiling while a stream read is outstanding |
| `retryPolicy` | harness default | Provider-owned retry policy |

### Context windows

The LLMTR listing publishes ids and operations, not capacities, so every model is sized at `defaultContextWindow` until you say otherwise. Correct the ones you actually use:

```yaml
llm-llmtr:
  modelOverrides:
    openai/gpt-5.4:
      contextWindow: 400000
    llmtr/muse-glimmer-30b-tr:
      contextWindow: 32768
      maxTokens: 8192
```

### Reasoning effort

Reasoning **output** works out of the box: the adapter reads both `reasoning_content` and `reasoning` deltas, so a reasoning model's thinking shows up in the transcript with no configuration.

Selecting an effort **level** is opt-in, because the gateway fans requests out to upstreams that reject `reasoning_effort` on non-reasoning models. Declare what your models accept and the harness offers exactly those:

```yaml
llm-llmtr:
  reasoningEfforts: [low, medium, high]
```

With the list empty, no selector is shown and nothing reaches the wire. A request carrying an undeclared effort is refused by name rather than silently dropped.

### Pinning a model list

A deployment that must fix exactly which models its users can reach replaces the listing instead of filtering it:

```yaml
llm-llmtr:
  models:
    - id: llmtr/muse-glimmer-30b-tr
      name: Muse Glimmer 30B
      contextWindow: 32768
    - id: llmtr/trendyol-asure-12b
```

With `models` non-empty the gateway listing is never fetched.

## Scope

- **Chat completions only.** Requests go to `/v1/chat/completions` with streaming and usage reporting on. LLMTR's embeddings, image, video, rerank, and realtime models are listed by `catalogScope: all` but cannot be routed by this adapter, which is why `chat` is the default.
- **Text in, text out.** Image content is refused before it is sent, naming the model, rather than being flattened away.
- **One attempt per call.** Retries belong to the harness retry policy, so a failure is visible once rather than hidden inside the adapter.

## Privacy

The plugin sends the harness's standard `User-Agent`, an `x-llmtr-client` header naming this package and version, and your API key. No session id, prompt text, file path, or user identifier is added to any header.

## Development

```sh
npm install
npm test          # 80 unit tests, no network and no key required
npm run typecheck
npm run build
```

`tests/mock-server.ts` stands in for the gateway, so the suite runs offline.

## License

[MIT](LICENSE). Portions derive from DeepSeek Harness — see [NOTICE](NOTICE).