dsh-llm-databricks

Readme

# dsh-llm-databricks

[中文](README.zh.md)

An independent, community-maintained DeepSeek Harness bundle for conversation
models served through Databricks AI Gateway. It installs out of tree, uses the
public Harness LLM/settings/credentials seams, and does not patch Harness or
`node_modules`.

> This package is not an official DeepSeek AI or Databricks product.

## Support matrix

| Databricks route | `protocol` | Streaming | Tools | Notes |
|---|---|---:|---:|---|
| MLflow Chat Completions `/ai-gateway/mlflow/v1/chat/completions` | `mlflow-chat` | Yes | Yes | OpenAI Chat wire format. Removes unsupported `store` and tool `strict`. Covers Databricks-served Claude, GPT, Gemini, Llama, DeepSeek, and other conversation models. |
| MLflow Responses `/ai-gateway/mlflow/v1/responses` | `mlflow-responses` | Yes | Yes | OpenAI Responses/OpenResponses behavior available through the Harness/pi-ai vocabulary. Removes unsupported `store` and tool `strict`. |
| Native Anthropic Messages `/ai-gateway/anthropic/v1/messages` | `anthropic-messages` | Yes | Yes | Uses only `Authorization: Bearer`; suppresses `x-api-key`. Preserves text, tool use, and thinking blocks supported by the model. |
| Native OpenAI Responses `/ai-gateway/openai/v1/responses` | `openai-responses` | Yes | Yes | Uses bearer authentication and the native Responses wire format. |

Native Gemini API and embeddings are not implemented in 0.1.0. Gemini
conversation models are already usable through MLflow Chat Completions.
Harness's current LLM seam does not consume embeddings. Native Gemini is a
roadmap item.

## Requirements

- Node.js `^22.19.0` or `>=24`
- DeepSeek Harness `0.1.0-rc.7` compatible seams
- A Databricks personal access token, OAuth access token, or other bearer token
  accepted by the configured workspace gateway

## Install

From npm after a public release:

```sh
dsh plugin --profile web add dsh-llm-databricks
```

From the public GitHub repository:

```sh
dsh plugin --profile web add github:zhuchengde0214-ctrl/dsh-llm-databricks
```

For a packed local release:

```sh
pnpm pack
dsh plugin --profile web add ./dsh-llm-databricks-0.1.0.tgz
```

The package's `dsh.bundle.patch` adds a dormant `llm-databricks` plugin row.
It registers routes only after configuration supplies providers.

## Configure

Add an `llm-databricks` section to the isolated profile's user settings (or use
a Harness configuration surface that writes the same namespace):

```yaml
llm-databricks:
  providers:
    databricks-main:
      displayName: Databricks Main
      workspaceUrl: https://dbc-example.cloud.databricks.com
      protocol: mlflow-chat
      credentialRef: DATABRICKS_TOKEN
      models:
        - id: example-chat-endpoint
          name: Example Chat Model
          contextWindow: 128000
          maxTokens: 8192
          input: [text]
        - id: example-reasoning-endpoint
          name: Example Reasoning Model
          contextWindow: 200000
          maxTokens: 16384
          reasoningEfforts:
            off:
            low: low
            high: high
```

`id` is the model-serving endpoint/model service identifier sent in the model
field. Capacities are deployment metadata and must match the served model.
`input` defaults to `[text]`; declare `[text, image]` only when the endpoint
actually accepts images.

Use one provider route per protocol when the same workspace exposes several
wire formats:

```yaml
llm-databricks:
  providers:
    databricks-anthropic:
      workspaceUrl: https://adb-example.azuredatabricks.net
      protocol: anthropic-messages
      credentialRef: DATABRICKS_TOKEN
      models:
        - id: example-anthropic-endpoint
          contextWindow: 200000
          maxTokens: 16384
    databricks-responses:
      workspaceUrl: https://workspace.gcp.databricks.com
      protocol: openai-responses
      credentialRef: DATABRICKS_TOKEN
      models:
        - id: example-responses-endpoint
          contextWindow: 128000
          maxTokens: 8192
```

Store the token through Harness's credentials service (for example, the web
Models page) or expose the referenced environment variable to the Harness
process. Settings contain only `credentialRef`; the plugin never stores a token
in settings or logs it.

Inspect the composed bundle without sending a model request:

```sh
dsh --profile web --dump-config
```

## Security boundary

- HTTPS is mandatory. Userinfo, query strings, fragments, and paths in
  `workspaceUrl` are rejected.
- By default, bearer credentials are sent only to a workspace subdomain under
  `.cloud.databricks.com`, `.azuredatabricks.net`,
  `.gcp.databricks.com`, or legacy `.databricks.net`. Matching is
  label-boundary safe; lookalike suffixes are rejected.
- Official hosts must use port 443.
- Each protocol is pinned to its exact gateway request path. Redirect following
  is disabled by the request transport.
- Custom/private DNS requires `allowCustomHost: true`. This opts into forwarding
  the bearer credential to that hostname. Review DNS ownership, TLS termination,
  proxies, and logging before enabling it.
- A non-443 custom port additionally requires `allowNonStandardPort: true`.
  This is intended for controlled private gateways and tests.
- Native Anthropic requests delete `x-api-key` and send bearer authentication
  only. The plugin never takes over official Anthropic or other non-Databricks
  provider routes.

## Errors and retries

The adapter preserves HTTP status, Databricks/request IDs, and valid
`Retry-After`/`retry-after-ms` delays. It maps:

- 401/403 to authentication failures
- 429 quota messages (including workspace input-tokens-per-minute limits) to
  quota failures, and other 429 responses to rate limits
- explicit context-length messages to context overflow
- ordinary or bodyless 400 responses to invalid requests, never context overflow
- 5xx responses to server failures

SDK retries are disabled; Harness's provider retry policy owns retries. The
default policy retries transient failures only, honors provider delays, and
does not loop on deterministic 4xx responses. Configure `retryPolicy` per route
only when a deployment needs different bounded backoff.

### Troubleshooting 429

1. Read the error's retry delay and request ID; do not immediately resubmit.
2. Check the workspace AI Gateway usage and the model serving endpoint's input
   tokens/minute quota.
3. Reduce concurrent sessions, prompt size, or tool output retained in context.
4. Request a quota increase or route workloads across separately governed
   endpoints.

## Migration and uninstall

Configuration is portable: copy the `llm-databricks` settings section and
re-create the named credential in the destination Harness home. Do not copy
credential values into source-controlled settings.

Remove the plugin and then delete its settings section:

```sh
dsh plugin --profile web remove dsh-llm-databricks
```

Remove `DATABRICKS_TOKEN` from the credentials service only when no other route
uses it.

## Development

```sh
corepack pnpm install --frozen-lockfile
pnpm run typecheck
pnpm run lint
pnpm test
pnpm run build
```

Tests use stub transports and an isolated HTTPS mock gateway on an ephemeral
port. CI never contacts Databricks and requires no real credentials.

## Release and pack

1. Update `CHANGELOG.md` and the package version.
2. Run all development checks above.
3. Run `pnpm pack --pack-destination .artifacts`.
4. Inspect the tarball with
   `tar -tzf .artifacts/dsh-llm-databricks-<version>.tgz`; it must contain only
   built `lib`, the bundle patch, package metadata, documentation, changelog,
   and license.
5. Install that tarball into an isolated `DSH_HOME` and inspect
   `--dump-config` before publishing.

Do not publish from an unreviewed workspace. This repository intentionally
keeps `private: false` for a future public npm release, but publishing is a
separate maintainer action.

## License

MIT