dsh-file-reference-everything

Readme

# `dsh-file-reference-everything`

English | [中文](README.zh.md)

Whole-disk implementation of `ctx.fileReferences` for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness). It upgrades the dialog's `@` file completion from "current workspace" to "any file on the machine": when [Everything](https://www.voidtools.com/) is running, bare queries are answered in milliseconds through the `es.exe` command-line client; otherwise the plugin falls back to its own bounded whole-disk index. The fallback index works on macOS and Linux too, so the plugin is cross-platform. Workspace results are always merged in and ranked ahead of whole-disk hits.

## Requirements

Best experience — all three on Windows:

- [Everything](https://www.voidtools.com/) installed and running.
- [es.exe](https://www.voidtools.com/support/everything/command_line_interface/) discoverable: on `PATH`, in a common install location (official installer, `%LOCALAPPDATA%\Everything`, chocolatey, scoop), or pointed at by the `esPath` config key.
- Windows (Everything indexes NTFS in real time).

Without any of these the plugin still works: it detects the missing backend automatically and serves whole-disk queries from its built-in bounded scan index instead.

Note on installing the dsh CLI itself: prefer pnpm (e.g. `pnpm add -g @deepseek-ai/dsh`, or `pnpm add @deepseek-ai/dsh` in a dedicated directory). In real installs, npm/npx stalls for a long time (>10 minutes) resolving this package's large peer-dependency set.

## Installation

Install from GitHub into the `web` profile (recommended):

```sh
dsh plugin --profile web add github:lumose0/dsh-file-reference-everything
```

dsh's Web UI is its own profile (`dsh web` ≡ `dsh --profile web`), and the `@` file completion UI lives only there; the official `file-reference-local` row also exists only in the web profile. Installing into another profile works too, but `--dump-config` then prints the expected warning `patch: entry "file-reference-local" not found` — harmless; the plugin's own row is inserted regardless.

A git install fetches sources, not built artifacts; this package ships a `prepare` script that builds `lib/` from source at install time. pnpm ≥ 10 refuses to run that script until you allow it — the first `add` fails and prints the package key to copy into the profile's `pnpm-workspace.yaml`:

```yaml
allowBuilds:
  dsh-file-reference-everything: true
```

then re-run the `add`. That allowance is permission to execute this package's code on your machine at install time, so review the source and pin a commit (`github:lumose0/dsh-file-reference-everything#<sha>`) so a later push cannot silently change what runs. Once the package is published to npm, `dsh plugin --profile web add dsh-file-reference-everything` installs prebuilt code and needs no build allowance.

## How it takes effect

The package is a dsh bundle: its `cordis.patch.yml` layer disables the official workspace-only provider row (`id: file-reference-local`, the same row id the `dsh-web-app` bundle inserts) and inserts this plugin's row in its place. If the profile has no such row (a non-web profile), the disable patch warns and is skipped, and this provider still mounts. Verify the composed layers without booting:

```sh
dsh --profile web --dump-config
```

The output should contain a `# == dsh-file-reference-everything` layer with the `file-reference-everything` row inserted; in the web profile the official `file-reference-local` row shows `disabled: true`.

## Query syntax

Everything after `@` (or `@"` for paths containing spaces) is the query. Outside regex mode, `\` counts as `/`.

| Query | Behavior |
|---|---|
| `@config` (bare text) | Fuzzy search over the workspace index merged with the whole-disk backend; workspace hits rank first. |
| `@re:pattern` / `@regex:pattern` | Regular-expression match, workspace and whole-disk. |
| `@D:/...`, `@//server/share/...`, `@/usr/...`, `@~/...` | Browse that absolute directory (`~` expands to the home directory). |
| `@src/comp` (relative path containing `/`) | Browse relative to the agent cwd, same as the official provider. |

When Everything answers, bare queries pass through as Everything native syntax: space means AND, wildcards and the rest of the [Everything search syntax](https://www.voidtools.com/support/everything/searching/) work as-is.

## Configuration

| Key | Default | Contract |
|---|---:|---|
| `maxResults` | `20` | Maximum ranked candidates returned for one query. |
| `workspaceMaxEntries` | `10000` | Maximum indexed files and directories per agent workspace. |
| `globalMaxEntries` | `200000` | Maximum indexed files and directories in the whole-disk fallback index. |
| `excludedDirectories` | `[".git", "node_modules", "$Recycle.Bin", "System Volume Information", "Windows", "Program Files", "Program Files (x86)", "ProgramData", "AppData", ".cache", ".npm", ".pnpm-store", "Library", "proc", "sys", "dev"]` | Directory basenames never traversed or offered by the fallback index. |
| `roots` | `[]` | Whole-disk scan roots; empty selects automatic detection — every existing drive letter on Windows, the home directory elsewhere. |
| `esPath` | unset | Explicit `es.exe` path; unset enables automatic detection (`PATH`, then common install locations). |
| `everythingTimeoutMs` | `2000` | Timeout in milliseconds for one `es.exe` invocation; a timeout falls back to the built-in index. |
| `refreshSeconds` | `600` | Seconds the whole-disk fallback index stays fresh before the next query triggers a lazy rebuild. |
| `workspaceBonus` | `100` | Additive score keeping workspace candidates ahead of whole-disk hits in merged results. |

Override the row in the profile's `cordis.patch.yml` (applied after every bundle layer):

```yaml
- id: file-reference-everything
  name: dsh-file-reference-everything
  config:
    maxResults: 30
    roots: ['C:/', 'D:/']
```

A patch replaces the row's entire `config` value rather than deep-merging keys, so restate every non-default key you want; keys you omit take the schema defaults above.

## Model Experience

### File-reference guidance when `read` is available

#### What the model sees

When the addressed agent has an effective `read` tool, the provider contributes the same stable system-prompt section as the official provider:

```markdown
Paths prefixed with @ are files explicitly referenced by the user. Use the read tool when their contents are needed; do not claim to have inspected a file before reading it.
```

#### Token effect

Conditional and fixed: the one sentence is present while `read` is visible to the addressed agent; candidate lookup itself adds no tokens, and a selected path contributes only its ordinary user-message characters.

## Known limitations

- **A selected path is a text reference only.** Reading its contents still requires the model to call the `read` tool, subject to the deployment's filesystem namespace and permissions — a whole-disk path may be visible in completion yet unreadable by the tool.
- **The fallback index is bounded and lazy.** Paths beyond `globalMaxEntries` are omitted, the index rebuilds only when a query arrives after `refreshSeconds`, and it has no ignore-file semantics — only `excludedDirectories` basenames are skipped.
- **`es.exe` option collision.** A query starting with `-` may be parsed by `es.exe` as an option instead of a search term.
- **Non-ASCII `where.exe` output.** When `where.exe` cannot report the `es.exe` path reliably (non-ASCII paths under some console code pages), detection falls back to probing the static install locations.
- **Possible duplicate entries.** The same file can appear twice: once as a workspace-relative candidate and once as a whole-disk absolute candidate.
- **Fixed score for Everything-only syntax.** Results the plugin's own scorer cannot match (for example wildcard hits) keep a fixed low score instead of a precise ranking.

## Development

```sh
npm install
npx vitest run
npx tsdown
```

Do not verify the RPC path with `dsh plugin --profile <name> add <local directory>`: a directory install uses a pnpm `link:`, so Node resolves dependencies by realpath from the dev directory's own `node_modules` and loads a private copy of `@deepseek-ai/dsh-file-reference`. The host's Typert gateway then cannot discover the `fileReferences/list` remote method across the mismatched module instances — completion goes silently empty. Verify against the packed artifact instead: `npm pack`, then `dsh plugin --profile web add ./dsh-file-reference-everything-<version>.tgz` (or install via `github:`); peer dependencies then resolve to the host's instances and the path works out of the box.

## License

MIT