dsh-memory-search-plus

Readme

# Memory Search Plus for DeepSeek Harness

English | [简体中文](README.zh.md)

Memory Search Plus is an independent [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) Web plugin for local, cross-session full-text search. It maintains a rebuildable, storage-bounded SQLite FTS5 derivative; source conversations remain authoritative.

This repository is tagged with the [`dsh-plugin`](https://github.com/topics/dsh-plugin) topic for discoverability.

## What it does

- **Search across all sessions, not just the open one.** Every indexed user message, assistant answer and tool call becomes searchable from one place.
- **Full-text index, not a filter.** Built on SQLite FTS5 with a custom tokenizer pipeline (Intl.Segmenter for Chinese/English words, CJK bigrams and single characters, English case normalization), so Chinese substrings, English phrases, mixed-language queries and multi-keyword AND queries all work.
- **Adaptive two-stage search.** Recall starts at 128 candidates, expands only as needed, stops at 2,048 candidates or its time budget, then performs literal verification and highlighting. Responses default to 100 and cap at 200 hits instead of sending thousands of rows to the UI.
- **Precise message targeting.** Results are grouped by session and time-descending; clicking a hit loads older history as needed, scrolls to the exact message and highlights it.
- **ChatGPT/VSCode-style UI.** A "find in conversations" pill in the wide sidebar opens a floating results panel with role filters (All / User / Agent / Tool). Open it with the pill or <kbd>Ctrl</kbd>/<kbd>Cmd</kbd>+<kbd>Shift</kbd>+<kbd>F</kbd>.
- **Incremental indexing.** New events are appended to the index as they arrive; compaction checkpoints rebuild only their session; removed sessions are deleted. Settings shows rebuild progress. A startup and every-5-minute background reconcile keeps the index consistent. Branch/fork conversations are indexed; true subagent sessions (`origin=subagent`) are not.
- **Compact and bounded.** Schema v3 uses contentless FTS (no duplicated document copy inside FTS), `detail=none`, and a separate document table. User/Agent messages index up to 8 Ki characters; tool output indexes a balanced 4 Ki head/tail preview and is labelled as such in results.
- **Runs fully locally.** By default the WAL database lives at `${DSH_HOME}/storages/memory-search/index.db`. Settings can point it at another directory or `.db` file; `DSH_MEMORY_SEARCH_DB` does the same without the UI. Default logical budget is 1 GiB. At the budget only the oldest derived rows are evicted—never source conversations. Settings reports disk use, coverage start, and eviction count. No network calls or telemetry are used.

## Performance targets

| Scenario | Target |
| --- | --- |
| Local index write path, 1M synthetic messages | < 60 s |
| Query (mixed Chinese/English/phrase) | < 300 ms |
| Incremental update | only changed events are touched |

Measured numbers from the bundled benchmark (`scripts/bench.mjs`) are recorded in the QA output.

### Capacity policy

- Default: 1 GiB.
- `DSH_MEMORY_SEARCH_MAX_INDEX_MB=<MiB>` changes the limit (64–16384 MiB).
- `DSH_MEMORY_SEARCH_MAX_INDEX_MB=0` explicitly selects an unlimited index.
- Settings can choose the index database path (a directory or `.db` file). Saving moves the live `index.db` (and WAL sidecars) there and deletes the old files. The chosen location is stored at `${DSH_HOME}/storages/memory-search/location.json`.
- `DSH_MEMORY_SEARCH_DB=<absolute path>` is used when there is no UI override.

The index is a derivative cache. Eviction never modifies DSH session files; a manual rebuild rereads the source and converges to the configured budget again.

## Install

Requirements: DeepSeek Harness `0.1.1-rc.2` or later and its supported Node.js version (`^22.19 || >=24`).

Install the plugin into the Web profile, then restart Web (or your existing `dsh-tunnel` / gateway process, so the Host half reloads):

```sh
dsh plugin --profile web add github:the-thinker0/dsh-memory-search-plus
dsh web
```

**Local development install** from a clone of this repository:

```sh
dsh plugin --profile web add ./
```

The repository commits the required `lib/` artifacts, so these installs do not ask pnpm to run a package build script.

Open the actual URL printed after `dsh web:` in the startup output (default `http://127.0.0.1:3080`).

### Updates

DeepSeek Harness does **not** silently push plugin updates onto machines that already installed this package. A new GitHub release is not installed until the profile fetches it.

If you installed with `github:the-thinker0/dsh-memory-search-plus`, pull the latest commit and restart Web (or your existing `dsh-tunnel` / gateway):

```sh
dsh plugin --profile web add github:the-thinker0/dsh-memory-search-plus
```

`dsh plugin` forwards to pnpm in the profile directory, so `dsh plugin --profile web update dsh-memory-search-plus` also works when that subcommand is available. After the Host half changes, restart the process that loads plugins; a browser refresh is enough only for the Web client.

A `file:` / `link:` local checkout does not follow GitHub. Pull that clone yourself, then re-add or restart as you already do for development.

Optional: community tools such as [dsh-market](https://github.com/dsh-market/dsh-market) or [dsh-update-copilot](https://github.com/hezhongtang/dsh-update-copilot) can check GitHub HEAD against the pinned lockfile commit and run the same `dsh plugin add` for you. They still require an explicit click; nothing is auto-installed in the background.

Remove the plugin with:

```sh
dsh plugin --profile web remove dsh-memory-search-plus
```

> Note: the plugin registers `memory-search` as its Cordis plugin id; removing the package removes the whole plugin, including its index database (a prompt will confirm removal of the registered patch).

## How this differs

- **vs. built-in sidebar search / sidebar-conversation filters** (e.g. dsh-better-sidebar): those filter or search within the *currently open* conversation. Memory Search Plus runs its own cross-session full-text index and jumps to the exact message in any conversation.
- **vs. conversation-landmarks**: that plugin provides a navigation rail over the open conversation's turns. Memory Search Plus is content search — find *what was said*, then jump.
- **vs. official conversation search in the harness**: this is a community index of your local session storage with Chinese-aware segmentation and message-level targeting; nothing leaves your machine.

## Develop

```sh
pnpm install
pnpm run check   # typecheck + build (tsdown) + pack dry-run
```

Layout: the package is a Cordis Service plugin. `src/index.ts` is the Host half (SQLite FTS5 indexer + two-stage search API, exported as `{ name, inject, apply }` via `lib/index.js`); `src/client/` is the Web half (sidebar pill + results panel, delivered as `lib/client.js` wrapped in `window.__ModuleLoader__.load(...)`).

The **host API contract** — module interfaces (`tokenize`, `extractSessionEventText`, `anchorKeyFor`, `SearchIndex`, `computeSpans`/`buildSnippet`), RPC result shapes and the degradation guarantees — is specified as an executable acceptance contract in [tests/README.md](tests/README.md) (see its QA test contract section). Unit tests live under `tests/`; the reproducible benchmark is `scripts/bench.mjs` (`--impl lib/index.js` measures the real implementation).

## Privacy

Everything is local: session events are read from local DSH storage into a bounded SQLite derivative. The index file stays on the machine you choose (default under `DSH_HOME`, or a path set in Settings / `DSH_MEMORY_SEARCH_DB`). The plugin registers no network calls, analytics, telemetry, or external CDN assets. SQLite/FTS secure deletion is enabled, and only the newest corrupt-index backup generation is retained.

## Status

**v1.0.0.** This is a community project, not an official DeepSeek release. Current compatibility target: DeepSeek Harness `0.1.1-rc.2`.

## Feedback

Report bugs or request features in [Issues](https://github.com/the-thinker0/dsh-memory-search-plus/issues). Please search existing issues before opening a new one.

## License

[MIT](LICENSE)