dsh-llm-wiki

Readme

# dsh-llm-wiki

[简体中文](README.zh-CN.md)

A workspace-scoped, local-first LLM Wiki plugin for DeepSeek Harness. It imports text from the active Harness workspace, builds a disposable local search index, and gives the main agent explicit tools to search and maintain sourced Wiki pages.

## What it does

- Isolates data by the canonical realpath of `exec.agent.session.header.cwd`.
- Keeps Markdown and text files as the source of truth; the PGlite index can be deleted and rebuilt.
- Combines a deterministic 384-dimensional feature-hashing embedding, PostgreSQL full-text search, CJK substring fallback, and `[[wikilink]]` graph signals with reciprocal-rank fusion.
- Builds graph edges deterministically from Wiki links and same-page entity co-occurrence.
- Uses the main Harness agent for Wiki maintenance. It does not make hidden nested model calls, so model inputs and outputs remain in the normal session log.
- Prevents accidental replacement with read-before-write SHA-256 checks.

## Requirements

- Node.js `^22.19.0` or `>=24.0.0`
- DeepSeek Harness `0.1.0-rc.6` or a compatible release
- pnpm 11

## Install from GitHub

Install the plugin into a Harness profile:

```sh
dsh plugin --profile headless add github:wangning19940904/dsh-llm-wiki
```

The repository ships TypeScript source and uses `prepare` to produce `dist/` during a GitHub install. pnpm blocks unreviewed dependency build scripts. If installation reports that the plugin build was ignored, review it and add this entry to the profile's `pnpm-workspace.yaml` before installing again:

```yaml
allowBuilds:
  dsh-llm-wiki: true
```

The bundle patch mounts `dsh-llm-wiki` and defaults `dataRoot` to `dshHomePath('llm-wiki')`.

## Tool workflow

1. `llm_wiki_search` searches existing knowledge before the agent answers.
2. `llm_wiki_import` copies relative `.md`, `.markdown`, and `.txt` files from the current workspace into `raw/docs`.
3. `llm_wiki_ingest` updates the derived index. New or changed sources attach a logged maintenance notice for the agent.
4. The agent reads sources with `llm_wiki_read` and writes sourced pages under `wiki/sources`, `wiki/entities`, or `wiki/concepts` with `llm_wiki_write`.
5. `llm_wiki_graph` explores entities, mentioning pages, relations, and paths up to three hops.

Search results include plugin-relative paths, source line ranges, excerpts, the fused score, and each signal's contribution. A successful write is indexed immediately.

## Data layout

Each canonical workspace gets a separate directory:

```text
<dataRoot>/workspaces/<workspace-id>/
├── workspace.json
├── purpose.md
├── schema.md
├── raw/docs/
├── wiki/
│   ├── index.md
│   ├── log.md
│   ├── sources/
│   ├── entities/
│   └── concepts/
└── .index/
```

`raw/docs` and `wiki` are durable knowledge. `.index` only contains derived PGlite data and is safe to remove; the next full `llm_wiki_ingest` rebuilds it.

## Configuration

The included patch supplies all defaults:

```yaml
- insert:
    - id: llm-wiki
      name: dsh-llm-wiki
      config:
        dataRoot: !!js dshHomePath('llm-wiki')
        chunkSizeChars: 1600
        chunkOverlapChars: 200
        embeddingDimensions: 384
        importMaxFiles: 50
        importMaxBytesPerFile: 2000000
        readMaxBytes: 100000
        searchDefaultLimit: 8
        searchMaxLimit: 20
        rrfK: 60
        vectorWeight: 1
        keywordWeight: 1
        graphWeight: 0.6
        maintenanceBatchSize: 5
```

Invalid integer bounds, negative or non-finite weights, an overlap greater than or equal to the chunk size, and a default search limit greater than its maximum stop plugin activation.

## Safety model

- Every tool requires an agent session cwd; otherwise it returns `WIKI_WORKSPACE_REQUIRED`.
- Imports reject absolute paths, traversal, directories, files resolved outside the workspace through symlinks, unsupported extensions, invalid UTF-8, NUL-containing binary data, and configured size/count overages.
- Reads are limited to `raw/docs` and `wiki` and bounded by `readMaxBytes`.
- Writes are limited to Markdown files under `wiki`. Creating a page needs no digest; replacing a page requires the latest digest returned by `llm_wiki_read`. Missing and stale observations return `WIKI_NOT_OBSERVED` and `WIKI_STALE`.
- Database handles close when the Cordis plugin is unloaded.

## MVP limits

This release intentionally has no Web UI, MCP server, trace system, background model task, PDF/image conversion, standalone CLI, or npm publication. It supports local UTF-8 Markdown and text files only. Feature-hashing embeddings favor predictable offline behavior over semantic quality from a neural embedding model.

## Development

```sh
pnpm install
pnpm lint
pnpm typecheck
pnpm test
pnpm build
pnpm pack
pnpm verify:package
```

`verify:package` packs the project, installs the tarball into a temporary Harness profile, and confirms that `--dump-config` contains the mounted plugin.

## License

MIT