dsh-browser-companion

Readme

# dsh-browser-companion

<p align="center">
  <strong>English</strong> | <a href="README.zh-CN.md">简体中文</a>
</p>


<p align="center">
  <img src="assets/icon.png" alt="dsh-browser-companion" width="120" height="120">
</p>


A personal DeepSeek Harness (DSH) browser plugin.

It gives your agent a **dedicated, persistent, visible browser**:

- You can watch what the agent is doing in a real browser window.
- Logins are done by you in the popup window; passwords and OAuth secrets never pass through the agent.
- Cookies and login state are saved in the same profile, so logins persist across restarts.

## Features

- ✅ Persistent browser profile (cookies / localStorage / login state)
- ✅ Visible window mode (default `headless: false`)
- ✅ Human-in-the-loop login: `browser_login` opens a window and you type your password / complete OAuth yourself
- ✅ Agent tools: open, snapshot, click, type, press, screenshot, see, select, check, upload, hover, scroll, tabs, status
- ✅ Security guard: `browser_type` refuses to fill password fields
- ✅ `browser_see` uses a local vision model (OpenAI-compatible) so the agent can understand images / visual layouts when needed
- ✅ Based on [agent-browser](https://github.com/vercel-labs/agent-browser), so the hard browser automation is battle-tested

## Out-of-the-box

The plugin works with zero configuration for basic browsing:

- Uses the bundled browser (agent-browser's Chromium) by default
- Uses a default persistent profile under `~/.dsh/browser-profile`
- All basic tools work: open, snapshot, click, type, tabs, etc.

Optional settings can be changed in the DSH WebUI plugin settings:

- `executablePath`: set to your real Chrome/Edge to avoid login blocks (e.g. Google)
- `visionBaseUrl` / `visionModel`: enable `browser_see` with a local vision model
- `profilePath`: use a custom browser profile location

If `browser_see` is not configured, it returns a friendly message instead of breaking the agent.


## Install

### 1. Install dependencies

```bash
npm install -g agent-browser
agent-browser install   # first-time Chromium download
```

### 2. Install the plugin

From npm or GitHub (once published):

```bash
dsh plugin --profile web add dsh-browser-companion
```

> **pnpm users:** when installing from GitHub, pnpm may block `agent-browser`'s build script.  
> Add this to your profile's `pnpm-workspace.yaml` and re-run:
> ```yaml
> allowBuilds:
>   agent-browser: true
> ```


Or for local development, add this to `~/.dsh/profiles/web/cordis.patch.yml`:

```yaml
- insert:
    - id: dsh-browser-companion
      name: 'file:///absolute/path/to/dsh-browser-companion/src/index.js'
      config:
        profilePath: '/home/your-name/.dsh/browser-profile'
        headless: false
        executablePath: '/path/to/your/chrome'
        browserArgs: '--disable-blink-features=AutomationControlled'
        defaultTimeout: 30000
```

See [examples/dsh-browser-companion.config.yml](examples/dsh-browser-companion.config.yml) for a full sample config with placeholders.

> ⚠️ Never commit real paths, tokens, or personal profile data. Use the sample config and replace values locally.

### 3. Restart DSH

```bash
dsh web
```

## Agent Tools

| Tool | Description |
|---|---|
| `browser_open` | Open a URL (visible window by default) |
| `browser_snapshot` | Get an accessibility snapshot with `@e1` refs |
| `browser_click` | Click an element |
| `browser_type` | Type text (never into password fields) |
| `browser_press` | Press a key |
| `browser_screenshot` | Save a screenshot locally |
| `browser_see` | Screenshot + local vision model description (use sparingly) |
| `browser_get_text` | Read element text |
| `browser_get_url` / `browser_get_title` | Get URL / title |
| `browser_wait` | Wait for time / element / URL |
| `browser_back` / `forward` / `reload` | Navigation |
| `browser_select` | Select a dropdown option |
| `browser_check` / `browser_uncheck` | Check / uncheck a checkbox |
| `browser_upload` | Upload files |
| `browser_hover` | Hover an element |
| `browser_scroll` | Scroll the page |
| `browser_tabs` / `browser_new_tab` / `browser_close_tab` / `browser_switch_tab` | Tab management |
| `browser_status` | Browser status |
| `browser_login` | Open a visible login window for the user |
| `browser_wait_login` | Wait until login completes (cookie / URL) |
| `browser_close` | Close the browser (profile is kept) |

## Typical Login Flow

```text
Agent: I need to log in to a site.
Agent calls browser_login("https://example.com/login")
→ You see a popup and log in manually
→ You tell the agent "done"
→ Agent continues with the same cookies
```

## How the Agent “Sees” a Page

- **Default:** `browser_snapshot` — fast, free accessibility tree with refs.
- **Only when needed:** `browser_see` — screenshot + local vision model, for images / visual layout / complex UI.

## Configuration

| Key | Default | Description |
|---|---|---|
| `profilePath` | `~/.dsh/browser-profile` | Persistent profile directory |
| `headless` | `false` | Show browser window |
| `defaultTimeout` | `30000` | Per-operation timeout (ms) |
| `agentBrowserPath` | auto-detect | Override agent-browser CLI path |
| `executablePath` | auto | Chrome/Edge executable path (recommended for Google login) |
| `browserArgs` | `--disable-blink-features=AutomationControlled` | Extra browser launch args |
| `visionBaseUrl` | `http://127.0.0.1:1234/v1` | Local vision model endpoint |
| `visionModel` | `your-vision-model` | Local vision model name |

## Security Notes

- The agent is instructed never to fill password fields.
- Passwords and OAuth secrets are entered by the user in the visible browser window.
- For stronger protection, future versions can add encrypted cookie storage and proxy-based credential injection.

## Development

```bash
npm install
npm test
```

## License

MIT