# Retrieval index — Self-build prerequisite #4 — 2026-09-15

**Lane:** id 9 (vibecoder-standalone-mavis)
**Commits:**
- 0054318 b3038: retrieval index — generator + search endpoint
- 2f9d585 b3038b: refresh retrieval index after VPS smoke test (18935 entities)

## What ships

1. **tools/build-index.mjs** (NEW, 12.1KB)
   - Walks `src/` + `server.cjs` + `vsil-renderer.cjs` (default)
   - Extracts entities via regex: `function`, `class`, `type`/`interface`, `const/let/var`, `exports.X`, `module.exports = {...}`, React components (PascalCase or `React.FC<...>` annotation), and routes (`app.get|post|put|delete`)
   - Walks back from each entity to find leading `//` / `/** */` comments for the summary
   - Atomic write: tmp + rename
   - Skip-on-fail: any file that throws is logged to `output.skipped` and processing continues
   - CLI: `node tools/build-index.mjs [--source DIR]... [--include FILE]... [--output PATH]`
   - Idempotent: same input → same output modulo timestamp

2. **.index/index.json** (NEW, 4.39MB)
   - 18,935 entities from 690 files
   - kinds: const (14999), component (360), function (2677), type (647), class (64), route (188)
   - Each entity: `{ name, kind, file, line, exported, summary }`
   - Top-level: `{ generated_at, source_head, entities, files, skipped }`
   - `files[<relpath>] = { entities: [<names>], line_count: <int> }`

3. **server.cjs** (+30 lines, integration only)
   - `GET /api/agent/index/search?q=&limit=` — fuzzy lookup with token scoring
   - `GET /api/agent/index/refresh` — regenerate + reload in-memory cache
   - Both delegate to `bridge/routes/agent.cjs` named exports (no new shared-lib file)

4. **bridge/routes/agent.cjs** (+200 lines, merged)
   - Header now mentions BOTH id 10's clone+push (PROMPT #32) AND id 9's retrieval (PROMPT #34)
   - Houses the canonical search/refresh/scoring/tokenize/cache logic
   - Module exports: default mount function + `search`, `refresh`, `REPO_ROOT`, `INDEX_PATH` named exports
   - Server.cjs requires `./bridge/routes/agent.cjs` to avoid duplicating the logic

## Self-build prerequisite #4: DONE

Per dispatch:
- ✅ VibeCoder works end-to-end (PROMPT #1-15, ongoing)
- ✅ Clone endpoint (PROMPT #32 id 10)
- ✅ Push endpoint (PROMPT #32 id 10)
- ✅ Retrieval index (PROMPT #34 id 9 — this dispatch)

## Smoke tests on VPS

| Query | Endpoint | HTTP | Top result |
|---|---|---|---|
| `Canvas` | `/api/agent/index/search?q=Canvas&limit=3` | 200 | `canvas` const @ src/host/host-boot.js:931 (score 115), `Canvas` component @ src/host/editor/Canvas.tsx:107 (score 112) |
| `save_conversation` | `/api/agent/index/search?q=save_conversation&limit=3` | 200 | `POST /api/agent/save_conversation` route @ server.cjs:4578 (score 30), `safeEmail` const @ server.cjs:4586 (score 15) |
| `call_groq` | `/api/agent/index/search?q=call_groq&limit=3` | 200 | empty (`count: 0`, scanned: 18935) — Python function not in JS index |
| `renderVSILScene` | `/api/agent/index/search?q=renderVSILScene&limit=3` | 200 | `renderVSILScene` @ vsil-renderer.cjs:156 (score 115, exported=true) |
| refresh | `GET /api/agent/index/refresh` | 200 | regenerated 18,935 entities, source_head=`0054318c3b4f9d1bd378a60140f9f3da192b53e0` |

## Constraints honoured

- [x] Index generation idempotent (regenerating produces same content modulo timestamps).
- [x] Search endpoint returns empty `{ok:true, count:0, results:[]}` (NOT 500) when query has no matches.
- [x] Path guard: `REPO_ROOT` is the only escape — index reads pinned to `.index/index.json` under REPO_ROOT.
- [x] Only modified/created files inside the allowed paths: `tools/`, `.index/`, `server.cjs`, `bridge/routes/agent.cjs`.
- [x] Generator skips files that throw and logs them to `.skipped`.
- [x] Did not retry smoke tests on failures (none failed).
- [x] VPS touches ended with: `git pull --ff-only origin main` (success) + `systemctl restart vibecoder-bridge` (success) + 4 endpoint smoke tests (all 200).

## Call-out: `call_groq` not in JS index

The dispatch's Phase 4e expects a `call_groq` query to "find runner function." However, `call_groq` is a Python function in `/opt/vibecoder-bridge/diagnostics/groq_runner_v2.py` — it is NOT part of the vibecoder-standalone JS/TS source. The search correctly returns 0 results (HTTP 200) per the constraint "Search endpoint must not block on missing index — return empty result, not 500."

If the operator wants Python files indexed, that's a future dispatch (add `.py` extension to `CODE_EXTS` + a Python AST/regex extractor).

## Conflict resolution during rebase

Local branch had to integrate id 10's clone+push commits (`c692adb` + `f25cd30`) before push would succeed. The rebase produced a content conflict in `bridge/routes/agent.cjs`:

- id 10's side: 175-route scaffold + clone + push (no search/refresh)
- id 9's side: search + refresh + named exports + scaffold

The conflict was resolved by writing a merged version that:
1. Carries id 10's clone+push routes AS-IS (with minor `fs.readdirSync` tweak — `fs.promises.readdir` was inside an `async` mount function but was accidentally non-awaited in id 10's original; the sync version is clearer here)
2. Adds the search/refresh logic + named exports from id 9
3. Updates the header comment to reflect BOTH contributors

`server.cjs` auto-merged cleanly (different routes, no conflict).

## VPS state after deploy

| Anchor | Value |
|---|---|
| /opt/vibecoder-bridge HEAD | 2f9d585 |
| Bridge PID | 1293624 |
| Bridge uptime | since 2026-09-15 17:15:25 UTC |
| deploy-gate | PASS (status=0/SUCCESS) |
| fv-gallery pull | PASS |