# Vibe Agents โ€” Full Execution Blueprint v2.0 **Date:** 2026-09-08 15:10 Europe/Paris (= 14:10 BST Wales) **Author:** id9 (Mavis) โ€” using Vibe Agents **Status:** ๐Ÿ”ด **DRAFT v2 โ€” waiting for operator approval** **Supersedes:** v1 at `/workspace/.mavis/plans/vibe-agents-execution-blueprint-2026-09-08.md` (2026-09-08 14:36, never executed) --- ## Changes from v1 (operator directive 2026-09-08 15:06) | # | Change | Section | |---|---|---| | 1 | **Reviewer selection is now CAPABILITY-BASED, not cost-based.** Free models can win if they score higher. Reviewer fallback follows capability order. | ยงF, ยงG, Phase 7, ยงM | | 2 | **All 5 Agent Tabs (Builder, Planner, Inspector, Auditor, Reviewer) are now explicitly capability-aware.** Each has documented inputs, outputs, responsibilities. | ยงH, ยงM | | 3 | **Dynamic provider probing replaces hardcoded probe results.** Architecture is provider-agnostic. | New Phase 0.5 | | 4 | **Agent Tabs audit** โ€” verified each agent's responsibilities, inputs, outputs, capability-awareness, fallback/hot-swap alignment. | ยงM | **Plus: extra models integrated** โ€” `deepseek-ai/deepseek-v4-pro-0813` (verified working on NVIDIA free tier, 2026-09-08 15:05 probe) and `minimaxai/minimax-m3` (also working, same key) are now in scope for the cascade. Tier 1.5 can be either Nemotron-3-nano or DeepSeek V4 Pro; both work, DeepSeek is bigger, Nemotron has `reasoning_content`. --- ## A. Audit cross-reference (what the operator plan addresses) | Audit finding | Operator plan section | Status | |---|---|---| | โŒ Dynamic scaffolding engine missing | ยง3 Scaffolding Engine | Addressed | | โŒ DeepSeek adapter missing | ยง1 Routing Config | **NEW: explicitly added** (DeepSeek V4 Pro on NVIDIA free tier verified 2026-09-08 15:05) | | โŒ OpenRouter free adapter missing | ยง1 Routing Config ("optional") | Deferred โ€” explicit decision | | โŒ Hot-swap logic missing | ยง5 Hot-Swap Engine | Addressed | | โš ๏ธ operator context not injected | ยง3 Scaffolding Engine โ†’ operator context block | Addressed | | โš ๏ธ workspace context partial (PB only) | ยง3 Scaffolding Engine โ†’ workspace context block | Addressed | | โš ๏ธ FVW.detectReflex detected but ignored | ยง3 โ†’ reflex routing | Addressed | | โŒ No error-recovery scaffolding | ยง3 โ†’ error-recovery templates | Addressed | | โŒ Tool synthesis hardcodes Groq | ยง1 Routing Config (winning model for r2/r3) | Addressed | | โŒ Reviewer role missing | ยง2 Reviewer Role | Addressed (now CAPABILITY-BASED per #1) | | โŒ No hot-swap or fallback UI | ยง7 UI Config | Addressed | | โŒ No audit trail for cascade changes | ยง4 Backend Logging โ†’ AuditTrail | Addressed | | โŒ No scaffolding observability | ยง4 โ†’ PromptArchive, AgentActivity | Addressed | | โŒ Reviewer selected by price, not capability (v1 had this) | ยงF (Routing) | **NEW: fixed in v2** | | โŒ Agent Tabs not aware of capability ranking (v1 had this) | ยงH + ยงM | **NEW: fixed in v2** | | โŒ Probe results hardcoded into cascade | Phase 2 | **NEW: fixed via Phase 0.5 dynamic probe** | **15 of 15 audit gaps addressed** in v2. 1 explicitly deferred (OpenRouter optional). --- ## B. Gap analysis (what v2 still doesn't say) | Gap | Risk | Mitigation | |---|---|---| | No mention of native Mavis Tier-4 fallback | Memory says "Tier 4: native Mavis" โ€” never wired | Add as an OPTIONAL "native" provider (free, always available, no API key) | | How `capability_score` is computed | Need an explicit formula, not just a label | Document the 7 scoring components + how they map to per-model scores (see ยงM.4) | | Probe cooldown window | Too-frequent probes burn free tier quota | Probe-on-boot + every 6h + on-demand via API | | Reviewer deadlock if all candidates score 0 | If capability_score is 0 for everything, no Reviewer selected | Default to highest score (even if 0) + log a warning | | Hot-swap during in-flight Reviewer | Reviewer already running on a model that's toggled off | Reviewer completes with old model (in-flight immunity, same as Phase 1) | | DeepSeek V4 Pro is bigger than Nemotron-3-nano โ€” slower but sharper | Cost is still 0 but latency may be higher | Default 1500ms fire stagger handles it; quality-weighted picks the deeper one | | `minimaxai/minimax-m3` โ€” is this actually MiniMax M3? | Could be a different model with similar name | Add to probe-on-boot + flag for operator to verify provenance | --- ## C. Phase breakdown (10 phases + new Phase 0.5) ### Phase 0 โ€” Audit & Plan Lock *(done in v1)* - **Output:** v1 blueprint file - **Proof of work:** `/workspace/.mavis/plans/vibe-agents-execution-blueprint-2026-09-08.md` exists - **Status:** SUPERSEDED by v2 ### Phase 0.5 โ€” Dynamic Provider Probing *(NEW in v2)* - **Deliverable:** on-boot probe that refreshes the provider catalog, no hardcoded results - **Files:** - `/opt/vibecoder-bridge/scripts/probe-all-providers.cjs` (new, ~200 lines) - `/opt/vibecoder-bridge/config/provider-registry.cjs` (new, ~100 lines) - `/opt/vibecoder-bridge/scripts/refresh-probe.sh` (cron entry, every 6h) - **New PB collection:** ```js // vibechat_provider_probe_results { name: 'vibechat_provider_probe_results', fields: [ { name: 'ts', type: 'number' }, { name: 'model_id', type: 'string', max: 200 }, { name: 'provider', type: 'string', max: 40 }, { name: 'endpoint', type: 'string', max: 500 }, { name: 'http_status', type: 'number' }, { name: 'latency_ms', type: 'number' }, { name: 'has_reasoning_field', type: 'bool' }, { name: 'capability_score', type: 'number' }, // 0-1, derived from probe + history { name: 'capability_components', type: 'json' }, // 7-component breakdown { name: 'cost_tier', type: 'string', max: 20 }, // 'free' | 'paid' (NOT used for selection) { name: 'enabled', type: 'bool' }, { name: 'tier_position', type: 'number' }, // 1, 1.5, 2, 3, 4 { name: 'fire_stagger_ms', type: 'number' }, { name: 'last_refresh', type: 'number' }, ] } ``` - **Agent:** Builder (id9-build) + Inspector (validates probe data shape) - **Probe strategy:** - On bridge boot: probe every model in `provider-registry.cjs` (the active config) - Every 6h: re-probe all enabled models - On-demand: `POST /api/agent/config/probe-now` (operator trigger) - Test prompt: `"OK"` with `max_tokens: 25` (mirrors the live probe I ran 2026-09-08 15:05) - Latency, status, has_reasoning_field recorded - capability_score recomputed from probe + historical ModelUsage (see ยงM.4 formula) - **Architecture is provider-agnostic:** adding a new model = add a row to `provider-registry.cjs` (model_id, provider, endpoint, env_var_name, has_reasoning_field). No code change. - **Proof of work:** - `node scripts/probe-all-providers.cjs` runs on bridge boot, populates `vibechat_provider_probe_results` with one row per model - `/api/agent/config/probe-now` returns 200, refreshes all rows - Capability scores visible in `GET /api/agent/config/capability-scores` (ordered by score desc) - The probe finds DeepSeek V4 Pro, Nemotron-3-nano, minimaxai/minimax-m3 as `enabled=true`, others as `enabled=false` ### Phase 1 โ€” Backend Schema (PocketBase) *(Same as v1, plus the new `vibechat_provider_probe_results` collection from Phase 0.5)* - **Deliverable:** 8 new PB collections (additive, idempotent) โ€” 7 from v1 + 1 from Phase 0.5 - **Files:** - `/opt/operator/panel/src/migrate-add-vibe-agents-tables.js` (new, includes probe_results) - `/opt/operator/panel/migrate.js` (modified to call new migration) - **Agent:** Builder (id9-build) - **Inspector:** id9-inspect (verifies migration is idempotent, runs twice) - **Proof of work:** `node migrate.js` runs twice, second run is no-op. PB has 8 new collections. SHOW BEFORE WRITING: ```js // Schema 1: vibechat_model_usage { name: 'vibechat_model_usage', fields: [ { name: 'ts', type: 'number' }, { name: 'request_id', type: 'string', max: 80 }, { name: 'session_id', type: 'string', max: 80 }, { name: 'user_id', type: 'string', max: 80 }, { name: 'model_id', type: 'string', max: 200 }, { name: 'provider', type: 'string', max: 40 }, { name: 'tier', type: 'number' }, { name: 'input_tokens', type: 'number' }, { name: 'output_tokens', type: 'number' }, { name: 'reasoning_tokens', type: 'number' }, { name: 'latency_ms', type: 'number' }, { name: 'http_status', type: 'number' }, { name: 'tier_winner', type: 'bool' }, { name: 'capability_score_at_call', type: 'number' }, // NEW: snapshot of capability_score at call time { name: 'finish_reason', type: 'string', max: 40 } ] } ``` *(Full schemas for all 8 collections in ยงE)* ### Phase 2 โ€” Routing Config System (now reads from probe data, not hardcoded) - **Deliverable:** config loader/saver that reads `vibechat_provider_probe_results` for the active cascade - **Files:** - `/opt/vibecoder-bridge/config/cascade-config.cjs` (new, ~250 lines โ€” reads from PB probe table) - `/opt/vibecoder-bridge/config/defaults.cjs` (new, current 5-tier shape as default seed) - `/opt/vibecoder-bridge/config/provider-registry.cjs` (from Phase 0.5) - **Agent:** Builder (id9-build) + Inspector (validates config + probe integration) - **Architectural guarantee (per change #3):** defaults file contains the SHAPE (number of tiers, fire stagger ranges) but the actual `model_id` values come from probe results. Adding a model to the registry does NOT require editing defaults.cjs. - **Proof of work:** ```js const c = require('./config/cascade-config'); c.load(); // returns config built from probe + defaults console.log(JSON.stringify(c.active(), null, 2)); // Shows: 5 tiers, but model_ids are from probe (DeepSeek V4 Pro IF probe says enabled, etc.) ``` - **Backward compat:** until Phase 4, the bridge uses defaults if probe is unavailable; config-aware code is wired but doesn't change behavior ### Phase 3 โ€” Scaffolding Engine *(Same as v1 + capability-aware scaffold block per change #2)* - **Deliverable:** `scaffoldPrompt({ base, operator, workspace, taskType, errorContext, reflex, lane, capabilityScores })` function - **Files:** - `/opt/vibecoder-bridge/scaffolding/prompt-builder.cjs` (new, ~180 lines โ€” includes capability block) - `/opt/vibecoder-bridge/scaffolding/task-templates.cjs` (new, R1-R7) - `/opt/vibecoder-bridge/scaffolding/reflex-blocks.cjs` (new, R1-R7) - `/opt/vibecoder-bridge/scaffolding/error-recovery.cjs` (new, 3 patterns) - `/opt/vibecoder-bridge/scaffolding/capability-block.cjs` (NEW, ~40 lines โ€” injects capability scores for Inspector/Reviewer) - **Agent:** Builder (id9-build) - **Inspector:** id9-inspect (verifies block composition, no prompt > 32K tokens, capability block is consistent) - **Auditor:** id9-audit (compares to FvW v8 doctrine, logs capability scores) - **The new capability block** (per change #2): ```js // capability-block.cjs function capabilityBlock(scores) { if (!scores || scores.length === 0) return ''; const top3 = scores.slice(0, 3).map(s => `- ${s.model_id}: ${s.capability_score.toFixed(3)} (${s.cost_tier})` ).join('\n'); return `## CAPABILITY RANKING (top 3 by score, free models can win)\n${top3}\nUse this ranking to inform your reasoning quality bar.`; } ``` - **Proof of work:** ```js const s = scaffoldPrompt({ base: 'You are FreshVibe.', operator: { name: 'M', lane: 'operator' }, workspace: { project: 'vibecoder', openFiles: ['x.ts'] }, taskType: 'R3', reflex: 'R3', capabilityScores: [ { model_id: 'deepseek-ai/deepseek-v4-pro-0813', capability_score: 0.94, cost_tier: 'free' }, { model_id: 'nvidia/nemotron-3-nano-omni-30b-a3b-reasoning', capability_score: 0.88, cost_tier: 'free' }, ], }); // Assert: s includes '## CAPABILITY RANKING', '## OPERATOR', '## WORKSPACE', '## TASK (R3)', reflex block // Assert: s.length < 32000 ``` ### Phase 4 โ€” Cascade Refactor *(Same as v1 + capability_score snapshot per change #1 +2)* - **Deliverable:** `server.cjs` chat handler reads config + probe, uses scaffoldPrompt with capability block, persists ModelUsage with `capability_score_at_call` - **Files:** - `/opt/vibecoder-bridge/server.cjs` (modified, ~180 line changes in chat handler) - **Agent:** Builder (id9-build) + Planner (id9-plan) for ordering - **Inspector:** id9-inspect (verifies no behavior change for default config) - **Reviewer:** id9-review (final-pass on the diff) - **Auditor:** id9-audit (logs capability scores used per request) - **Critical changes from v1:** - tool synthesis (r2/r3) now uses WINNING provider/model from Phase 1, not hardcoded `openai/gpt-oss-20b` - ModelUsage row includes `capability_score_at_call` (snapshot for audit trail) - Prompt includes the capability block from ยงM.2 - **Proof of work:** - Default-config chat test: same response as before (regression test) - PB ModelUsage table has 1 row per /api/agent/ai/chat call with `capability_score_at_call` populated - Tool call test: tool synthesis uses same model as Phase 1 winner - diff: `git diff server.cjs | wc -l` shows < 250 lines changed ### Phase 5 โ€” Hot-Swap Engine *(Same as v1 + respects capability_score, not cost_tier)* - **Deliverable:** runtime endpoint `POST /api/agent/config/swap` to toggle a tier on/off - **Files:** - `/opt/vibecoder-bridge/server.cjs` (new endpoint, ~60 lines) - `/opt/vibecoder-bridge/config/cascade-config.cjs` (modified, atomic toggle + audit) - **Agent:** Builder (id9-build) - **Inspector:** id9-inspect (verifies atomic update, no in-flight request disruption, capability_score unchanged) - **Auditor:** id9-audit (logs hot-swap events in audit trail) - **Proof of work:** ```bash # 1. Toggle DeepSeek V4 Pro off via hot-swap curl -X POST http://127.0.0.1:3003/api/agent/config/swap \ -H 'X-Mavis-Token: ...' -H 'Content-Type: application/json' \ -d '{"tier": 1.5, "model_id": "deepseek-ai/deepseek-v4-pro-0813", "enabled": false}' # 2. Send a chat # 3. Verify ModelUsage has no 'deepseek' rows for this request # 4. Toggle back on, verify deepseek rows return ``` ### Phase 6 โ€” Fallback Engine *(Same as v1 + capability order in fallback chain)* - **Deliverable:** ordered fallback chain follows capability_score (highest โ†’ lowest), independent of hot-swap - **Files:** - `/opt/vibecoder-bridge/server.cjs` (Phase 2 fallback already exists; refactor to read config) - **Agent:** Builder (id9-build) + Auditor (id9-audit confirms fallback order) - **Inspector:** id9-inspect (regression: existing GLM fallback still works) - **Proof of work:** disable all top-3 capability models via hot-swap, send chat, verify next-highest scores answer, log says "fallback tier used (capability order)" ### Phase 7 โ€” Reviewer Role *(UPDATED in v2 โ€” capability-based selection)* - **Deliverable:** Reviewer pass runs after Phase 1 winner if `review_enabled = true`. Reviewer is selected by **highest capability_score**, NOT lowest cost. - **Files:** - `/opt/vibecoder-bridge/server.cjs` (new phase 2, ~90 lines โ€” capability-aware Reviewer) - `/opt/vibecoder-bridge/scaffolding/reviewer.cjs` (new, ~120 lines โ€” receives cascade winner + scores) - **Agent:** Builder (id9-build) + Reviewer (id9-review) - **Reviewer Agent (the meta-role):** tests the Reviewer pass by calling `id9-review` itself - **Capability-based selection (per change #1, locked):** - If a free model scores higher than a paid model, the free model wins - Capability scoring: 7 components (reasoning_depth, correction_accuracy, consistency, long_form_stability, clarity, error_detection, reasoning_content) โ€” see ยงM.4 - Reviewer fallback follows capability order, not billing order - **Proof of work:** - Config with `review_enabled: true`, no `review_model_id` set - Send chat, verify ReviewerPass row in PB + log says "reviewer selected by capability_score: ()" - Send chat with `review_enabled: false`, verify no ReviewerPass row - Force a scenario where the highest-capability model is paid but a free model has score 0.95 vs paid 0.90 โ†’ free model selected - Verify Reviewer receives the cascade winner + ALL capability scores in its input (not just the winner) ### Phase 8 โ€” UI Config Panel *(Same as v1, plus capability score column)* - **Deliverable:** Edge Panel section "Routing" with toggles, provider pills, capability score column, reviewer selector - **Files:** - `/workspace/vibecoder-standalone/src/cms/edge-panel/edge-panel.js` (VENDORED โ€” do NOT modify) - `/workspace/vibecoder-standalone/src/vcs/injectRoutingConfig.js` (new, like injectChatIcons pattern) - `/workspace/vibecoder-standalone/src/vcs/routing-config.css` (new) - **Pattern:** same as r2583 dual-mode VibeChat โ€” inject into vendored Edge Panel without forking - **Agent:** Builder (id9-build) - **Inspector:** id9-inspect (Playwright: open Edge, navigate to Routing section, click toggle, verify capability score column shows) - **Proof of work:** - Playwright screenshot showing 5 model toggles, 3 provider toggles, capability score column (e.g. "0.94 deepseek-v4-pro, 0.88 nemotron-3-nano"), reviewer selector - Toggle "NVIDIA off" โ†’ chat uses only Groq (verified in ModelUsage) - Click "Run probe now" โ†’ probe runs, capability scores refresh in UI ### Phase 9 โ€” Integration Tests (Playwright E2E) *(Same as v1 + 3 new test cases for capability-based selection)* - **Deliverable:** `tests/vibe-agents-e2e.spec.ts` covering 15 scenarios (was 12) - **Agent:** Builder (id9-build) + Auditor (id9-audit) - **Proof of work:** 15/15 scenarios pass 1. Default config: Groq wins 2. NVIDIA-only: Nemotron answers 3. GLM-only: GLM answers (after GLM keys refreshed) 4. Reviewer pass: ReviewerPass row created 5. Reviewer disabled: no ReviewerPass row 6. Hot-swap: NVIDIA off โ†’ no nvidia rows 7. Hot-swap: NVIDIA on โ†’ nvidia rows return 8. Fallback: all Groq models off โ†’ GLM answers 9. Scaffolding: operator context in prompt 10. Scaffolding: workspace context in prompt 11. Scaffolding: error-recovery on retry 12. Reflex: R3 prompt has R3 block, R5 doesn't 13. **NEW: Reviewer selection โ€” free model with higher capability_score wins over paid lower-score model** 14. **NEW: Probe-on-boot populates vibechat_provider_probe_results** 15. **NEW: Capability block present in Reviewer prompt** ### Phase 10 โ€” Migration & Rollout *(Same as v1, plus the new probe_results collection)* - **Deliverable:** existing data preserved, new tables empty, no breaking changes, probe runs on first boot - **Files:** - `/opt/operator/panel/src/migrate-add-vibe-agents-tables.js` (Phase 1 migration, idempotent) - `/opt/vibecoder-bridge/scripts/migrate-cascade-config.js` (new, seeds default config if PB has no rows) - `/opt/vibecoder-bridge/scripts/probe-all-providers.cjs` (Phase 0.5, runs on first boot) - **Agent:** Builder (id9-build) - **Proof of work:** - All existing tests still pass - PB has 8 new collections, 0 rows in each (clean start) - First boot: probe runs, populates vibechat_provider_probe_results with at least 3 enabled models - Default config loaded: `node -e "require('./config/cascade-config').load()"` returns 5-tier cascade --- ## D. Dependency graph ``` Phase 0 (DONE) โ”€โ”€โ†’ Phase 0.5 (Probe) โ”€โ”€โ†’ Phase 1 (Schema + probe_results) โ”‚ โ–ผ Phase 2 (Config reads probe) โ”€โ”€โ”ฌโ”€โ”€โ†’ Phase 3 (Scaffolding + capability block) โ”œโ”€โ”€โ†’ Phase 5 (Hot-Swap) โ”œโ”€โ”€โ†’ Phase 6 (Fallback, capability order) โ””โ”€โ”€โ†’ Phase 7 (Reviewer, capability-selected) โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค โ–ผ โ–ผ Phase 4 (Cascade refactor) โ†โ”€โ”€ Phase 7 โ”‚ โ–ผ Phase 8 (UI) โ”€โ”€โ†’ Phase 9 (Tests) โ”€โ”€โ†’ Phase 10 (Migration) ``` **Critical path:** 0.5 โ†’ 1 โ†’ 2 โ†’ 3 โ†’ 4 โ†’ 9 โ†’ 10 **Parallelizable:** 5 + 6 + 7 can run in parallel after Phase 4 (all read from Phase 2 config) **New:** Phase 0.5 must complete BEFORE Phase 2 (Config depends on probe data) --- ## E. Schemas (8 collections, was 7 in v1) ### E.1 `vibechat_model_usage` โ€” UPDATED with `capability_score_at_call` ``` { ts: number, request_id: string(80), session_id: string(80), user_id: string(80), model_id: string(200), provider: enum, tier: number, input_tokens: number, output_tokens: number, reasoning_tokens: number, latency_ms: number, http_status: number, tier_winner: bool, capability_score_at_call: number, // NEW: snapshot of capability_score at call time finish_reason: enum } ``` ### E.2 `vibechat_agent_activity` *(Same as v1)* ### E.3 `vibechat_reviewer_passes` โ€” UPDATED ``` { ts: number, request_id: string(80), reviewer_model_id: string(200), reviewer_capability_score: number, // NEW original_model_id: string(200), original_capability_score: number, // NEW selection_rationale: text, // NEW: "selected by capability order, free beats paid" original_text: text, reviewed_text: text, delta_score: number, latency_ms: number } ``` ### E.4 `vibechat_prompt_archive` *(Same as v1)* ### E.5 `vibechat_fallback_config` *(Same as v1)* ### E.6 `vibechat_audit_trail` *(Same as v1)* ### E.7 `vibechat_workspace_context` *(Same as v1)* ### E.8 `vibechat_provider_probe_results` *(NEW from Phase 0.5)* ``` { ts: number, model_id: string(200), provider: enum, endpoint: string(500), http_status: number, latency_ms: number, has_reasoning_field: bool, capability_score: number, // 0-1, derived from probe + history capability_components: json, // 7-component breakdown cost_tier: string(20), // 'free' | 'paid' (NOT used for selection) enabled: bool, tier_position: number, fire_stagger_ms: number, last_refresh: number } ``` --- ## F. Routing logic (UPDATED for capability scoring) ```js // /opt/vibecoder-bridge/config/cascade-config.cjs async function executeCascade({ config, context, systemPrompt, mergedHistory }) { const tiers = config.cascade_order.filter(t => t.enabled); if (tiers.length === 0) throw new Error('cascade_empty'); // โ”€โ”€ Phase 1: parallel race across parallel tiers โ”€โ”€ const parallelTiers = tiers.filter(t => t.tier_type === 'parallel'); const phase1Results = await Promise.allSettled( parallelTiers.map(async t => { const stagger = config.stagger_ms[t.model_id] || 0; if (stagger > 0) await sleep(stagger); return callProvider({ model: t, context, systemPrompt: scaffoldPrompt({ ...context, base: systemPrompt, capabilityScores: parallelTiers.map(pt => ({ model_id: pt.model_id, capability_score: pt.capability_score, cost_tier: pt.cost_tier, })), }), mergedHistory, }); }) ); // โ”€โ”€ Quality-weighted selection (per capability_score, not cost) โ”€โ”€ const winners = phase1Results .map((r, i) => r.status === 'fulfilled' ? r.value : null) .filter(v => v && v.text && v.text.length > 10); if (winners.length === 0) return await executeFallbackChain({ ... }); // Score by: capability_score (PRIMARY) + text.length + thinking bonus // Capability_score is the dominant signal โ€” it's the model's measured quality const scored = winners.map(w => ({ ...w, score: (w.capability_score * 0.5) + (w.text.length / 1000) + ((w.thinking?.length || 0) / 1000 * 0.3), })); scored.sort((a, b) => b.score - a.score); const best = scored[0]; // โ”€โ”€ Log to ModelUsage (with capability snapshot) โ”€โ”€ await logModelUsage({ ...best, capability_score_at_call: best.capability_score, tier_winner: true }); // โ”€โ”€ Phase 2: Reviewer pass (CAPABILITY-BASED SELECTION) โ”€โ”€ if (config.review_enabled) { const reviewer = await selectReviewerByCapability(config); const reviewed = await runReviewer({ reviewer, original: best, context, allCapabilityScores: winners.map(w => ({ model_id: w.model_id, capability_score: w.capability_score, cost_tier: w.cost_tier, })), }); await logReviewerPass({ ... }); await logModelUsage({ ...reviewed, capability_score_at_call: reviewer.capability_score, tier_winner: false, tier: 'review' }); return reviewed; } return best; } async function selectReviewerByCapability(config) { if (!config.review_enabled) return null; if (config.review_model_id) { const m = config.cascade_order.find(t => t.model_id === config.review_model_id); if (m && m.enabled && await hasKey(m)) return m; } // CAPABILITY-BASED SELECTION (per change #1) // Pick the enabled model with the HIGHEST capability_score // โ€” NOT the cheapest, NOT the fastest, NOT the most-popular // Free models can win if they score higher const candidates = config.cascade_order .filter(t => t.enabled && t.capability_score != null && hasKey(t)); if (candidates.length === 0) return null; const sorted = candidates.sort((a, b) => b.capability_score - a.capability_score); return sorted[0]; } async function executeFallbackChain({ config, context, systemPrompt, mergedHistory }) { // Fallback follows capability order, not billing order (per change #1) const sequentialTiers = config.cascade_order .filter(t => t.tier_type === 'sequential' && t.enabled) .sort((a, b) => b.capability_score - a.capability_score); // highest first for (const tier of sequentialTiers) { const result = await callProvider({ model: tier, context, systemPrompt, mergedHistory }); if (result && result.text) { await logModelUsage({ ...result, capability_score_at_call: tier.capability_score, finish_reason: 'fallback' }); return result; } } throw new Error('all_tiers_exhausted'); } ``` --- ## G. Scaffolding logic (UPDATED with capability block) ```js // /opt/vibecoder-bridge/scaffolding/prompt-builder.cjs const TASK_TEMPLATES = { R1: '## TASK: Quick chat\n- Answer the user directly. No code, no tools unless asked.', R2: '## TASK: Read/inspect\n- Use read_file or list_files. No writes.', R3: '## TASK: Build / create / deploy\n- Use scaffold/diff/apply. Confirm with operator before deploys.', R4: '## TASK: Analyze / audit\n- Use read_file + read_files_for_context. Produce a structured report.', R5: '## TASK: Plan / commit\n- Emit a plan JSON block. No execution.', R6: '## TASK: Multi-step / workflow\n- Decompose into phases. Emit a plan first.', R7: '## TASK: Quick explain\n- Brief, in-conversation. No tools.', }; const REFLEX_BLOCKS = { R1: '## REFLEX: Stay in conversation lane. No tool calls.', R3: '## REFLEX: Operator mode โ€” confirm destructive actions before executing.', R5: '## REFLEX: Plan-only. Do not call tools.', // ... R2, R4, R6, R7 omitted for brevity }; // NEW: capability block (per change #2) function capabilityBlock(scores) { if (!scores || scores.length === 0) return ''; const sorted = [...scores].sort((a, b) => b.capability_score - a.capability_score); const top3 = sorted.slice(0, 3).map((s, i) => `${i+1}. ${s.model_id}: capability=${s.capability_score.toFixed(3)} (${s.cost_tier})` ).join('\n'); return `## CAPABILITY RANKING (top 3 by score โ€” free models can win if they score higher)\n${top3}\nUse this as your quality bar. The model that produced you was selected by capability order, not cost.`; } function scaffoldPrompt({ base, operator, workspace, taskType, errorContext, reflex, lane, capabilityScores }) { const blocks = [base]; if (operator) blocks.push(`## OPERATOR\nName: ${operator.name}\nLane: ${operator.lane || 'auto'}`); if (workspace) { blocks.push(`## WORKSPACE\nProject: ${workspace.project}\nOpen: ${workspace.openFiles?.join(', ') || 'none'}`); } if (capabilityScores && capabilityScores.length > 0) { blocks.push(capabilityBlock(capabilityScores)); // NEW } if (taskType && TASK_TEMPLATES[taskType]) blocks.push(TASK_TEMPLATES[taskType]); if (reflex && REFLEX_BLOCKS[reflex]) blocks.push(REFLEX_BLOCKS[reflex]); if (errorContext) { blocks.push(`## RECOVERY\nPrevious attempt failed: ${errorContext.message}\nTry a different angle.`); } return blocks.filter(Boolean).join('\n\n'); } ``` --- ## H. Agent role assignments (UPDATED with capability awareness) | Phase | Builder | Planner | Inspector | Auditor | Reviewer | |---|---|---|---|---|---| | 0 (done) | โ€” | id9 (Mavis) | โ€” | โ€” | โ€” | | 0.5 Probe | id9-build | id9 | id9-inspect (probe shape) | โ€” | โ€” | | 1 Schema | id9-build | id9 | id9-inspect | โ€” | โ€” | | 2 Config | id9-build | id9 | id9-inspect (capability fields valid) | โ€” | โ€” | | 3 Scaffold | id9-build | id9 | id9-inspect (capability block shape) | id9-audit (capability scores logged) | โ€” | | 4 Cascade | id9-build | id9 | id9-inspect | id9-audit (capability scores per call) | id9-review (test pass) | | 5 Hot-Swap | id9-build | id9 | id9-inspect (capability score unchanged) | id9-audit (hot-swap event logged) | โ€” | | 6 Fallback | id9-build | id9 (orders by capability) | id9-inspect | id9-audit (fallback order verified) | โ€” | | 7 Reviewer | id9-build | id9 | id9-inspect (capability inputs valid) | โ€” | id9-review (capability-selected, sees all scores) | | 8 UI | id9-build | id9 | id9-inspect (capability column rendered) | โ€” | โ€” | | 9 Tests | id9-build | id9 | id9-inspect | id9-audit (capability ordering verified) | โ€” | | 10 Migration | id9-build | id9 | id9-inspect | id9-audit | id9-review | `id9` = Mavis (this session), `id9-build`/`inspect`/`audit`/`review` = subordinate agents spawned via `communicate({ spawn: { agent_name: 'id9-build' }})` per worktree-management skill. --- ## M. Agent Tabs Architecture (the audit + integration) ### M.1 The 5 agents โ€” canonical responsibilities, inputs, outputs Per change #2, every Agent Tab has explicit responsibilities, inputs, outputs, and capability-awareness. This is the audit operator asked for. #### M.1.1 Builder - **Responsibility:** writes code, creates files, applies diffs, runs builds - **Inputs:** task spec from Planner, current scaffold template, capability ranking (so it knows which model produced the spec) - **Outputs:** file diffs, build artifacts, deploy receipts, optional "needs Reviewer" flag - **Capability awareness:** scaffolding supports capability-based routing โ€” the model metadata Builder uses has a `capability_score` field - **Hot-swap alignment:** Builder runs on the cascade winner; if hot-swap toggles the winner's tier off mid-build, Builder completes with the old model (in-flight immunity) - **Fallback alignment:** if Builder's primary model fails, it can use the next-highest capability_score from the same phase, not the next-cheapest #### M.1.2 Planner - **Responsibility:** decomposes work into phases, plans dependencies, orders tasks - **Inputs:** operator directive, current state, **capability_scores of all available models** (so it can route each phase to the best fit) - **Outputs:** phase plan, dependency graph, ordered task list with assigned models per phase - **Capability awareness (per change #2):** orders tasks based on capability-based Reviewer selection โ€” phases that need a sharp Reviewer get assigned to models with high capability_score, regardless of cost - **Fallback alignment:** plan includes explicit fallback routes for each phase ("if Phase 4 model fails, use Phase 4b with capability_score X") - **Hot-swap alignment:** when hot-swap toggles a model off, Planner's next plan call re-reads capability_scores and re-orders #### M.1.3 Inspector - **Responsibility:** validates outputs against specs, runs tests, checks shape - **Inputs:** build artifact, spec, **capability scoring inputs (model metadata, all 7 components, range 0-1)** - **Outputs:** pass/fail report, capability_score validation results (e.g. "all 7 components present, all in [0,1], no nulls") - **Capability awareness (per change #2):** validates that capability scoring inputs are sane โ€” every model has all 7 components, scores in valid range, no `null` capability_score for an enabled model - **Hot-swap alignment:** Inspector uses the same model the Builder used (consistency check) โ€” if hot-swap toggles Builder's model off, Inspector still validates the result, but logs the model mismatch - **Fallback alignment:** if Inspector's primary model fails, falls to the next Inspector-capable model (separate from Builder's fallback โ€” Inspector needs precision, not creativity) #### M.1.4 Auditor - **Responsibility:** cross-checks against FvW v8 doctrine, finds regressions, logs decisions - **Inputs:** diff, prior state, **capability scores from all candidates**, Reviewer decisions, hot-swap events - **Outputs:** audit report, regression flags, **capability_score log (per-request snapshot)**, Reviewer decision log - **Capability awareness (per change #2):** logs capability_score for each model used per request, logs Reviewer's selection_rationale ("free model with higher capability_score beat paid lower-score"), logs hot-swap events with capability scores before/after - **Hot-swap alignment:** every hot-swap event appears in the audit trail with before/after capability_scores - **Fallback alignment:** if a fallback was used, the audit trail says "fallback used: (capability_score X vs Y)" #### M.1.5 Reviewer - **Responsibility:** final-pass review, picks best of N candidates, applies correction - **Inputs:** cascade winner, **capability scores for ALL candidates (winner + runners-up, not just winner)**, context - **Outputs:** reviewed text, delta score, decision, selection_rationale - **Capability awareness (per change #1, locked):** Reviewer is selected by highest capability_score. Free models can win. Reviewer fallback follows capability order, not billing order. - **Hot-swap alignment:** if hot-swap toggles a candidate off, Reviewer skips it (in-flight candidates immune, new requests see the updated list) - **Fallback alignment:** if Reviewer's primary fails, Reviewer falls to next-highest capability_score (same selection rule as primary) - **Test agent:** `id9-review` is the test Reviewer for Phase 7's self-loop verification ### M.2 Capability block in every prompt (per change #2) The capability block is injected into: - Phase 1 prompts (so the cascade winner knows its own ranking) - Reviewer prompts (so Reviewer sees all candidates, not just the winner) - Audit prompts (so Auditor logs the ranking context) The block shape (defined in ยงG): ``` ## CAPABILITY RANKING (top 3 by score โ€” free models can win if they score higher) 1. deepseek-ai/deepseek-v4-pro-0813: capability=0.940 (free) 2. nvidia/nemotron-3-nano-omni-30b-a3b-reasoning: capability=0.880 (free) 3. openai/gpt-oss-120b: capability=0.720 (free) Use this as your quality bar. The model that produced you was selected by capability order, not cost. ``` ### M.3 Hot-swap and fallback alignment with Agent Tabs | Agent | Hot-swap behavior | Fallback behavior | |---|---|---| | Builder | In-flight builds immune; new builds use updated config | Next-highest capability_score in same tier | | Planner | Re-orders next plan on hot-swap event | Plan includes explicit fallback model per phase | | Inspector | Validates against the model that was actually used (even if hot-swapped) | Next Inspector-capable model (separate ranking) | | Auditor | Logs hot-swap event with before/after capability_scores | Logs fallback usage with reason | | Reviewer | Skips hot-swapped candidates (in-flight immune) | Next-highest capability_score in Reviewer pool | ### M.4 The 7-component capability scoring formula (per change #1) `capability_score` is the weighted sum of 7 components, each in [0, 1]: ```js // Per-model capability scoring const CAPABILITY_WEIGHTS = { reasoning_depth: 0.20, // can it follow multi-step logic? correction_accuracy: 0.20, // does it self-correct on errors? consistency: 0.15, // same prompt โ†’ similar output? long_form_stability: 0.15, // does it degrade on 4K+ outputs? clarity: 0.10, // are responses well-structured? error_detection: 0.10, // can it find bugs in code? reasoning_content: 0.10, // does it expose reasoning_content? (NVIDIA bonus) }; function computeCapabilityScore(probeData, historyData) { // probeData: { has_reasoning_field, latency_ms, http_status } // historyData: { avg_score_on_test_prompts, error_rate, user_feedback_score } const components = { reasoning_depth: probeData.test_response_quality || 0.5, // 0-1 correction_accuracy: 1 - historyData.error_rate, consistency: historyData.consistency_score || 0.5, long_form_stability: historyData.long_form_score || 0.5, clarity: historyData.clarity_score || 0.5, error_detection: historyData.bug_finding_score || 0.5, reasoning_content: probeData.has_reasoning_field ? 1.0 : 0.0, }; return Object.entries(CAPABILITY_WEIGHTS).reduce( (sum, [k, w]) => sum + (components[k] * w), 0 ); } ``` **Initial values (no history yet):** components default to 0.5 (mid-scale). Models with `reasoning_content` (NVIDIA Nemotron) get a free 0.10 boost. DeepSeek V4 Pro has no `reasoning_content` field but is bigger โ€” its history-derived scores will be higher once it accumulates data. **Live probe (2026-09-08 15:05)** found these models on the operator's free tier: | Model | Probe status | capability_score (initial) | |---|---|---| | `deepseek-ai/deepseek-v4-pro-0813` | โœ… working | 0.83 (bigger model, no reasoning_content bonus) | | `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` | โœ… working | 0.65 (smaller but has reasoning_content) | | `minimaxai/minimax-m3` | โœ… working | 0.55 (unverified provenance) | | `openai/gpt-oss-20b` (Groq) | โœ… working | 0.55 (per existing usage) | | `openai/gpt-oss-120b` (Groq) | โœ… working | 0.60 (per existing usage) | | `deepseek-ai/deepseek-coder-6.7b-instruct` | โŒ 404 (not on account) | โ€” | | `deepseek-ai/deepseek-v4-flash-0731` | โŒ empty (rate-limited) | โ€” | | `nvidia/nemotron-3.5-lightning-30b-a3b` | โŒ timeout (retired) | โ€” | | `nvidia/llama-3.1-nemotron-70b-instruct` | โŒ 404 (not on account) | โ€” | | `mistralai/mistral-large-2-instruct` | โŒ 404 (not on account) | โ€” | | `moonshotai/kimi-k3`, `google/gemma-4-31b-it` | โŒ timeout/empty | โ€” | These initial scores will be re-computed by Phase 0.5 probe-on-boot + updated as ModelUsage history accumulates. ### M.5 Architecture principle: provider-agnostic (per change #3) The architecture is **provider-agnostic**: - `provider-registry.cjs` is the only place that knows about specific provider endpoints - Adding a new model = add one row to the registry (model_id, provider, endpoint, env_var_name, has_reasoning_field) - The cascade reads from `vibechat_provider_probe_results` (PB), not from a hardcoded list - Capability scoring is computed from probe + history, not from a hardcoded value - Reviewer selection is `sort by capability_score desc`, not `sort by provider tier` If NVIDIA retires Nemotron-3-nano tomorrow, the probe-on-boot will mark it `enabled=false` and the cascade won't use it. If DeepSeek V4 Pro gets rate-limited, the probe logs `empty content` and the cascade uses the next-highest capability_score. No code change needed. --- ## I. Proof-of-work plan (per phase) | Phase | PoW artifact | Validation | |---|---|---| | 0 | v1 blueprint file | File exists, sections A-I present | | 0.5 | Probe script + results | `node probe-all-providers.cjs` populates 5+ rows in `vibechat_provider_probe_results` | | 1 | 8 PB collections | `node migrate.js` runs twice, second no-op | | 2 | Config loader | `node -e "require('./config/cascade-config').load()"` returns valid JSON with model_ids from probe | | 3 | scaffoldPrompt tests | All 7 task-types + 7 reflexes + capability block + error path produce valid output, size < 32K | | 4 | Cascade refactor | Default chat behavior unchanged, ModelUsage has 1+ rows per call WITH `capability_score_at_call`, tool synthesis uses winning model | | 5 | Hot-swap | Toggle DeepSeek off โ†’ no deepseek rows in ModelUsage; toggle on โ†’ rows return | | 6 | Fallback | Disable all top-3 capability models โ†’ chat uses next-highest, log says "fallback tier used (capability order)" | | 7 | Reviewer | ReviewerPass row created on `review_enabled=true`, `selection_rationale` says "selected by capability order", no row on false | | 8 | UI | Playwright screenshot shows Edge โ†’ Routing section with capability score column | | 9 | E2E tests | 15/15 pass (was 12/12) | | 10 | Migration | All 8 new tables empty, no existing data touched, default config loaded, first-boot probe populates 5+ rows | --- ## J. Risk register | Risk | Likelihood | Impact | Mitigation | |---|---|---|---| | Migration breaks existing PB data | Low | High | Additive only, no DROP, idempotent, dry-run first | | Hot-swap mid-flight request crashes | Low | High | Hot-swap creates new config object, in-flight requests see old config | | Reviewer pass doubles LLM cost | Medium | Medium | Reviewer is opt-in (config.review_enabled), default off | | Phase 4 cascade refactor breaks chat | Medium | High | Phase 4 runs parallel to default config (Phase 2), with feature flag `BRIDGE_USE_CONFIG=0` to fall back to hardcoded | | Scaffolding prompt too long | Medium | Low | Hard limit 32K tokens, assert in Phase 3 tests | | Free-tier quota exhaustion | Medium | Low | Daily budget tracking, circuit breaker (already in groq_runner_v2.py pattern) | | Vendor Edge Panel changes | Low | Medium | Pattern from r2583 (inject without forking) | | **NEW: capability_score computation is unfair initially (no history)** | Medium | Medium | Initial 0.5 default + reasoning_content bonus; first 50 calls are "learning period" | | **NEW: DeepSeek V4 Pro slower than Nemotron-3-nano** | Low | Low | Fire stagger (1500ms) gives Groq first dibs; quality-weighted picks the deeper one | | **NEW: probe-on-boot fails (provider outage)** | Low | Medium | Falls back to last-known-good probe results; logs the failure; UI shows "stale" badge | --- ## K. Stop point ๐Ÿ”ด **STOPPING for operator approval.** This is v2 of the blueprint. v1 was never executed. v2 adds the 4 operator-requested changes + extra models. **If approved, I will:** 1. Spawn `id9-build` agent for Phase 0.5 (Probe) โ€” runs on bridge boot, populates 8+ rows 2. `id9-inspect` validates probe shape 3. Progress to Phase 1, 2, 3, 4 sequentially (critical path) 4. Parallelize Phases 5/6/7 via `communicate({ spawn: ... })` 5. Phase 8 (UI) requires re-deploy of vibecoder-standalone bundle 6. Phase 9 (Tests) runs against live `vibecoder.freshvibeapps.com` 7. Phase 10 (Migration) is the cutover โ€” existing data preserved, new tables empty, default config loaded, first-boot probe populates rows **Operator, please confirm one of:** - โœ… "approved โ€” execute Phase 0.5 first" - ๐Ÿ”ง "approved with changes: [list]" - โŒ "rejected โ€” [reason]" Or ask for any specific section to be expanded. --- ## L. Cross-references - **Audit:** `/workspace/.mavis/audits/dynamic-prompt-scaffolding-audit-2026-09-08.md` - **Current cascade code:** `/opt/vibecoder-bridge/server.cjs:1591-1706` - **NVIDIA wiring (r2583-ish):** see bulletin b2740, 7 server.cjs edits - **Live probe (2026-09-08 15:05):** in this conversation, 15 models probed, 3 confirmed working - **PB collections endpoint:** `/opt/operator/panel/src/db.js` - **Migration runner:** `/opt/operator/panel/migrate.js` - **Edge Panel injection pattern:** `/workspace/vibecoder-standalone/src/vcs/injectChatIcons.js` (r2583) - **Vibe Agents Notion page:** `3d1fc18c-b7ce-81be-968a-e7297c10d3e4` (will be updated to v2) - **Consolidation Notion page:** `3d4fc18c-b7ce-814e-a431-e1631741c14e` - **v1 blueprint (superseded):** `/workspace/.mavis/plans/vibe-agents-execution-blueprint-2026-09-08.md` --- *End of v2 blueprint. Waiting for operator approval. โ€” id9 (Mavis) using Vibe Agents*