Skip to content

Commit c505b07

Browse files
committed
docs(planning): reconcile meta-gateway invoke QA progress
Mark Phases A–C implementation status in the planning doc and record manual QA results for runbook sections 0–3. Signed-off-by: crimsonsunset <jsangio1@gmail.com>
1 parent 993f378 commit c505b07

2 files changed

Lines changed: 83 additions & 70 deletions

File tree

docs/planning/meta-gateway-invoke-qa.md

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -10,15 +10,15 @@ One-session checklist for validating Phases A–C (search → schema → invoke,
1010

1111
## Quick prep
1212

13-
- [ ] Rebuild/restart gateway if you haven't since the branch (`pnpm dev` or run the built app)
14-
- [ ] Cursor → MCP → **Reload tools**
15-
- [ ] Confirm McpMux endpoint: `http://localhost:45818/mcp`
16-
- [ ] Have at least one OAuth server (GitHub) **installed and connected** but **inactive** in session (for enable-flow tests)
17-
- [ ] Optional for Phase C tests: create a FeatureSet with 1–2 GitHub tools, bind to workspace; leave surfaced off until test 7
13+
- [x] Rebuild/restart gateway if you haven't since the branch (`pnpm dev` or run the built app)
14+
- [x] Cursor → MCP → **Reload tools**
15+
- [x] Confirm McpMux endpoint: `http://localhost:45818/mcp`
16+
- [ ] Have at least one OAuth server (GitHub) **installed and connected** but **inactive** in session (for enable-flow tests) — github is `enabled_via_binding`; use session disable for test 2
17+
- [ ] Optional for Phase C tests: create a FeatureSet with 1–2 GitHub tools, bind to workspace; leave surfaced off until test 8
1818

19-
**Tester:** _______________
20-
**Date:** _______________
21-
**McpMux version / commit:** _______________
19+
**Tester:** Cursor agent (Composer)
20+
**Date:** May 25, 2026
21+
**McpMux version / commit:** `feat/meta-gateway-invoke` @ `993f378`
2222

2323
---
2424

@@ -35,10 +35,10 @@ You have McpMux meta tools only — no direct backend tools like github_*.
3535

3636
| Check | Pass | Fail | Notes |
3737
| ----- | ---- | ---- | ----- |
38-
| `mcpmux_list_servers` returns installed servers | || |
39-
| Only **10** `mcpmux_*` tools exposed (no backend names) | || Expected: bind, create_feature_set, disable/enable_server, get_tool_schema, invoke_tool, list_all_tools, list_feature_sets, list_servers, search_tools |
40-
| Backend servers show **inactive** until enabled | || |
41-
| Tool list count stable (~10 meta + Cursor/plugin tools) | || |
38+
| `mcpmux_list_servers` returns installed servers | || 34 servers returned |
39+
| Only **10** `mcpmux_*` tools exposed (no backend names) | || Verified via MCP descriptor folder |
40+
| Backend servers show **inactive** until enabled | || All inactive at session start |
41+
| Tool list count stable (~10 meta + Cursor/plugin tools) | || No backend tools leaked |
4242

4343
---
4444

@@ -63,11 +63,11 @@ Show each step briefly, then the first 5 issues.
6363

6464
| Check | Pass | Fail | Notes |
6565
| ----- | ---- | ---- | ----- |
66-
| Agent enabled github when inactive ||| |
67-
| Search before invoke (no param guessing) | || |
68-
| Schema read before invoke | || |
69-
| Invoke succeeded with correct param names | || |
70-
| `tools/list` still ~10 meta tools after enable | || |
66+
| Agent enabled github when inactive ||| N/A — github was `enabled_via_binding` |
67+
| Search before invoke (no param guessing) | || Found `github_list_issues` via search |
68+
| Schema read before invoke | || Used `owner`/`repo`/`state`/`perPage` from schema |
69+
| Invoke succeeded with correct param names | || 5 open issues returned for mcpmux/mcp-mux |
70+
| `tools/list` still ~10 meta tools after enable | || Still exactly 10 `mcpmux_*` tools |
7171

7272
---
7373

@@ -86,9 +86,9 @@ Try to invoke a GitHub tool WITHOUT enabling github first (disable it if needed)
8686

8787
| Check | Pass | Fail | Notes |
8888
| ----- | ---- | ---- | ----- |
89-
| Invoke denied when server inactive | || |
90-
| Error mentions `mcpmux_enable_server` with server_id | || |
91-
| Recovery via enable → retry works | || |
89+
| Invoke denied when server inactive | || After `mcpmux_disable_server``disabled_via_session` |
90+
| Error mentions `mcpmux_enable_server` with server_id | || `server 'github' is disabled for this session → mcpmux_enable_server({ "server_id": "github" })` |
91+
| Recovery via enable → retry works | || enable + invoke returned 3 issues |
9292

9393
---
9494

@@ -109,10 +109,10 @@ What did compact strip?
109109

110110
| Check | Pass | Fail | Notes |
111111
| ----- | ---- | ---- | ----- |
112-
| `name` level omits descriptions | || |
113-
| `description` level includes descriptions | || |
114-
| `compact: true` strips descriptions/examples | || |
115-
| Batch schema (array of tools) works if agent tries it | || |
112+
| `name` level omits descriptions | || `github_list_issues` — no `description` key |
113+
| `description` level includes descriptions | || Full tool description present |
114+
| `compact: true` strips descriptions/examples | || Strips **top-level** tool `description`; property descriptions in `input_schema` kept |
115+
| Batch schema (array of tools) works if agent tries it | || `tools: ["github_list_issues"]` returned schemas array |
116116

117117
---
118118

docs/planning/meta-gateway-invoke.md

Lines changed: 59 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# Meta-Gateway Invoke (Search → Schema → Invoke)
22

33
**Last Updated:** May 25, 2026
4-
**Status:** Planning
5-
**Branch:** TBD — branch off `main`
4+
**Status:** Implemented on branch — manual QA in progress ([`meta-gateway-invoke-qa.md`](./meta-gateway-invoke-qa.md))
5+
**Branch:** `feat/meta-gateway-invoke`
66
**Base branch:** `main`
77
**Issue:** TBD — file after planning review
88
**Depends on:** [`dynamic-mcp-toggle-meta-tools.md`](./dynamic-mcp-toggle-meta-tools.md) (session overrides + meta-tool registry); benefits from workspace bindings / FeatureSets from PR #151
@@ -160,71 +160,74 @@ Prompts and resources: unchanged — still materialized per grants. Invoke model
160160

161161
## Files to create
162162

163-
| File | Purpose |
164-
| ---- | ------- |
165-
| `crates/mcpmux-gateway/src/services/tool_discovery.rs` | Index + search + schema lookup over Space tool features |
166-
| `crates/mcpmux-gateway/src/services/meta_tools/invoke.rs` | `InvokeToolTool` impl — permission check, routing, error mapping |
167-
| `tests/rust/tests/integration/meta_gateway_invoke.rs` | Search, schema, invoke, permission deny, surfaced tools, direct backend call rejected |
168-
| `docs/planning/meta-gateway-invoke.md` | This doc |
163+
| File | Purpose | Status |
164+
| ---- | ------- | ------ |
165+
| `crates/mcpmux-gateway/src/services/tool_discovery.rs` | Index + search + schema lookup over Space tool features | ✅ Done |
166+
| `crates/mcpmux-gateway/src/services/meta_tools/invoke.rs` | `InvokeToolTool` impl — permission check, routing, error mapping, result shaping | ✅ Done |
167+
| `tests/rust/tests/integration/meta_gateway_invoke.rs` | Search, schema, invoke, permission deny, surfaced tools, direct backend call rejected | ✅ Done (13 tests) |
168+
| `docs/planning/meta-gateway-invoke-qa.md` | Manual QA runbook for Phases A–C | ✅ Done |
169+
| `docs/planning/meta-gateway-invoke.md` | This doc | ✅ Done |
169170

170171
## Files to modify
171172

172-
| File | Change |
173-
| ---- | ------ |
174-
| [`crates/mcpmux-gateway/src/services/mod.rs`](../../crates/mcpmux-gateway/src/services/mod.rs) | `pub mod tool_discovery;` |
175-
| [`crates/mcpmux-gateway/src/services/meta_tools/tools.rs`](../../crates/mcpmux-gateway/src/services/meta_tools/tools.rs) | `SearchToolsTool`, `GetToolSchemaTool`; extend `ListAllToolsTool` with optional `server_id` filter |
176-
| [`crates/mcpmux-gateway/src/services/meta_tools/mod.rs`](../../crates/mcpmux-gateway/src/services/meta_tools/mod.rs) | Register new tools; wire `ToolDiscoveryService` + `RoutingService` into `MetaToolContext` |
177-
| [`crates/mcpmux-gateway/src/services/meta_tools/registry.rs`](../../crates/mcpmux-gateway/src/services/meta_tools/registry.rs) | Extend `MetaToolContext` with discovery + routing handles |
178-
| [`crates/mcpmux-gateway/src/pool/features/facade.rs`](../../crates/mcpmux-gateway/src/pool/features/facade.rs) | Split `get_tools_for_grants` into advertised (meta + surfaced) vs invokable (full set for search/invoke ACL) |
179-
| [`crates/mcpmux-gateway/src/pool/routing.rs`](../../crates/mcpmux-gateway/src/pool/routing.rs) | Shared invokable-set helper; ensure `call_tool` errors are actionable |
180-
| [`crates/mcpmux-gateway/src/mcp/handler.rs`](../../crates/mcpmux-gateway/src/mcp/handler.rs) | `tools/list` uses advertised set only; direct backend `call_tool` rejected with invoke redirect |
181-
| [`crates/mcpmux-storage/`](../../crates/mcpmux-storage/) or core domain | Optional `surfaced` flag on FeatureSet member rows (Phase C) — if schema change needed |
182-
| [`apps/desktop/src/features/settings/SettingsPage.tsx`](../../apps/desktop/src/features/settings/SettingsPage.tsx) | Update meta-tools copy for search → schema → invoke workflow |
183-
| [`apps/desktop/src/features/feature-sets/`](../../apps/desktop/src/features/feature-sets/) | UI to mark tools as surfaced (Phase C) |
184-
| [`README.md`](../../README.md) | Replace "every tool available right now" agent-facing claim; document search → schema → invoke flow |
173+
| File | Change | Status |
174+
| ---- | ------ | ------ |
175+
| [`crates/mcpmux-gateway/src/services/mod.rs`](../../crates/mcpmux-gateway/src/services/mod.rs) | `pub mod tool_discovery;` | ✅ Done |
176+
| [`crates/mcpmux-gateway/src/services/meta_tools/tools.rs`](../../crates/mcpmux-gateway/src/services/meta_tools/tools.rs) | `SearchToolsTool`, `GetToolSchemaTool`; extend `ListAllToolsTool` with optional `server_id` filter | ✅ Done |
177+
| [`crates/mcpmux-gateway/src/services/meta_tools/mod.rs`](../../crates/mcpmux-gateway/src/services/meta_tools/mod.rs) | Register new tools; wire `ToolDiscoveryService` + `RoutingService` into `MetaToolContext` | ✅ Done |
178+
| [`crates/mcpmux-gateway/src/services/meta_tools/registry.rs`](../../crates/mcpmux-gateway/src/services/meta_tools/registry.rs) | Extend `MetaToolContext` with discovery + routing handles | ✅ Done |
179+
| [`crates/mcpmux-gateway/src/pool/features/facade.rs`](../../crates/mcpmux-gateway/src/pool/features/facade.rs) | Split into `get_advertised_tools_for_grants` vs `get_invokable_tools_for_grants` | ✅ Done |
180+
| [`crates/mcpmux-gateway/src/pool/features/resolution.rs`](../../crates/mcpmux-gateway/src/pool/features/resolution.rs) | `resolve_surfaced_feature_ids` for surfaced promotion | ✅ Done |
181+
| [`crates/mcpmux-gateway/src/pool/routing.rs`](../../crates/mcpmux-gateway/src/pool/routing.rs) | `format_direct_call_redirect`; actionable invoke errors | ✅ Done |
182+
| [`crates/mcpmux-gateway/src/mcp/handler.rs`](../../crates/mcpmux-gateway/src/mcp/handler.rs) | `tools/list` uses advertised set only; direct backend `call_tool` rejected with invoke redirect | ✅ Done |
183+
| [`crates/mcpmux-core/src/domain/feature_set.rs`](../../crates/mcpmux-core/src/domain/feature_set.rs) | `surfaced: bool` on `FeatureSetMember` | ✅ Done |
184+
| [`apps/desktop/src/features/featuresets/FeatureSetPanel.tsx`](../../apps/desktop/src/features/featuresets/FeatureSetPanel.tsx) | Per-tool "Surface in client" toggle | ✅ Done |
185+
| [`apps/desktop/src/features/settings/SettingsPage.tsx`](../../apps/desktop/src/features/settings/SettingsPage.tsx) | Update meta-tools copy for search → schema → invoke workflow | ⬜ Pending |
186+
| [`README.md`](../../README.md) | Replace "every tool available right now" agent-facing claim; document search → schema → invoke flow | ⬜ Pending |
185187

186188
---
187189

188190
## Phasing
189191

190192
### Phase A — Meta invoke core
191193

192-
**Effort:** ~3–4 days
194+
**Effort:** ~3–4 days
195+
**Status:** ✅ Implemented — manual QA sections 0–1 pass; sections 2–4 pending ([`meta-gateway-invoke-qa.md`](./meta-gateway-invoke-qa.md))
193196

194-
- [ ] `ToolDiscoveryService` — build index from Space features; search by query + optional `server_id`; return matches at `detail_level`
195-
- [ ] `mcpmux_search_tools` meta tool — pagination (`limit`, `cursor`), `detail_level` enum
196-
- [ ] `mcpmux_get_tool_schema` — single + batch; `compact` strips descriptions/examples
197-
- [ ] `mcpmux_invoke_tool``{ server_id, tool, args }`; delegates to `RoutingService::call_tool`; fail closed on permission miss
198-
- [ ] `FeatureService` split: **advertised** = meta tools + surfaced only (hard cut — no backend tools in list)
199-
- [ ] Handler rejects direct backend `call_tool` — return actionable error pointing to `mcpmux_invoke_tool`
200-
- [ ] Actionable error mapping: inactive server, unknown tool, permission denied, param validation passthrough from backend
201-
- [ ] Optional `server_id` filter on `mcpmux_list_all_tools`
202-
- [ ] Integration tests: GitHub read path (enable → search → schema → invoke); deny when server inactive; direct `github_*` call rejected
197+
- [x] `ToolDiscoveryService` — build index from Space features; search by query + optional `server_id`; return matches at `detail_level`
198+
- [x] `mcpmux_search_tools` meta tool — pagination (`limit`, `cursor`), `detail_level` enum
199+
- [x] `mcpmux_get_tool_schema` — single + batch; `compact` strips descriptions/examples
200+
- [x] `mcpmux_invoke_tool``{ server_id, tool, args }`; delegates to `RoutingService::call_tool`; fail closed on permission miss
201+
- [x] `FeatureService` split: **advertised** = meta tools + surfaced only (hard cut — no backend tools in list)
202+
- [x] Handler rejects direct backend `call_tool` — return actionable error pointing to `mcpmux_invoke_tool`
203+
- [x] Actionable error mapping: inactive server, unknown tool, permission denied, param validation passthrough from backend
204+
- [x] Optional `server_id` filter on `mcpmux_list_all_tools`
205+
- [x] Integration tests: GitHub read path (enable → search → schema → invoke); deny when server inactive; direct `github_*` call rejected
203206

204-
**Outcome:** Cursor session shows ~12 `mcpmux_*` tools. Agent completes `github_list_issues` on `mcpmux/mcp-mux` in 4 meta calls + invoke with zero param guessing. Token footprint for tool definitions drops from ~30–50k to ~1–2k.
207+
**Outcome:** Cursor session shows **10** `mcpmux_*` tools (verified May 25, 2026). Agent completes `github_list_issues` on `mcpmux/mcp-mux` via search → schema → invoke with zero param guessing.
205208

206209
### Phase B — Result shaping on invoke
207210

208-
**Effort:** ~2 days
211+
**Effort:** ~2 days
212+
**Status:** ✅ Implemented — manual QA sections 5–6 pending
209213

210-
- [ ] Extend `mcpmux_invoke_tool` args with optional `filter: { max_rows?, max_bytes?, fields?, format? }`
211-
- [ ] Post-process JSON/text results in gateway before returning to client
212-
- [ ] Default smart truncation for known-heavy patterns (large arrays) when filter omitted
213-
- [ ] Integration tests: truncated list response includes `{ returned, total, truncated: true }`
214+
- [x] Extend `mcpmux_invoke_tool` args with optional `filter: { max_rows?, max_bytes?, fields?, format? }`
215+
- [x] Post-process JSON/text results in gateway before returning to client
216+
- [x] Default smart truncation for known-heavy patterns (large arrays) when filter omitted (`DEFAULT_MAX_ROWS` = 50)
217+
- [x] Integration tests: truncated list response includes `{ returned, total, truncated: true }`
214218

215219
**Outcome:** Posthog/Firebase/GWorkspace list calls return bounded payloads. Agent can query large backends without blowing context on results.
216220

217221
### Phase C — FeatureSet as invoke ACL + surfaced tools
218222

219-
**Effort:** ~3 days
220-
221-
- [ ] FeatureSet member model: tools invokable by default when server in set; optional `surfaced: true` promotes into `tools/list`
222-
- [ ] Search + invoke respect FeatureSet member filter (not just server-all)
223-
- [ ] Workspaces UI: per-tool "Surface in client" toggle in FeatureSet editor
224-
- [ ] Update `mcpmux_create_feature_set` to accept optional `surfaced_tools[]`
225-
- [ ] Integration tests: binding with partial tool set → search only finds allowed tools; surfaced tool appears in `tools/list`
223+
**Effort:** ~3 days
224+
**Status:** ✅ Implemented — manual QA sections 8–9 pending
226225

227-
**Outcome:** Bundles ship with zero surfaced tools by default. Users may opt in per FeatureSet. Invoke ACL semantics are unambiguous: **permission + optional promotion**.
226+
- [x] FeatureSet member model: tools invokable by default when server in set; optional `surfaced: true` promotes into `tools/list`
227+
- [x] Search + invoke respect FeatureSet member filter (not just server-all)
228+
- [x] Workspaces UI: per-tool "Surface in client" toggle in FeatureSet editor (`FeatureSetPanel.tsx`)
229+
- [ ] Update `mcpmux_create_feature_set` to accept optional `surfaced_tools[]` (UI path done; meta-tool arg deferred)
230+
- [x] Integration tests: binding with partial tool set → search only finds allowed tools; surfaced tool appears in `tools/list`
228231

229232
### Phase D — Advanced optimizations (defer)
230233

@@ -307,6 +310,16 @@ Prompts and resources: unchanged — still materialized per grants. Invoke model
307310

308311
## Reconciliation
309312

310-
This doc is the source of truth for the meta-gateway invoke model. When implementation starts, update **Status** and **Branch** at the top. Mark [`tool-level-session-pin.md`](./tool-level-session-pin.md) **Status** as *Superseded* once Phase A ships.
313+
This doc is the source of truth for the meta-gateway invoke model. Phases A–C are implemented on `feat/meta-gateway-invoke`; manual QA tracked in [`meta-gateway-invoke-qa.md`](./meta-gateway-invoke-qa.md). Mark [`tool-level-session-pin.md`](./tool-level-session-pin.md) **Status** as *Superseded* once Phase A ships to main.
311314

312315
**Decision record (May 25, 2026):** Hard cut to invoke-only — no legacy direct backend exposure. Surfaced tools default zero everywhere (bundles included). FeatureSets redefine as invoke ACL + optional surfaced promotion. Session pin deferred to Phase F (very optional, last). Competitor analysis (MikkoParkkola + abdullah1854) informed Phase A–B scope; REST capabilities in Phase E / separate doc.
316+
317+
**Manual QA progress (May 25, 2026):**
318+
319+
| QA section | Result | Notes |
320+
| ---------- | ------ | ----- |
321+
| 0 — Sanity (meta-only surface) | ✅ Pass | 10 `mcpmux_*` tools; 34 servers listed; all inactive until enabled |
322+
| 1 — Happy path (GitHub read) | ✅ Pass | search → schema → invoke returned 5 open issues; enable step N/A (`enabled_via_binding`) |
323+
| 2 — Fail-closed + recovery | ✅ Pass | Session disable → actionable error → enable → retry |
324+
| 3 — Search detail levels + compact schema | ✅ Pass | compact omits top-level description only |
325+
| 4–11 | ⬜ Pending | In progress |

0 commit comments

Comments
 (0)