LumabyteCo/clarifyprompt-mcp

Summary

The first eval-gate-driven release. Three real bug fixes — caught by the live CI eval gate against gpt-4o-mini the moment we wired up the OPENAI_API_KEY secret.

Headline fix — #2

Memory store hardcoded its sqlite-vec table to 768 dimensions. Pre-1.5.2, anyone configuring a non-768 embedding model — OpenAI text-embedding-3-small (1536), Voyage voyage-3 (1024), Cohere embed-english-v3.0 (1024), or any other — would hit:

Dimension mismatch for query vector for the "vec" column.
Expected 768 dimensions but received N.

…on the first memory_search or load_knowledge_pack call. Silent failure mode: the engine was effectively broken for anyone not on the default Ollama / nomic-embed-text setup.

Fix: the store now derives the table name from embedder.dimension at construct time and issues an idempotent CREATE VIRTUAL TABLE IF NOT EXISTS embeddings_<dim> for non-768 dims at boot. Existing 768-dim installs unaffected — migration 1 still creates embeddings_768 for legacy back-compat.

Also in this release

• LLM_TIMEOUT_MS env-var override on the LLM client. Default stays 30s; bump it (e.g. LLM_TIMEOUT_MS=120000) for slow hosted models. CI eval workflow uses 120s for gpt-4o-mini.
• Eval harness hardened — no longer crashes when an MCP tool throws an exception. One bad fixture used to tank the entire run; now per-fixture errors are reported cleanly and the run continues.
• evals.yml workflow split out of ci.yml so the evals: passing badge on the README points at a dedicated workflow whose status actually means evals.
• Cross-workflow publish gate — ci.yml#publish waits for evals.yml on the same SHA before publishing. Eval failures still block npm publish on tag pushes.

Discovery story (process note)

The eval gate (live runs against gpt-4o-mini) caught all four issues this release fixes within 30 minutes of OPENAI_API_KEY being added. Three CI iterations, each surfacing exactly one bug:

1. ❌ Run 1 → harness JSON.parse crash → fixed → run 2
2. ❌ Run 2 → dimension mismatch (the headline) → fixed → run 3
3. ❌ Run 3 → brittle fixture content-check → reframed → run 4
4. ✅ Run 4 → 20 passed / 0 failed / 3 skipped / 100%

Total CI cost: ~$0.10 against the configured $5/mo budget cap. That's exactly what the eval gate was built for: real-LLM regressions caught before they ship.

Compatibility

• No env-var surface changes (only optional LLM_TIMEOUT_MS added).
• No MCP tool surface changes — still 20 tools, 1 resource.
• Existing 768-dim installs require no action; their memory.db is fully forward-compat.
• Anyone running with non-768 embeddings should upgrade — pre-1.5.2 their persistent-memory pipeline was silently broken at the first memory_search call.

Install

```bash
npx clarifyprompt-mcp
```

Full diff: https://github.com/LumabyteCo/clarifyprompt-mcp/compare/v1.5.1...v1.5.2