blackwell-systems/agent-lsp

v0.13.0: 30-51% fewer tokens on every tool response

agent-lsp now ships with GCF (Graph Compact Format) as the default output encoding. Every tool response uses 30-51% fewer tokens than JSON, with zero configuration.

Most MCP servers return raw JSON. agent-lsp now returns 30-51% fewer tokens out of the box.

How it works

GCF replaces JSON's per-record field name repetition with a single header declaration and positional pipe-delimited rows:

JSON (334 tokens):

[
  {"name": "AuthMiddleware", "kind": "Function", "detail": "func(http.Handler) http.Handler", "file": "middleware.go", "start_line": 45, "end_line": 78},
  {"name": "ValidateToken", "kind": "Function", "detail": "func(string) (*Claims, error)", "file": "token.go", "start_line": 23, "end_line": 56},
  {"name": "Claims", "kind": "Struct", "detail": "struct", "file": "token.go", "start_line": 10, "end_line": 18}
]

GCF (165 tokens):

## [3]{detail,end_line,file,kind,name,start_line}
func(http.Handler) http.Handler|78|middleware.go|Function|AuthMiddleware|45
func(string) (*Claims, error)|56|token.go|Function|ValidateToken|23
struct|18|token.go|Struct|Claims|10

Same data. 50.6% fewer tokens. Savings grow with record count.

Measured savings on real tool responses

| Tool | JSON | GCF | Savings |
|------|------|-----|---------|
| list_symbols (10 symbols) | ~334 tokens | ~165 tokens | 50.6% |
| find_references (50 locations) | ~858 tokens | ~437 tokens | 49.1% |
| get_diagnostics (5 diagnostics) | ~213 tokens | ~133 tokens | 37.6% |
| blast_radius (5 symbols, 6 callers) | ~526 tokens | ~365 tokens | 30.6% |

Benchmark: go run scripts/gcf-benchmark.go

What this means for you

Every tool call returns more information per token budget. On a typical code exploration session (blast_radius, explore_symbol, find_callers, inspect), the cumulative savings compound across calls. Your agent sees more context, makes better decisions, and costs less.

Configuration

GCF is enabled by default. To revert to JSON:

export AGENT_LSP_OUTPUT_FORMAT=json

Why GCF over JSON or TOON?

JSON repeats field names on every record. TOON improves on JSON but still carries per-record overhead. We tested GCF against both on TOON's own benchmark, using TOON's own datasets and tokenizer:

| Track | GCF | TOON | JSON |
|-------|-----|------|------|
| Mixed-structure (nested, semi-uniform) | 169,554 | 227,896 (+34%) | 291,620 (+72%) |
| Semi-uniform event logs | 107,269 | 154,032 (+44%) | N/A |
| Flat tabular (TOON's home turf) | 66,026 | 67,837 (+3%) | 198,560 |

GCF wins all three tracks. The gap on structured data (34-44%) is not marginal; it's architectural. TOON is a general-purpose format that repeats field names and type annotations per record. GCF uses positional encoding, abbreviated kinds, local ID references, and distance-tier grouping to eliminate per-record overhead that TOON cannot.

On real code intelligence payloads (500 symbols), GCF uses 32% fewer tokens than TOON at identical 100% comprehension accuracy. JSON drops to 66.7% accuracy at this scale.

What is GCF?

GCF (Graph Compact Format) is a token-optimized wire format for LLM tool responses.

• Spec
• Go
• Python
• TypeScript
• Playground

Other changes

Added:

• AGENT_LSP_OUTPUT_FORMAT environment variable for format control
• Enum constraints on direction, detail_level, and level parameters (improves schema quality for MCP clients)
• Standalone mcp-assert dispatch workflow for independent CI testing
• New documentation:
  • MCP Client Configs: copy-paste setup for Claude Code, Cursor, Windsurf, Continue.dev
  • Troubleshooting: common issues and fixes
  • Common Workflows: "I want to..." mapped to tools and skills
  • Environment Variables: all AGENT_LSP_* variables

Changed:

• Documentation restructured into getting-started/, guide/, reference/, architecture/ for clearer navigation

Dependencies:

• Added github.com/blackwell-systems/gcf-go v0.1.1 (zero transitive dependencies)