shinpr/mcp-local-rag releases

Patch release improving read_chunk_neighbors input validation.

Fixes

• read_chunk_neighbors now treats an empty or whitespace-only filePath/source as not provided. Passing source: "" alongside a valid filePath returns the document window instead of resolving to an empty raw-data path and returning nothing.
• The filePath/source validation error now matches the actual situation: "Provide either filePath or source, not both" when both are given, "Either filePath or source must be provided" when neither is.

Maintenance

• Upgraded dev dependencies (@biomejs/biome, @types/node, knip, lint-staged, dpdm) to their latest non-major releases.

Contributors

• Thanks to @dburner for the initial work on empty filePath/source handling.

Patch release for dependency upgrades.

Dependencies

• @huggingface/transformers ^4.0.0 → ^4.2.0
  • Minor version bump in the embedding library. Embedding output may differ slightly from v0.13.0; re-ingest existing documents if you observe behavior changes.
• @lancedb/lancedb ^0.26.2 → ^0.27.2
• @modelcontextprotocol/sdk ^1.28.0 → ^1.29.0
• jsdom ^27.4.0 → ^29.1.1
• turndown 7.2.2 → 7.2.4

Transitive

• protobufjs 7.5.4 → 7.5.5 (#108)
• @xmldom/xmldom 0.8.11 → 0.8.13 (#109)

What's New

New tool: read_chunk_neighbors — Expand a search result by reading the chunks immediately before and after it in the same document.

After finding a relevant chunk via query_documents, pass its chunkIndex and filePath (or source) to retrieve surrounding context in a single call. Useful when a hit answers a question only partially.

MCP Tool

read_chunk_neighbors({ filePath: "/path/to/doc.md", chunkIndex: 5 })

CLI

npx mcp-local-rag read-neighbors --file-path /path/to/doc.md --chunk-index 5

Key details

• Asymmetric window: --before and --after control each direction independently (default 2, max 50)
• Target marking: The requested chunk is included with isTarget: true
• Lenient boundaries: Out-of-range indices return only existing chunks (no error)
• Dual input: Accepts filePath (from ingest_file) or source (from ingest_data)
• Skills docs updated: Agent Skills now include context-expansion guidance

Added

• CHUNK_MIN_LENGTH environment variable and --chunk-min-length CLI flag to configure minimum chunk length in characters (range: 1–10,000, default: 50)
• CHUNK_MIN_LENGTH entry in MCP server manifest (server.json)

Changed

• Minimum chunk length default (50) is now exported as a single source of truth from the chunker module
• Error message for zero-chunk ingestion now reflects the actual configured value instead of hardcoded "50"
• Warning and error messages across all env var parsers and CLI flags now truncate user input to 100 characters

Security

• Upgrade @huggingface/transformers from v3 to v4 (#97)
  • Resolves 3 high-severity tar CVEs (GHSA-34x7, GHSA-8qq5, GHSA-83g3)
• Replace madge with dpdm for circular dependency checking (#97)
  • Resolves minimatch ReDoS vulnerabilities

Internal

• Add pnpm-workspace.yaml with allowBuilds for pnpm v10 build script security

Breaking changes

• Minimum Node.js version raised from 20 to 22 (#96)
  • Node.js 20 reaches EOL on April 30, 2026

Internal

• TypeScript target updated from ES2020 to ES2023

Bug fixes

• Reject non-numeric values for --max-file-size and --limit CLI flags (#92)

Security

• Update @modelcontextprotocol/sdk 1.27.1 → 1.28.0
• Update mammoth 1.11.0 → 1.12.0
• Update vitest 4.0.18 → 4.1.2 (resolves rollup vulnerability)
• Update knip 5.88.0 → 6.1.0 (resolves picomatch vulnerability)

Full CLI support — all MCP tools now available as CLI commands

mcp-local-rag can now be used entirely without an MCP server. All six MCP tools have CLI equivalents:

npx mcp-local-rag ingest ./docs/           # Bulk ingest files
npx mcp-local-rag query "search text"      # Search documents
npx mcp-local-rag list                     # Show ingestion status
npx mcp-local-rag status                   # Database stats
npx mcp-local-rag delete ./docs/old.pdf    # Remove by file path
npx mcp-local-rag delete --source "https://..."  # Remove by source URL

query, list, status, and delete output JSON to stdout for piping (| jq).

CLI architecture

• Global options (--db-path, --cache-dir, --model-name) are now placed before the subcommand and shared across all commands
• Input validation for paths (rejects sensitive system directories), model names (charset + traversal prevention), and file size limits
• Depth-limited directory traversal with BFS for safe recursive scanning

Breaking change

Global options must now appear before the subcommand:

# Before (v0.9.0)
npx mcp-local-rag ingest --db-path ./db ./docs/

# After (v0.10.0)
npx mcp-local-rag --db-path ./db ingest ./docs/

Bug fixes

• Fix #79: extra positional arguments now produce an error instead of being silently ignored
• Fix vulnerable transitive dependencies (hono, express-rate-limit)

Internal improvements

• Test quality: split monolithic integration test (1487 lines) into 7 focused files, removed tautology/placeholder tests, added proper rollback coverage
• Layer cleanup: moved raw-data-utils.ts from server/ to utils/ to eliminate cli→server layer violation
• Shared helpers: createVectorStore/createEmbedder factory functions in cli/common.ts
• Tooling: replaced ts-prune with knip for unused export detection

Skills & documentation

• Skill description rewritten per Agent Skills best practices (intent-centered, cross-LLM compatible)
• Capability-based expressions (HTTP fetch, browser/web tool) instead of implementation-specific terms
• Score interpretation table expanded with 0.5–0.7 band
• CLI reference updated to cover all commands

CLI ingest subcommand

New CLI command for ingesting multiple files or entire directories — no MCP timeout constraints, significantly faster for bulk operations.

npx mcp-local-rag ingest [options] <path>

• Single file or recursive directory ingestion
• Optimizes the database once at the end, not per-file
• All config available as CLI flags (--db-path, --base-dir, --cache-dir, --model-name, --max-file-size)
• Priority: CLI flags > environment variables > defaults
• Skip-and-continue on errors with summary and exit codes
• --help for all options and defaults

AI assistants with shell access (Claude Code, Cursor, Codex) can use this directly for batch ingestion instead of calling ingest_file repeatedly.

Other improvements

• Fix Transformers.js dtype warning on model load

Reduce optimize() overhead per ingest from 2x to 1x

• Move table.optimize() out of insertChunks()/deleteChunks() and into the server layer
• Each ingest_file and delete_file now calls optimize() once instead of twice
• Expose VectorStore.optimize() as a public method for future bulk ingestion support

Fix stale database reads after external modifications

• Set readConsistencyInterval: 0 on LanceDB connect() so every read checks for updates from other processes
• Previously, queries failed with "Failed to search vectors" when another process (e.g., CLI ingestion from a different terminal) modified the database
• Overhead is negligible: ~0.03ms per query, constant regardless of data size

What's New

PDF Parser Migration to MuPDF

• Replaced pdfjs-dist with mupdf (WASM-based) for more accurate PDF text extraction
• Improved multi-line PDF title extraction using font metadata
• Added 2-stage header/footer detection (block-attribute pre-filter + embedding similarity)
• Normalized tab characters in extracted text
• Migrated project from CommonJS to ESM

Configuration Warnings via MCP Annotations

• Invalid environment variable values (e.g., RAG_MAX_FILES=0) are now surfaced to users through MCP response annotations instead of being silently ignored
• Warnings appear in status responses (always) and query_documents responses (first call)
• Fixed parseMaxDistance accepting Infinity as a valid input

Breaking Changes

• list_files response shape changed — The flat array response is replaced with a structured object containing baseDir, files, and sources fields. Each file entry now includes an ingested boolean indicating whether it has been indexed. (#59)

New Features

• BASE_DIR filesystem scan — list_files now scans BASE_DIR recursively for supported files (PDF, DOCX, TXT, MD) and shows their ingestion status, making it easy to see which files are available to ingest. (#59)
• System path exclusion — Internal directories (dbPath, cacheDir) are automatically excluded from the list_files scan so that raw-data files and model cache don't appear as user documents. (#60)

Contributors

• @jarrah31 (#59)

What's New

File-based search filtering (RAG_MAX_FILES)

Added RAG_MAX_FILES environment variable to limit search results to chunks from the top N best-scoring files. Useful for reducing noise when querying large document collections.

Document title extraction (fileTitle)

Search results now include a fileTitle field extracted from each document, providing LLMs with document-level context for each chunk.

Title extraction supports all formats:

• PDF: metadata /Title or semantic chunking of page 1
• HTML: Readability-extracted title
• Markdown: YAML frontmatter title or first H1 heading
• DOCX: first <h1> from converted HTML
• TXT: first line (when followed by blank line)

All formats fall back to a humanized file name when no title is detected.

.meta.json sidecar files

ingest_data now stores metadata (title, source, format) in a separate .meta.json file alongside the raw-data .md file, eliminating title duplication in chunk text.

LanceDB schema migration

The fileTitle column is automatically added to existing databases on startup. No manual migration required. Existing data continues to work with fileTitle: null.

What's Changed

Security

• Defend validateFilePath against symlink traversal attacks using realpath() (#52)

Bug Fixes

• Improve type safety of formatErrorMessage for non-Error arguments (#51)

Refactoring

• Split server/index.ts into focused modules: tool definitions, type definitions, error utilities (#48)
• Remove unnecessary exports from internal modules (#50)
• Separate CLI and MCP server entry points (#43)

Documentation

• Add CONTRIBUTING.md with setup guide, quality checks, PR requirements, and review standards (#53)

Full Changelog: https://github.com/shinpr/mcp-local-rag/compare/v0.5.5...v0.5.6

Dependencies

• Bump @modelcontextprotocol/sdk from 1.25.2 to 1.26.0

What's Changed

Bug Fixes

• Prevent data loss on re-ingest: When re-ingesting a file that produces 0 chunks (empty or unextractable content), existing data is now preserved instead of being deleted
  • Added fail-fast check before delete operation
  • Validates chunk count before any destructive action

Security Improvements

• Proper error codes: Use -32602 (InvalidParams) instead of -32603 (InternalError) for user input errors (empty files)
• Hide stack traces by default: Stack traces only shown when NODE_ENV=development (secure by default for distributed MCP servers)
• Clearer error messages: Explicitly state that existing data has been preserved on failure

Example

Before:

MCP error -32603: Failed to ingest file: Error: No chunks generated...
    at RAGServer.handleIngestFile (...)

After:

MCP error -32602: No chunks generated from file: /path/to/file.md. The file may be empty or all content was filtered (minimum 50 characters required). Existing data has been preserved.

Security

• Update @modelcontextprotocol/sdk to 1.25.2 to fix ReDoS vulnerability in UriTemplate regex patterns

Changes

• CLI Consolidation: Merged mcp-local-rag-skills into main CLI as a subcommand
  • Old: npx mcp-local-rag-skills --claude-code
  • New: npx mcp-local-rag skills install --claude-code

Migration

Update your commands:

# Claude Code (project-level)
npx mcp-local-rag skills install --claude-code

# Claude Code (user-level)
npx mcp-local-rag skills install --claude-code --global

# Codex
npx mcp-local-rag skills install --codex

Bug Fixes

Fix jsdom missing from production dependencies (#31)

jsdom was incorrectly placed in devDependencies instead of dependencies, causing MODULE_NOT_FOUND errors when installing via npx -y mcp-local-rag.

Before (broken):

npx -y [email protected]
# Error: Cannot find module 'jsdom'

After (fixed):

npx -y [email protected]
# Works correctly

CI Improvements

Add production dependency validation

Added report-missing-dependencies to CI pipeline to automatically detect when devDependencies are incorrectly used in production code. This prevents similar issues in future releases.

New Features

ingest_data Tool

Ingest raw content (HTML, text, markdown) directly without requiring a file path. Perfect for web pages fetched via LLM or browser tools.

HTML Parser

Clean content extraction using Mozilla Readability + Turndown. Removes navigation, ads, and boilerplate while preserving document structure.

Agent Skills

RAG optimization skills for Claude Code, Codex, and other AI assistants supporting agentskills.io.

• Query optimization guidance
• Result refinement patterns
• HTML ingestion workflow

Install with: npx mcp-local-rag-skills

Improvements

• Source URL preserved and shown in search results
• list_files now displays source info for ingested web content
• delete_file supports deletion by source identifier
• Raw data stored in {DB_PATH}/raw-data/ for re-processing