ndjordjevic/pinrag

v0.4.0 Breaking

This release includes breaking changes for platform teams planning a safe upgrade.

Published 2mo MCP Developer Tools

✓ No known CVEs patched

✓ No known CVEs patched in this version

Topics

chromadb cursor discord github-repos langchain mcp

+8 more

mcp-server model-context-protocol pdf pypi python llm vscode youtube

Summary

AI summary

Introduced an evaluation framework with LangSmith metrics and optional Cohere reranking.

Full changelog

Evaluation framework, Cohere reranking, config defaults, and docs cleanup since v0.3.0.

Defaults — Retrieve k=20; rerank (when enabled) k=20→top_n=10; ORACLE_RAG_USE_RERANK default false; LLM default anthropic.
Evaluator — Optional Anthropic judge; context-heavy evaluators use a smaller model to avoid rate limits.

Framework — LangSmith evaluators (correctness, relevance, groundedness, retrieval_relevance); run_evaluation.py with dataset/prefix/limit.
Target — Evaluation target uses config for k and rerank; golden and hard-10 datasets.
Scripts — min_k_per_question.py to find minimum k per question (dataset arg); create_eval_dataset EHB question and polished Q&As.

Cohere reranking — Optional ContextualCompressionRetriever with Cohere Rerank; config via ORACLE_RAG_USE_RERANK, ORACLE_RAG_RERANK_RETRIEVE_K, ORACLE_RAG_RERANK_TOP_N.

README — MCP section: when .env is used vs mcp.json env; config table and load order aligned with code.
Notes — evaluation-strategy §8 simplified; implementation-checklist rerank bullet updated; oracle-rag-golden-failing references removed.

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Track ndjordjevic/pinrag

Get notified when new releases ship.

About ndjordjevic/pinrag

RAG for PDFs, YouTube, GitHub repos, Discord exports; index documents and query with citations.