Rubric

Rubric 0.2 repositions around its core strength: testing agent behavior — what your agent did, not just what it said.

✨ New

• LangGraph / LangChain auto-capture — run_langgraph(agent, scenarios) runs your agent and extracts tool calls, arguments, outputs, errors, the reasoning trace, latency, and token usage automatically. from_langgraph(result) converts an existing invoke() result; from_messages() works on any OpenAI-format message list, no langchain install required.
• Regression diffing — rubric compare current.json --baseline baseline.json classifies pass→fail regressions, score drops, fixes, and new/removed tests, with failing-metric reasons inline. --fail-on-regression for CI.
• GitHub Action — runs your evals on every PR and posts the regression diff as a comment:
  uses: Kareem-Rashed/[email protected]

Also in this release (previously unreleased)

• HallucinationScore metric (LLM judge + NLI modes)
• capture() / @track for zero-friction call recording
• LangFuse & LangSmith trace importers
• Per-test metrics and LLM-judge flakiness detection

pip install rubric-eval — zero required dependencies, fully local, MIT.
Try the no-API-key demo: python examples/langgraph_eval.py