hidai25/eval-view

v0.2.3 Feature

This release adds 5 notable features for engineering teams evaluating rollout.

Published 4mo Developer Productivity

View tool

✓ No known CVEs patched

Read the diff → Tool health → What is this tool? →

✓ No known CVEs patched in this version

Topics

agent-benchmark agent-evaluation agentic-ai ai-agents anthropic autogen

+12 more

cli crewai evaluation langchain-agent langgraph llm mcp openai-assistants pytest python regression-testing testing

Summary

AI summary

Sequence evaluation now supports partial credit scoring.

Full changelog

What's New

CLI Statistical Mode Flags

--runs N flag: Run each test N times for statistical evaluation (pass@k metrics)
--pass-rate flag: Set required pass rate for --runs mode (default: 0.8)
--difficulty filter: Filter tests by difficulty level

Difficulty Levels for Test Cases

New difficulty field: trivial, easy, medium, hard, expert
Console reporter shows difficulty column and breakdown

Partial Credit for Sequence Evaluation

Sequence scoring now uses partial credit instead of binary pass/fail
Example: 3/5 expected steps completed = 60% score (not 0%)

Fixed

--runs CLI flag now properly implemented (was documented but missing)

View diff on GitHub

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Share on X Share on Bluesky

Track hidai25/eval-view

Get notified when new releases ship.

About hidai25/eval-view

Regression testing framework for AI agents. Save golden baselines, detect behavioral drift, and block regressions in CI. Works with LangGraph, CrewAI, OpenAI, Claude, and any HTTP API.

All releases →