Skip to content

evidently

Monitoring & Metrics

An open-source Python library to evaluate, test, and monitor ML and LLM systems—from experiments to production.

Jupyter Notebook Latest v0.7.21 · 2mo ago Security brief →

Features

  • Works with tabular and text data
  • Supports evals for predictive and generative tasks (classification to RAG)
  • Provides 100+ built‑in metrics (data drift detection, LLM judges, etc.)
  • Python interface for custom metric development
  • Both offline evaluations and live monitoring

Recent releases

View all 3 releases →
v0.7.21 Breaking risk
Breaking changes
  • Python 3.9 support dropped
Notable features
  • Logo display in API documentation generation
  • Pandas 3 support
  • Embeddings drift metric in new API
Full changelog

What's Changed

  • feat: enable logo display in API documentation generation by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1793
  • fix: Add pagination to reports table to fix UI performance by @jasperbruin in https://github.com/evidentlyai/evidently/pull/1797
  • UI: snapshots pagination bit changes by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1804
  • Pandas 3 support by @Liraim in https://github.com/evidentlyai/evidently/pull/1807
  • CI: deploy artifacts better by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1812
  • fix: correct output prefix syntax in GitHub Actions workflow by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1814
  • ci: prevent deployment on cancelled workflow runs in GitHub Actions by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1815
  • fix: correct HTML report path in GitHub Actions workflow by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1817
  • CI: explicit download artifacts by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1822
  • Drop python 3.9 support by @mike0sv in https://github.com/evidentlyai/evidently/pull/1811
  • CI: remove redundant test by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1821
  • Enhance README with API Reference and documentation update by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1819
  • Update API reference documentation in README.md for clarity and structure by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1823
  • Implement Embeddings drift metric in new API by @Liraim in https://github.com/evidentlyai/evidently/pull/1828
  • Support nbinsx for ValueDrift metric. by @Liraim in https://github.com/evidentlyai/evidently/pull/1826
  • Fail early on empty input data in Report.run by @Si-ra-kri in https://github.com/evidentlyai/evidently/pull/1820
  • Add methods to handle near-zero values for MAPE metric. by @Liraim in https://github.com/evidentlyai/evidently/pull/1835
  • Fix MAPE warnings by @Liraim in https://github.com/evidentlyai/evidently/pull/1837
  • Examples Cleanup by @emeli-dral in https://github.com/evidentlyai/evidently/pull/1836
  • Fix CategoryCount with multiple labels if some labels are missing by @Liraim in https://github.com/evidentlyai/evidently/pull/1838

New Contributors

  • @Si-ra-kri made their first contribution in https://github.com/evidentlyai/evidently/pull/1820

Full Changelog: https://github.com/evidentlyai/evidently/compare/v0.7.20...v0.7.21

v0.7.19 Mixed
Security fixes
  • Write API routes for artifact and prompt operations protected with guard
  • Tracing API protected with guard
Notable features
  • New API for human feedback in tracing
  • API reference documentation now works for forks
  • Go to page link added to EvidentlyUIRunner show_service view
Full changelog

What's Changed

  • CI: fix API reference generation by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1778
  • Fix: protect write API routers with guard for artifact and prompt routes by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1779
  • move to UV by @mike0sv in https://github.com/evidentlyai/evidently/pull/1759
  • Feature/more api ref by @mike0sv in https://github.com/evidentlyai/evidently/pull/1780
  • feat: fetch-depth: 0 for checking out gh-pages by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1782
  • UI: (EvidentlyUIRunner) add Go to page link in show_service by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1787
  • Fix: protect tracing api with guard by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1788
  • all other docstrings by @mike0sv in https://github.com/evidentlyai/evidently/pull/1783
  • API for human feedback in tracing by @Liraim in https://github.com/evidentlyai/evidently/pull/1784
  • feat: api-reference works for forks by @DimaAmega in https://github.com/evidentlyai/evidently/pull/1789

Full Changelog: https://github.com/evidentlyai/evidently/compare/v0.7.18...v0.7.19

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

About

Stars
7,558
Forks
855
Languages
Jupyter Notebook Python TypeScript

Install & Platforms

Install via
pip

Community & Support

Beta — feedback welcome: [email protected]