DataEval/dingo

MCP Data & Storage

AI‑powered platform for evaluating and improving the quality of machine learning datasets, LLM training data, RAG systems, and production AI applications

Track releases GitHub Website

Python Latest v2.4.0 · 5d ago Security brief →

Features

Comprehensive evaluation of ML datasets, LLM training data, and Retrieval‑Augmented Generation (RAG) systems
Multi‑source data integration: local files, SQL databases, HuggingFace datasets, S3 storage
Hybrid assessment combining fast heuristic rules with LLM‑based deep analysis
Scalable execution from local runs to Spark for billion‑scale datasets
Rich reporting with GUI visualizations and field‑level insights

Recent releases

View all 7 releases →

No immediate action

v2.4.0 Mixed 5d

Clawhub badge + LLMHtmlExtractCompareV3 + llm metrics

Open

No immediate action

v2.3.0 Mixed 5d

clawhub badge + LLM metrics + custom rule

Open

v2.2.2 Breaking risk 2mo

Breaking changes

Removed `parameters` attribute from EvaluatorLLMArgs, enabling extension of the argument structure.

Notable features

Added clawhub skill badge
Added overall timeout to ArticleFactChecker and arXiv package (llm)
Added table and equation quality‑check prompts

Full changelog

What's Changed

feat: add clawhub skill badge by @e06084 in https://github.com/MigoXLab/dingo/pull/374
feat(llm): add overall timeout to ArticleFactChecker & add arXiv package by @seancoding-day in https://github.com/MigoXLab/dingo/pull/375
fix: LLMHtmlExtractCompareV2 metric by @e06084 in https://github.com/MigoXLab/dingo/pull/376
feat: table、equation质检prompt by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/377
docs: update wechat by @e06084 in https://github.com/MigoXLab/dingo/pull/378
feat: BaseTextQuality根据score来判断 by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/379
metric: update LLMTextQualityV5 by @e06084 in https://github.com/MigoXLab/dingo/pull/380
feat: EvaluatorLLMArgs删除parameters属性，允许扩展 by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/381
docs: update wechat (#378) by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/382
feat: v2.2.2 by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/383

Full Changelog: https://github.com/MigoXLab/dingo/compare/v2.2.1...v2.2.2

View release on GitHub

v2.2.1 Breaking risk 2mo

Breaking changes

Removed the dingo app demo code

Notable features

Refactored CLI to Agent-Native architecture
Added MCP server support to dingo CLI
Added article fact‑checking skills for Claude and OpenClaw

Full changelog

What's Changed

feat: optimize requirements build by @e06084 in https://github.com/MigoXLab/dingo/pull/359
docs: add v2.1.0 poster by @e06084 in https://github.com/MigoXLab/dingo/pull/361
docs: add zhihu post by @e06084 in https://github.com/MigoXLab/dingo/pull/362
skills: add clawhub skill by @e06084 in https://github.com/MigoXLab/dingo/pull/364
skills: update clawhub by @e06084 in https://github.com/MigoXLab/dingo/pull/365
feat: refactor CLI to Agent-Native by @e06084 in https://github.com/MigoXLab/dingo/pull/366
feat: dingo CLI add MCP server by @e06084 in https://github.com/MigoXLab/dingo/pull/367
docs: update readme by @e06084 in https://github.com/MigoXLab/dingo/pull/368
docs: update readme by @e06084 in https://github.com/MigoXLab/dingo/pull/369
feat: delete dingo app demo code by @e06084 in https://github.com/MigoXLab/dingo/pull/370
add article fact checking skills for Claude and OpenClaw by @seancoding-day in https://github.com/MigoXLab/dingo/pull/371
release 2.2.0 by @e06084 in https://github.com/MigoXLab/dingo/pull/372

Full Changelog: https://github.com/MigoXLab/dingo/compare/v2.1.0...v2.2.1

View release on GitHub

v2.1.0 New feature 2mo

Notable features

Add five RAG evaluation metrics
Support SQL data source multi‑column evaluation

Full changelog

What's Changed

add image rule guide by @pekopoke in https://github.com/MigoXLab/dingo/pull/237
fix:Layout Prompt by @chaserRen in https://github.com/MigoXLab/dingo/pull/239
add new prompt by @dt-yy in https://github.com/MigoXLab/dingo/pull/241
Fix/ade/dev by @decrystal in https://github.com/MigoXLab/dingo/pull/243
feat: add 5 RAG eval metrics by @e06084 in https://github.com/MigoXLab/dingo/pull/244
feat: support sql datasouce multi-column eval by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/259
feat: sql by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/260
feat: 以星河场景为例的介绍文档、example、rule by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/262
refactor: multi-turn dialog by @seancoding-day in https://github.com/MigoXLab/dingo/pull/264
feat: update readme by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/265
docs: update architecture pic by @e06084 in https://github.com/MigoXLab/dingo/pull/266
feat: label QUALITY_GOOD by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/267
fix: rag metric by @e06084 in https://github.com/MigoXLab/dingo/pull/269
fix: fix rag example by @e06084 in https://github.com/MigoXLab/dingo/pull/270
feat: add rag mock and eval by @e06084 in https://github.com/MigoXLab/dingo/pull/271
Dev lld: update 5 metrics and dataset for rags by @pekopoke in https://github.com/MigoXLab/dingo/pull/273
fix: rags of 5 metrics by @pekopoke in https://github.com/MigoXLab/dingo/pull/276
fix : embedding model change by @pekopoke in https://github.com/MigoXLab/dingo/pull/277
docs: update wechat by @e06084 in https://github.com/MigoXLab/dingo/pull/279
feat: gradio by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/280
feat: fix by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/281
feat: merge main by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/282
feat: update modelres by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/278
fix: fix rag example by @e06084 in https://github.com/MigoXLab/dingo/pull/283
feat: ci check import by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/285
feat: add score summary by @e06084 in https://github.com/MigoXLab/dingo/pull/286
Feature/ats keyword matcher and resume optimizer by @Kylie-dot-s in https://github.com/MigoXLab/dingo/pull/284
feat: summary support multi-column by @e06084 in https://github.com/MigoXLab/dingo/pull/287
fix: fix ut by @e06084 in https://github.com/MigoXLab/dingo/pull/288
feat: update auto-gen-metric by @e06084 in https://github.com/MigoXLab/dingo/pull/289
feat: update rag_eval_baseline by @e06084 in https://github.com/MigoXLab/dingo/pull/290
feat: update e2e RAG eval by @e06084 in https://github.com/MigoXLab/dingo/pull/292
feat: add PII detetction by @e06084 in https://github.com/MigoXLab/dingo/pull/293
feat: add LLMTextQualityV5 by @e06084 in https://github.com/MigoXLab/dingo/pull/294
fix : update answer_relevancy metric by @pekopoke in https://github.com/MigoXLab/dingo/pull/291
docs: update metrics docs by @e06084 in https://github.com/MigoXLab/dingo/pull/295
docs: update wechat doc by @e06084 in https://github.com/MigoXLab/dingo/pull/297
docs: update README by @e06084 in https://github.com/MigoXLab/dingo/pull/298
docs: update README by @e06084 in https://github.com/MigoXLab/dingo/pull/299
docs: update README by @e06084 in https://github.com/MigoXLab/dingo/pull/300
build: set numpy>=1.26.4 by @e06084 in https://github.com/MigoXLab/dingo/pull/301
feat: update gradio to v2.0 by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/302
feat: update embedding model init by @e06084 in https://github.com/MigoXLab/dingo/pull/303
feat: support excel by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/305
feat: fix bug gradio detail repeat by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/306
feat: unshare link by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/307
feat: update gradio image in readme, add limit of label in gradio by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/308
fix: fix embedding config load by @e06084 in https://github.com/MigoXLab/dingo/pull/309
Fix/docs and examples cleanup by @Kylie-dot-s in https://github.com/MigoXLab/dingo/pull/296
feat: support csv by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/310
feat: add Instruction Quality Evaluation by @e06084 in https://github.com/MigoXLab/dingo/pull/313
feat: add examples in metrics by @e06084 in https://github.com/MigoXLab/dingo/pull/314
feat: init agent&tool architecture by @seancoding-day in https://github.com/MigoXLab/dingo/pull/311
docs: update wechat by @e06084 in https://github.com/MigoXLab/dingo/pull/315
feat: fix spark by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/316
feat:front page V2.0 by @tenwanft in https://github.com/MigoXLab/dingo/pull/318
feat: support parquet file by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/320
Fix/docs only by @Kylie-dot-s in https://github.com/MigoXLab/dingo/pull/321
feat: use_browser by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/323
Feat/update all examples and tests by @Kylie-dot-s in https://github.com/MigoXLab/dingo/pull/322
feat: agent&tool docs/tests/examples by @seancoding-day in https://github.com/MigoXLab/dingo/pull/319
docs: update wechat doc by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/324
feat: add required fileds by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/327
feat: optimize label when data is empty in rag metrics by @e06084 in https://github.com/MigoXLab/dingo/pull/330
fix: update rag example by @e06084 in https://github.com/MigoXLab/dingo/pull/331
feat: add VLMRenderJudge metric by @e06084 in https://github.com/MigoXLab/dingo/pull/332
feat: x by @e06084 in https://github.com/MigoXLab/dingo/pull/334
feat: support sciencemetabench by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/333
feat: split langchain way and custom way; add description and docs for these two ways by @seancoding-day in https://github.com/MigoXLab/dingo/pull/336
feat: update front page jpg by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/339
Fix MCP stdio communication issues by @Kylie-dot-s in https://github.com/MigoXLab/dingo/pull/326
Feature/llm scout by @Kylie-dot-s in https://github.com/MigoXLab/dingo/pull/343
docs: README add SAAS apply by @e06084 in https://github.com/MigoXLab/dingo/pull/345
feat(agent): agent for article fact checking by @seancoding-day in https://github.com/MigoXLab/dingo/pull/348
package: optimize requirements by @e06084 in https://github.com/MigoXLab/dingo/pull/352
feat: Add cursor rule by @e06084 in https://github.com/MigoXLab/dingo/pull/353
feat: 2.1.0 by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/354
feat: cursor fix by @shijinpjlab in https://github.com/MigoXLab/dingo/pull/356

New Contributors

@decrystal made their first contribution in https://github.com/MigoXLab/dingo/pull/243
@tenwanft made their first contribution in https://github.com/MigoXLab/dingo/pull/318

Full Changelog: https://github.com/MigoXLab/dingo/compare/v1.11.1...v2.1.0

View release on GitHub

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.