This release keeps dependencies and maintenance posture current for teams operating this tool.
Published 1mo
RAG & Retrieval
✓ No known CVEs patched
✓ No known CVEs patched in this version
Topics
ai-agents
anthropic-claude
data-extraction
gemini
ingestion-pipeline
llm
+9 more
markdown-extraction
openai
pgvector
python
sitemap-crawler
structured-data
supabase
vector-db
webcrawler
Summary
AI summaryMinor fixes and improvements.
Full changelog
What's changed
- Benchmarks split into separate repo: AIMLPM/llm-crawler-benchmarks
- README benchmark links now point to the new repo
- CLAUDE.md trimmed to crawler-only rules
- Makefile, CI, .gitignore scoped to crawler code only
- Dockerfile cleaned up (non-root user, no benchmark references)
- CONTRIBUTING.md links to benchmark repo
No code changes to the crawler itself.
Weekly OSS security release digest.
The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.
No spam, unsubscribe anytime.
Share this release
About AIMLPM/markcrawl
Crawl websites into clean Markdown, search pages, and extract structured data with LLMs. Built-in MCP server for web research and RAG pipelines.
Beta — feedback welcome: [email protected]