webpeel/webpeel

v0.9.0 Feature

This release adds 3 notable features for engineering teams evaluating rollout.

Published 3mo MCP Developer Tools

View tool

✓ No known CVEs patched

Read the diff → Tool health → What is this tool? →

✓ No known CVEs patched in this version

Topics

ai-agent ai-agents browser-automation claude-code codex content-extraction

+14 more

cursor firecrawl-alternative mcp mcp-server model-context-protocol llm screenshot structured-data token-efficiency typescript web-crawler web-data web-scraping web-search

Summary

AI summary

Dynamic challenge detection and full escalation cascade improve stealth scraping.

Full changelog

What's New

🛡️ Stealth 10/10

Dynamic challenge detection — automatically detects Cloudflare, PerimeterX, Akamai, DataDome, Incapsula, and generic block pages with confidence scoring
Full escalation cascade: simple → browser → stealth → stealth+wait → return with warning
28 stealth domains with auto-detection (Amazon, eBay, Walmart, Nike, LinkedIn, and more)
Updated user agents (Chrome 132-136) with dynamic Sec-CH-UA header generation
challengeDetected flag on responses when a site can't be bypassed

🔍 Site-Aware Search

webpeel search --site ebay "charizard card" — 27 sites, 7 categories, no URL knowledge needed
Shopping: eBay, Amazon, Walmart, Target, Best Buy, Etsy, AliExpress, Newegg
General: Google, Bing, DuckDuckGo
Social: Reddit, YouTube, X/Twitter, LinkedIn
Tech: GitHub, Stack Overflow, npm, PyPI
Plus: jobs, real estate, food categories
webpeel sites to list all supported templates

🤖 Agent Mode

--agent flag sets --json, --silent, --extract-all, --budget 4000 in one shot
agentMode option in library API: peel(url, { agentMode: true })
MCP server defaults to 4,000 token budget for agent callers

📊 Listing Extraction

--extract-all auto-detects repeated DOM patterns (product cards, search results)
--table for Unicode box-drawing tables, --csv for CSV export
--pages <n> for multi-page pagination
--scroll-extract for infinite scroll extraction

Stats

368 tests passing
1,847+ lines added across 12 files
Live verified on 10+ real sites

Full Changelog: https://github.com/webpeel/webpeel/compare/v0.8.1...v0.9.0

View diff on GitHub

Weekly OSS security release digest.

The CVE patches and breaking changes that affected production tools this week. One email, every Sunday.

No spam, unsubscribe anytime.

Share this release

Share on X Share on Bluesky

Track webpeel/webpeel

Get notified when new releases ship.

About webpeel/webpeel

Smart web fetcher for AI agents with auto-escalation from HTTP to headless browser to stealth mode. Includes 9 MCP tools: fetch, search, crawl, map, extract, batch, screenshot, jobs, and agent. Achieved 100% success rate on a 30-URL benchmark.

All releases →