An open-source crawling toolkit that gives AI agents restricted, governed web access instead of unrestricted network access.
What to know▼
- How it works
- Static and rendered crawling, adaptive traversal, screenshots, PDFs, structured extraction via explicit origins, budgets, and runtime controls.
- What's different
- Focused on governed/restricted access for AI agents rather than open crawling.
Watched for 2 days · last checked 1d ago










