Diffbot

A set of APIs and a knowledge graph that turn the public web into structured, queryable data for AI agents and pipelines, solving the need to extract, crawl, and search web content without building scrapers. It is for developers, data and engineering teams in businesses that need web-scale data for research, market intelligence, and automation. It is delivered as SaaS and infrastructure APIs with self-hostable web search options and token-efficient agent skills.

Key features

• Extract structured data from any URL
• Crawl entire sites into datasets
• Natural language entity and relation extraction
• Query Knowledge Graph of people and organizations
• Live structured web search API
• Self-hostable web search index
• Bulk extract and bulk enhance jobs
• Custom extraction rules and selectors
• Agent skills for token-efficient querying