Top open source data and search projects for your next project

Top Picks

Crawl4AI
84.2k GitHub stars
Reads websites for you and turns them into clean text or tidy data your AI can use, even pages that only load in a browser.
- Web scraping & crawling
- RAG & agents
- Apache 2.0 with attribution clause
Apache Superset
74.9k GitHub stars
Apache Superset is a Data Visualization and Data Exploration Platform
- Business intelligence
- Apache 2.0
Scrapy
64.5k GitHub stars
A fast, long-trusted Python tool for collecting information from large websites and saving it as tidy data you can use.
- Web scraping & crawling
- Command line tools
- BSD 3-Clause
Meilisearch
59.4k GitHub stars
A lightning-fast search engine API bringing AI-powered hybrid search to your sites and applications.
- Search engines
- Other license
DuckDB
41.6k GitHub stars
DuckDB is an analytical in-process SQL database management system
- Analytical databases
- MIT License
Chroma
29.3k GitHub stars
Search infrastructure for AI
- Vector databases
- Apache 2.0
Crawlee
25.9k GitHub stars
Build web crawlers that keep going when things go wrong. Reads simple pages quickly and opens a real browser only when a page needs one.
- Web scraping & crawling
- RAG & agents
- Apache 2.0