Crawler Software: 34 Free Downloads
Every program tagged "Crawler" on LvRuan, newest first, each with an overview, screenshots and a download link.
Job Scout
Scrape job boards and get match alerts
Spider
Low-latency web crawling in Rust
JobSpy
Job scraper for LinkedIn, Indeed and more
soccerdata
Scrape soccer data from many sources
Upwork Jobs Scraper
Apify actor scraping Upwork job postings
Fredy
Automatic apartment search for Europe
AutoCLI
Fetch information from any website with one command
Universal Reddit Scraper
Full-featured Reddit scraper with dashboard
Reverse API Engineer
The agent that turns websites into APIs
CyberScraper 2077
LLM-powered web scraper
Goclone
Clone websites to your computer in seconds
Maxun
Turn any website into a structured API
Firecrawl Web Agent
Open-source web data agent
Open Agent Builder
Visual workflow builder for AI agents
Firecrawl CLI
Firecrawl CLI for AI agents
Newspaper3k
News article extraction in Python
Armada
Kubernetes-native bot and scraper orchestration
Crawlee for Python
Web scraping and browser automation library for Python
Ferret
Declarative data extraction language
CrawlerDetect
PHP class to detect bots by user agent