Crawlee

Provides features such as task queue, automatic retries, proxy management, and browser automation, suitable for large-scale web scraping.