Libraries tagged by crawling

Laravel package to enable crawling/parsing HTML pages and generating corresponding RSS feeds

Go to Download

Laravel package to enable crawling web pages for event data, generating corresponding ICS feeds

Go to Download

Effortless recursive web crawling for comprehensive site indexing

Go to Download

cURL wrapper for PHP. Supports parallel and non-blocking requests. For high speed crawling, see stil/curl-robot.

Go to Download

Laravel sitemap generator and crawler package for SEO optimization. Supports multilingual sites, image/video indexing, link validation, priority scoring, and advanced crawling automation.

Go to Download

A free proxy list based downloader for your crawling projects in PHP

Go to Download

A utility for crawling multiple gitlab repositories

Go to Download

This is a Wrapper for the API provided by most Wordpress Blogs. Intended for easy crawling of a Wordpress page.

Go to Download

Symfony 2.X bundle for easy crawling Facebook page

Go to Download

PHP Website Email Crawler, for crawling websites for emails.

Go to Download

Parallel URL crawling extension to curl-easy.

Go to Download

Crawl Engine is a PHP Library that helps to automate the Process of Login into password Protected Sites and Getting Needed information from them.

Go to Download

whm library for crwling websites

Go to Download

MessageBird REST API client for PHP

Go to Download

The SimpleData module simplifies calling Magento data structures.

Go to Download