Libraries tagged by zcrawler

A tool to crawl your own laravel installation checking your HTTP status codes

Go to Download

Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web search and testing solution. It can be used for a wide range of purposes, from extracting and indexing structured data to monitoring and automated testing.

Go to Download

A new generation of multi-process async event-driven spider engine based on Workerman

Go to Download

WebCrawler build with Laravel Dusk and Symfony DomCrawler

Go to Download

jQuery-inspired DOM manipulation extension for Symfony's Crawler

Go to Download

The standalone PHP Browscap parser Crossjoin\Browscap detects browser properties as well as device information based on the user agent string of the requesting browsers and search engines, using the data from the Browser Capabilities Project. It's several hundred times faster than the build-in PHP function get_browser(), and faster than other Browscap PHP libraries, with much lower memory consumption. Optionally Crossjoin\Browscap automatically updates the Browscap data, so you're always up-to-date. The newest version is build for PHP 7.x, for PHP >= 5.6 use version 2.x, for PHP >= 5.3 use version 1.x.

Go to Download

An Fully Automatic RESTful PHP Router.

Go to Download

Arca ORM by Scrawler

Go to Download

PHP Sitemap crawler

Go to Download

Kirby robots.txt plugin for blocking AI Crawlers and Bots

Go to Download

Utilities that are needed in multiple crawler packages.

Go to Download

Contao bundle helper class to detect search engines, bots, spiders, crawlers ...

Go to Download

A crawler to find all unique internal pages on a given website

Go to Download

宇润爬虫框架(Yurun Crawler) 是一个低代码、高性能、分布式爬虫采集框架，这可能是最一把梭的爬虫框架。

Go to Download

Crawls Website and can generate backstop json

Go to Download