Libraries tagged by cradle

Go to Download

Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping

Go to Download

The BVP Scraper for Boatrace.

Go to Download

A powerful web crawler and web scraper with Blackfire support

Go to Download

Check all links in a Laravel app

Go to Download

A simple and lightweight cURL library with support for asynchronous requests.

Go to Download

TYPO3 sitemap crawler

Go to Download

Go to Download

Sitemap crawler/generator. For the given URL it will return sitemap XML file with URLs and images.

Go to Download

An full search page crawler to enable complex and customized searching abilities.

Go to Download

Web Crawler using Laravel Dusk

Go to Download

Verify Googlebot (or others) by their IP addresses

Go to Download

Prevent crawlers from creating a session

Go to Download

Yii2 extension for CrawlerDetect library

Go to Download

Crawl, match, parse IP or IP range, check if IP or range is in another range. Support IPv4, IPv6, IP Interval, Wildcard and CIDR. Check if IP is Cloudflare node IP, Google bot IP. 爬取，正则匹配，解析 IP 和 IP 范围，检测 IP 或范围是否在另一个范围中。支持 IPv4，IPv6，区间、通配符或 CIDR 表示的 IP 范围。检测 IP 是否是 Cloudflare 节点或 Google 漫游器 IP

Go to Download