Libraries tagged by crawl

Clusteer is a Puppeteer wrapper written for PHP, with the super-power of parallelizing pages across multiple browser instances.

Go to Download

Headless manager for the well-known files every modern web app owes bots AND agents — robots.txt, .well-known/security.txt, llms.txt, humans.txt, sitemap.xml, and an agents/AI manifest. Define them once in code/config and serve them consistently. Zero runtime deps; ships a default-open robots evaluator so scrapers can honor robots honestly, and makes it hard to leak a protected path. Framework-agnostic core + Laravel adapter.

Go to Download

Laravel middleware za blokiranje AI crawlera i tražilica. Auto-registracija, konfigurabilan, podržava Laravel 10/11/12/13.

Go to Download

Crawl, match, parse IP or IP range, check if IP or range is in another range. Support IPv4, IPv6, IP Interval, Wildcard and CIDR. Check if IP is Cloudflare node IP, Google bot IP. 爬取，正则匹配，解析 IP 和 IP 范围，检测 IP 或范围是否在另一个范围中。支持 IPv4，IPv6，区间、通配符或 CIDR 表示的 IP 范围。检测 IP 是否是 Cloudflare 节点或 Google 漫游器 IP

Go to Download

Google search results crawler, get google search results that you need.

Go to Download

Sitemap crawler and generator class

Go to Download

Package to provide api-like access to foreign sites based on html parsing (fork of sleeping-owl/apist package with updated dependencies)

Go to Download

A Laravel package to fetch webpage metadata ( Open Graph | Twitter | Facebook | Article ) ect...

Go to Download

A simple website crawler & sitemap generator without a headless browser for Laravel 5.8+

Go to Download

PHPCreeper plugin for webman

Go to Download

Official, experimental (AI-generated and AI-maintained) Apify API client for PHP: run Actors, manage storages (datasets, key-value stores, request queues), schedules, webhooks and more.

Go to Download

High-performance device detection engines for the 51Degrees Pipeline API. Accurately identify devices, operating systems, browsers, and crawlers from HTTP headers.

Go to Download

Lib de validação de renavam no laravel

Go to Download

Lib de validação de inscrição estadual no laravel

Go to Download

Lib para converter HTML em PDF de uma maneira segura para as necessidades dos crawlers.

Go to Download