Libraries tagged by zcrawler

A PHP SDK for WebCrawler API - turn website into data

Go to Download

A PHP class for making a static copy (or indexing) a site

Go to Download

A Laravel package that provides a standardized .well-known/traffic-advice endpoint for communicating server load, maintenance windows, and access recommendations to automated clients, crawlers, and partner services.

Go to Download

Server-side AI crawler tracking middleware for Laravel applications.

Go to Download

Symfony Generative Engine Optimization kit (not geolocation): AI crawler robots policy via SeoKit, llms.txt, citation index and audit for FrankenPHP, php-fpm and Nginx.

Go to Download

Laravel middleware za blokiranje AI crawlera i tražilica. Auto-registracija, konfigurabilan, podržava Laravel 10/11/12/13.

Go to Download

Scrapes the information from the targeted page and provides a DomCrawler

Go to Download

Crawl, match, parse IP or IP range, check if IP or range is in another range. Support IPv4, IPv6, IP Interval, Wildcard and CIDR. Check if IP is Cloudflare node IP, Google bot IP. 爬取，正则匹配，解析 IP 和 IP 范围，检测 IP 或范围是否在另一个范围中。支持 IPv4，IPv6，区间、通配符或 CIDR 表示的 IP 范围。检测 IP 是否是 Cloudflare 节点或 Google 漫游器 IP

Go to Download

Google search results crawler, get google search results that you need.

Go to Download

Sitemap crawler and generator class

Go to Download

Package to provide api-like access to foreign sites based on html parsing (fork of sleeping-owl/apist package with updated dependencies)

Go to Download

A Laravel package to fetch webpage metadata ( Open Graph | Twitter | Facebook | Article ) ect...

Go to Download

A PHP desktop/mobile user agent parser with bot detection, based on Mobiledetect and CrawlerDetect.

Go to Download

Drop-in replacement for the unmaintained jenssegers/agent — same Agent API, on modern Mobile-Detect (v4 and v2)

Go to Download

A simple website crawler & sitemap generator without a headless browser for Laravel 5.8+

Go to Download