Libraries tagged by web crawler

A modern, fluent PHP package for managing robots.txt rules with type safety and great developer experience

Go to Download

PHP SDK for Firecrawl API - Web scraping and crawling service

Go to Download

Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web search and testing solution. It can be used for a wide range of purposes, from extracting and indexing structured data to monitoring and automated testing.

Go to Download

A fast, simple and powerful PHP web crawler (scraper/spider) 快速、简洁且强大的爬虫/采集框架

Go to Download

Obfuscate emails, phone numbers and other sensitive data using PHP and modern web features. Visible to humans, hidden from crawlers and headless bots until they interact.

Go to Download

Web Crawler to check few SEO basics.

Go to Download

A professional, production-ready PHP XML sitemap generator with advanced features: concurrency, robots.txt, caching, database storage, change detection, SEO analysis, JavaScript rendering, proxy support, webhooks, and more.

Go to Download

Lightweight proxy with full support for sessions, cookies, POST/FORM submissions, and URL rewriting.

Go to Download

Generates an XML site map for use by web crawlers

Go to Download

PHPCrawl is a webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.

Go to Download

Simple, elegant, extensible PHP Web Scraper (crawler/spider),Use the css3 dom selector,Based on phpQuery! 简洁、优雅、可扩展的PHP采集工具(爬虫)，基于phpQuery。

Go to Download

Lightweight web crawler for php

Go to Download

A Laravel package to detect web crawlers via the user agent

Go to Download

Web crawler based on PHP Guzzle HTTP Client with concurrency support for faster operation.

Go to Download

A web crawler with a fluent interface

Go to Download