Libraries tagged by web crawling

An extremely fast web scraper that parses megabytes of HTML in a blink of an eye. No dependencies. PHP5+

Go to Download

Web crawling and scraping library.

Go to Download

PHP SDK for the Firecrawl v2 API with Laravel support

Go to Download

A small PHP app to protect your site against DDoS attack or crawling web site by bots.

Go to Download

PHP SDK for Firecrawl API - Web scraping and crawling service

Go to Download

Generates an llms.txt file according to the llmstxt.org specification and serves pages as Markdown, giving Large Language Models a structured, machine-readable view of your site.

Go to Download

Crawlzone is a fast asynchronous internet crawling framework aiming to provide open source web search and testing solution. It can be used for a wide range of purposes, from extracting and indexing structured data to monitoring and automated testing.

Go to Download

Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping

Go to Download

High-performance web crawling engine

Go to Download

TYPO3 extension for generating llms.txt links based on the llmstxt.org specification and an on-site AI search assistant that helps visitors find information via ke_search and indexed_search.

Go to Download

A web crawler package

Go to Download

Just another curl wrapper for webCrawling purposes

Go to Download

A web scraper for PHP to easily extract data from web pages

Go to Download

A web scraper for PHP to easily extract data from web pages

Go to Download

A fluent PHP wrapper for web page content retrieval, screenshots, PDFs, Lighthouse audits, and concurrent site crawling using Chrome.

Go to Download