Libraries tagged by craw

Instrument headless chrome/chromium instances from php5.6

Go to Download

PHP SDK for Firecrawl API - Web scraping and crawling service

Go to Download

A modern, fluent PHP package for managing robots.txt rules with type safety and great developer experience

Go to Download

PHP SDK for the Firecrawl v2 API with Laravel support

Go to Download

A Laravel package to fetch webpage metadata ( Open Graph | Twitter | Facebook | Article ) ect...

Go to Download

A SilverStripe module to optimise the Meta, crawling, indexing, and sharing of your website content

Go to Download

Creates a sitemap.xml by parsing the whole website.

Go to Download

Instrument headless chrome/chromium instances from PHP

Go to Download

jQuery-inspired DOM manipulation extension for Symfony's Crawler

Go to Download

The standalone PHP Browscap parser Crossjoin\Browscap detects browser properties as well as device information based on the user agent string of the requesting browsers and search engines, using the data from the Browser Capabilities Project. It's several hundred times faster than the build-in PHP function get_browser(), and faster than other Browscap PHP libraries, with much lower memory consumption. Optionally Crossjoin\Browscap automatically updates the Browscap data, so you're always up-to-date. The newest version is build for PHP 7.x, for PHP >= 5.6 use version 2.x, for PHP >= 5.3 use version 1.x.

Go to Download

A crawler to find all unique internal pages on a given website

Go to Download

Encrypt e-mail addresses and phone numbers to protect against unwanted crawlers & spam.

Go to Download

A fast, simple and powerful PHP web crawler (scraper/spider) 快速、简洁且强大的爬虫/采集框架

Go to Download

LLMs.txt Generator - Generates llms.txt files for AI/LLM crawlers with website content in Markdown format, with optional API key protection.

Go to Download

Crawl the web with kohana or phantomjs.

Go to Download