Libraries tagged by crawlora

Community fork of roach-php/laravel with Laravel 10–13 compatibility, powered by optimized lazy-core

Go to Download

Community fork of roach-php/core with on-demand lazy DOM loading and native JSON response parsing

Go to Download

Serve static content of a one-page website to crawlers (e.g. Facebook, Google or Twitter) using PhantomJS

Go to Download

Detect robots and crawlers.

Go to Download

A Symfony bundle to crawl a site and audit technical SEO signals: canonical tags, indexing directives and redirects.

Go to Download

A Symfony bundle to crawl a site and audit on-page SEO signals (titles, meta descriptions, headings, image alt attributes).

Go to Download

A Symfony bundle to crawl a site and detect broken internal and external links.

Go to Download

A web scraper for PHP to easily extract data from web pages

Go to Download

PHP async scrapper used multi curl and reactphp, and proxy inspired by python grab

Go to Download

A class to help handle gateways to processing entities.

Go to Download

Event based site tool and tester.

Go to Download

Last.fm webservice client for php.

Go to Download

Pre-render your site for web crawlers

Go to Download

Marked Down serves Markdown to AI crawlers, command-line tools, and APIs while browsers get HTML. Zero-config per-section setup, automatic AI bot detection, .md URL suffix, /llms.txt, YAML front matter, and dedicated Markdown templates — out of the box.

Go to Download

Yo! Micro Web Crawler in PHP & Manticore

Go to Download