Libraries tagged by zcrawler

Craft plugin for detecting bots/crawlers/spiders

Go to Download

Modern library to handle RSS/Atom feeds

Go to Download

Standalone Laravel package for queued Flickr crawling with rate limiting and MySQL persistence.

Go to Download

build mirror of packagist.org

Go to Download

Extension for the crwlr/crawler package containing steps utilizing a headless browser.

Go to Download

Automated crawler-based generation and indexing to Solr.

Go to Download

Crawl sitemap.xml recursively and return collected urls as array

Go to Download

Makes data extraction from leboncoin.fr easy

Go to Download

A crawler application written with php and Laravel that finds email addresses on the internets.

Go to Download

A simple crawler (spider) writen in php just for fun

Go to Download

一个标准的短视频 APP 视频/图文抓取 SDK，支持抖音、小红书、快手等平台。

Go to Download

网页图片抓取 SDK：输入网页链接，返回图片链接数组。内置微信公众号文章等专项抓取，支持 ajax 与源码两种图片加载方式。

Go to Download

A Laravel package that provides a standardized .well-known/traffic-advice endpoint for communicating server load, maintenance windows, and access recommendations to automated clients, crawlers, and partner services.

Go to Download

Lightweight proxy with full support for sessions, cookies, POST/FORM submissions, and URL rewriting.

Go to Download

Symfony Spider Bundle

Go to Download