Libraries tagged by craw

【正方教务】爬虫，支持成绩查询、考试查询、课表查询、四六级成绩查询、四六级报名、选课查询、修改密码、获取用户菜单等功能，并且解析数据成易读格式，符合 psr 规范，拿来即用

Go to Download

Crawls Website and can generate backstop json

Go to Download

Boost Lighthouse and PageSpeed scores by hiding cookie banners, chat widgets and heavy scripts from crawlers via two Blade directives. Octane-safe.

Go to Download

Crawls all unique links.

Go to Download

Provides a way to create tests that crawl a site

Go to Download

Web Crawler to check few SEO basics.

Go to Download

Craft plugin for detecting bots/crawlers/spiders

Go to Download

Shared, framework-agnostic building blocks for polite and safe web crawlers: SSRF-aware HTTP throttling, robots.txt parsing, and HTML link discovery.

Go to Download

Modern library to handle RSS/Atom feeds

Go to Download

Crawlers package for extas

Go to Download

build mirror of packagist.org

Go to Download

Makes data extraction from leboncoin.fr easy

Go to Download

A crawler application written with php and Laravel that finds email addresses on the internets.

Go to Download

A PHP class for making a static copy (or indexing) a site

Go to Download

A Laravel package that provides a standardized .well-known/traffic-advice endpoint for communicating server load, maintenance windows, and access recommendations to automated clients, crawlers, and partner services.

Go to Download