Libraries tagged by scrawl

The Google Analytics module consists of 2 components that can be employed independently: The Google Logger injects the google analytics javascript snippet into your source code and logs relevant events (as of now only crawler visits) The Analyzer adds the Google Analytics UI to your CMS.

Go to Download

Yet another DOM crawler, inspired by symfony/dom-crawler

Go to Download

A generic rate limiter for the web and reactphp. Useful for API clients, web crawling, or other tasks that need to be throttled

Go to Download

Crawl the web with kohana or phantomjs.

Go to Download

The Push API allows you to *push* items and security identities, as opposed to letting standard Coveo Cloud V2 crawlers *pull* this data from a content repository. This is especially useful when you need to index content from a cloud or on-premises system for which no dedicated source type exists in the Coveo Cloud V2 platform.

Go to Download

Generates an XML site map for use by web crawlers

Go to Download

A custom middleware package for Laravel. Temporarily blocks crawlers scanning for vulnerabilities.

Go to Download

Strong link obfuscation with PHP and JavaScript. Object-oriented, using the Rot47 algorithm. Hides email, phone and sms links from being crawled by spambots.

Go to Download

Crawler for IHA News Agency in Turkey.

Go to Download

Crawler for DHA News Agency in Turkey.

Go to Download

Crawler for AA News Agency in Turkey.

Go to Download

URL spider which crawls a page and all its subpages

Go to Download

Simple, elegant, extensible PHP Web Scraper (crawler/spider),Use the css3 dom selector,Based on phpQuery! 简洁、优雅、可扩展的PHP采集工具(爬虫)，基于phpQuery。

Go to Download

Lightweight web crawler for php

Go to Download

宇润爬虫框架(Yurun Crawler) 是一个低代码、高性能、分布式爬虫采集框架，这可能是最一把梭的爬虫框架。

Go to Download