Libraries tagged by crawler
nirjharlo/complete-google-seo-scan
11 Downloads
WordPress Plugin with inbuilt SEO crawler
nicolas-joubert/grabit-front-feed-bundle
130 Downloads
Atom feed frontend for GrabitBundle
nicolas-joubert/grabit-bundle
208 Downloads
Grab it is R&D crawler to grab HTML pages
nicolas-joubert/grabit-admin-sonata-bundle
58 Downloads
Sonata Admin for GrabitBundle
mihaeu/tarantula
50 Downloads
Another PHP crawler based on Guzzle.
mediashare/spider
121 Downloads
Spider is a php library for crawling website that allows you to scrape informations & automated actions.
mediashare/scraper
289 Downloads
Scrapes the information from the targeted page and provides a DomCrawler
laurentvw/scrapher
2537 Downloads
A web scraper for PHP to easily extract data from web pages
itgalaxy/webcrawler-verifier
2992 Downloads
PHP library providing functionality to verify that user-agents are who they claim to be.
gstjohn/thumbsnag
29 Downloads
Thumbsnag crawls an HTML document and finds imagery that best represents the given page.
ghalambaz/googleplay-spider
235 Downloads
Another GooglePlay Website Scraper based on goutte
dkm/typo3-robots-guard
11 Downloads
Centrally manages robots.txt, the robots head meta tag and XML sitemap inclusion for any TYPO3 site (single-site or multisite), with environment-aware safety logic that guards production against accidental crawl-blocking while keeping staging/dev fully disallowed.
dimtrovich/user-agent
166 Downloads
A PHP desktop/mobile user agent parser with bot detection, based on Mobiledetect and CrawlerDetect.
diggin/diggin-robotrules
187 Downloads
parser/handler for Robots Exclusion Protocol (robots.txt and more)
cobaia/marsvin
113 Downloads
Marsvin it's a framework to write crawler applications