Libraries tagged by robots
monperrus/crawler-user-agents
6349 Downloads
This repository contains a list of of HTTP user-agents used by robots, crawlers, and spiders as in single JSON file.
georgringer/news-seo
102690 Downloads
Indivdual indexing/robot information for each news article record
ymlluo/group-robot
4941 Downloads
钉钉、企业微信、飞书 自定义群组机器人。支持链式语法创建发送消息。支持文本、Markdown、图片、文件、图文、卡片等消息。
tractorcow/silverstripe-robots
99396 Downloads
Simple robots generation module for Silverstripe
mguinea/laravel-robots
64551 Downloads
Laravel package to manage robots in an easy way
crwlr/robots-txt
35300 Downloads
Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping
tomverran/robots-txt-checker
64640 Downloads
Given a robots.txt file, user agent and URL path will tell you whether you're allowed to access a page
monsieurbiz/sylius-robots-txt-plugin
13617 Downloads
Manage your robot.txt from your Sylius admin pannel
ghostcompiler/laravel-hetzner-robot
2267 Downloads
Production-ready Laravel SDK for the Hetzner Robot API
frosh/robots-txt
36646 Downloads
Generate robots.txt
bnomei/kirby3-robots-txt
50179 Downloads
Manage a virtual robots.txt from the Kirby config file
vipnytt/useragentparser
995495 Downloads
User-Agent parser for robot rule sets
chowjiawei/laravel-help-plugin
1847 Downloads
Integrating useful auxiliary functions into laravel,Assistant tools based on laravel.
spryker/robotframework-suite-tests
98308 Downloads
Automated tests for the Robot Framework
eliashaeussler/typo3-sitemap-locator
322929 Downloads
Sitemap Locator - Looks up XML sitemaps as part of a configured site. Supports various sitemap providers, e.g. by configured page type or robots.txt, and allows to implement custom providers.