Download the PHP package spatie/robots-txt without Composer
On this page you can find all versions of the php package spatie/robots-txt. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download spatie/robots-txt
More information about spatie/robots-txt
Files in spatie/robots-txt
Package robots-txt
Short Description Determine if a page may be crawled from robots.txt and robots meta tags
License MIT
Homepage https://github.com/spatie/robots-txt
Informations about the package robots-txt
Parse robots.txt
, robots
meta and headers
Determine if a page may be crawled from robots.txt, robots meta tags and robot headers.
Support us
We invest a lot of resources into creating best in class open source packages. You can support us by buying one of our paid products.
We highly appreciate you sending us a postcard from your hometown, mentioning which of our package(s) you are using. You'll find our address on our contact page. We publish all received postcards on our virtual postcard wall.
Installation
You can install the package via composer:
Usage
You can also specify a user agent:
By default, Robots
will look for a robots.txt
file on https://host.com/robots.txt
.
Another location can be specified like so:
Testing
Changelog
Please see CHANGELOG for more information what has changed recently.
Contributing
Please see CONTRIBUTING for details.
Security Vulnerabilities
Please review our security policy on how to report security vulnerabilities.
Postcardware
You're free to use this package, but if it makes it to your production environment we highly appreciate you sending us a postcard from your hometown, mentioning which of our package(s) you are using.
Our address is: Spatie, Kruikstraat 22, 2018 Antwerp, Belgium.
We publish all received postcards on our company website.
Credits
- Brent Roose
- All Contributors
License
The MIT License (MIT). Please see License File for more information.