Download the PHP package spatie/crawler without Composer
On this page you can find all versions of the php package spatie/crawler. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
FAQ
Example:
In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
- Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
- To use Composer is sometimes complicated. Especially for beginners.
- Composer needs much resources. Sometimes they are not available on a simple webspace.
- If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
- Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Informations about the package crawler
Crawl the web using PHP
[](https://packagist.org/packages/spatie/crawler) [](LICENSE.md)  [](https://packagist.org/packages/spatie/crawler)This package provides a powerful, easy to use class to crawl links on a website. Under the hood, Guzzle promises are used to crawl multiple URLs concurrently.
Because the crawler can execute JavaScript, it can crawl JavaScript rendered sites. Under the hood, Chrome and Puppeteer are used to power this feature.
Here's a quick example:
Or collect all URLs on a site:
You can also test your crawl logic without making real HTTP requests:
If you need to stop a crawl based on external state, you can register a callback that receives the current crawler instance and is checked before scheduling each next request:
Support us
We invest a lot of resources into creating best in class open source packages. You can support us by buying one of our paid products.
We highly appreciate you sending us a postcard from your hometown, mentioning which of our package(s) you are using. You'll find our address on our contact page. We publish all received postcards on our virtual postcard wall.
Documentation
All documentation is available on our documentation site.
Testing
Changelog
Please see CHANGELOG for more information on what has changed recently.
Contributing
Please see CONTRIBUTING for details.
Security Vulnerabilities
Please review our security policy on how to report security vulnerabilities.
Credits
- Freek Van der Herten
- All Contributors
License
The MIT License (MIT). Please see License File for more information.
All versions of crawler with dependencies
guzzlehttp/guzzle Version ^7.3
guzzlehttp/psr7 Version ^2.0
spatie/robots-txt Version ^2.0
symfony/css-selector Version ^7.0|^8.0
symfony/dom-crawler Version ^7.0|^8.0