Download the PHP package slote/spider without Composer
On this page you can find all versions of the php package slote/spider. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
FAQ
Example:
In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
- Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
- To use Composer is sometimes complicated. Especially for beginners.
- Composer needs much resources. Sometimes they are not available on a simple webspace.
- If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
- Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Informations about the package spider
Spider
:dizzy: Spider is a PHP library with easily module integration for crawling website that allows you to scrape informations.
Spider is a crawler of website modulable write in PHP. The tool allows you to retrieve information and execute code on website pages. It can be useful for SEO or security audit purposes. Users have the possibility to use the modules created by the community or to create their own modules (written in PHP via a web interface).
What is a Crawler?
A crawler is an indexing robot, it automatically explores the pages of a website. Using a crawler can have several interests:
- Information search & retrieval
- Validation of the SEO of your website
- Integration test
- Execution of PHP code on several pages in an automated way
Features
- Get all links from website
- Check HTTP response
- Create your own Modules (Crawl & execute your PHP code)
- No database, Pure PHP
- Output json file
- Use default modules from the kernel for basic SEO audit. (Metadata, Images, HttpCode, Links...)
Libraries
- Autoloader php class for code integration easily. mediashare/modules-provider
- Website bot crawler. mediashare/crawler
- Scraper with DomCrawler integration. mediashare/scraper
I would be happy to receive your ideas and contributions to the project :smiley:
Getting started
Installation
Composer Usage
Use Spider library in your project & create your own modules.
Usage
Github
Execute the code from the console.
Binary file
Execute the code from the console.
Modules
Modules are tools created by the community to add features when crawling a website. Adding a module to a crawler allows the automation of code execution on one or more pages of a website. More information...
Requierements
- The name of your class needs to be the same as the name of the .php file.
- The entry point for executing modules is the run() function, so it is mandatory to have a run() function in your module.
Documentation
DomCrawler is symfony component for DOM navigation for HTML and XML documents. You can retrieve Documentation Here.
Create own module to execute actions with data scraped.
All versions of spider with dependencies
symfony/finder Version ^5.0
symfony/filesystem Version ^5.0
zumba/json-serializer Version ^2.2
mediashare/crawler Version *
mediashare/kernel Version *
symfony/console Version ^5.0