Download the PHP package marioungui/php-component-spider without Composer

On this page you can find all versions of the php package marioungui/php-component-spider. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.

FAQ

After the download, you have to make one include require_once('vendor/autoload.php');. After that you have to import the classes with use statements.

Example:
If you use only one package a project is not needed. But if you use more then one package, without a project it is not possible to import the classes with use statements.

In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
Some PHP packages are not free to download and because of that hosted in private repositories. In this case some credentials are needed to access such packages. Please use the auth.json textarea to insert credentials, if a package is coming from a private repository. You can look here for more information.

  • Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
  • To use Composer is sometimes complicated. Especially for beginners.
  • Composer needs much resources. Sometimes they are not available on a simple webspace.
  • If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
  • Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Please rate this library. Is it a good library?

Example code of marioungui/php-component-spider

Informations about the package php-component-spider

PHP Component Spider

License: MIT CodeFactor Latest Stable Version License PHAR Build

This PHP Component Spider is designed to scrape websites for specific components or search criteria defined by XPath filters. It uses the PHPScraper library to fetch and process web pages, and the League\Csv library to log the results in CSV files. This tool is easy to extend with custom XPath filters to meet various scraping needs.

Features

Requirements

Build & Run from Source Code

  1. Clone the repository:

  2. Navigate to the project directory:

  3. Install the dependencies using Composer:

  4. Build the Phar package:

  5. Run the batch spider.bat
  6. Follow the on-screen instructions to select the component to search for and the domain to scrape.

Filters

The filters are defined in filters.php and use XPath to identify specific components on the web pages. Here are the current filters available:

Component Index Filter
MVP Block 1 //*[@class='mvp-block']
Smart Question Search Engine Block 2 //*[@class='sqe-block']
Related Articles Block 3 //h2[text()='Artigos relacionados' or text()='Artigos Relacionados' or text()='Articulos Relacionados' or text()='Articulos relacionados' ]
Related Products Block 4 //h2[text()='Produtos Relacionados' or text()='Produtos Relacionados' or text()='Productos relacionados' or text()='Productos Relacionados']
Brands Block 5 //*[starts-with(@id, 'brands_block')]/@id
Stages Block 6 //*[starts-with(@id, 'stages_block')]
String Search 7 //*[contains(text(),'word')]
Action Bar 8 //div[contains(@class, 'action-bar__wrapper')]
Links Containing 9 //a[contains(@href, 'word')]
Stages Block using From Library 10 //div[contains(@class, 'paragraph--type--stages-block')]//div[contains(@class, 'grid-col-10')]

Extending with Custom Filters

Extending the tool with new filters is simple:

  1. Open the filters.php file.
  2. Add a new case in the switch statement with your component name or index.
  3. Define the $component and $filter variables with your custom XPath.

Example:

Contributing

Feel free to submit issues or pull requests if you have any improvements or new features you'd like to add.

License

This project is licensed under the MIT License.

All versions of php-component-spider with dependencies

PHP Build Version
Package Version
Requires php Version ^8.1.2
spekulatius/phpscraper Version ^2.0
symfony/http-kernel Version ^5.4
league/csv Version ^9.8
symfony/browser-kit Version ^6.4
Composer command for our command line client (download client) This client runs in each environment. You don't need a specific PHP version etc. The first 20 API calls are free. Standard composer command

The package marioungui/php-component-spider contains the following files

Loading the files please wait ....