PHP download

Download the PHP package mediamonks/crawler without Composer

On this page you can find all versions of the php package mediamonks/crawler. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.

Table of contents
Download mediamonks/crawler
More information about mediamonks/crawler
Files in mediamonks/crawler

Vendor mediamonks
Package crawler
Short Description Crawl your own website with various clients for SEO and indexing purposes.
License MIT
Homepage https://www.mediamonks.com/

Keywords search dom seo goutte spider crawler index domcrawler prerender robot prerender.io

FAQ

After the download, you have to make one include require_once('vendor/autoload.php');. After that you have to import the classes with use statements.

Example:

If you use only one package a project is not needed. But if you use more then one package, without a project it is not possible to import the classes with use statements.

In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.

Some PHP packages are not free to download and because of that hosted in private repositories. In this case some credentials are needed to access such packages. Please use the auth.json textarea to insert credentials, if a package is coming from a private repository. You can look here for more information.

Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
To use Composer is sometimes complicated. Especially for beginners.
Composer needs much resources. Sometimes they are not available on a simple webspace.
If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.

Please rate this library. Is it a good library?

Informations about the package crawler

MediaMonks Crawler

This tool allows you to easily crawl a website and get a DOM object for every url that was found. We use this to crawl our own site pages regardless if it was generated with server and/or client side content by using the Prerender.io client. The resulting data can be used for creating a full site search and/or improving SEO for single-page applications.

Highlights

Ships with Prerender & Prerender.io clients, uses Goutte by default
Supports any Symfony BrowserKit client
Supports both whitelisting and blacklisting of urls
Supports url normalization which allow you to prevent duplicates based on minor url differences
Implements the PSR-3 Logger Interface

Documentation

Documentation and examples can be found in the /doc folder.

System Requirements

You need:

PHP >= 5.5.0

To use the library.

Install

Install this package by using Composer.

Security

If you discover any security related issues, please email [email protected] instead of using the issue tracker.

License

The MIT License (MIT). Please see License File for more information.

All versions of crawler with dependencies

PHP Build Version

Package Version

Version 2.0.0 Release 04. Dec 2017
create-project require 1 people choosed require and
0 people choosed create-project.

Download

Download latest version of crawler from vendor mediamonks

Requires php Version ^5.5|^7.0
symfony/dom-crawler Version ^2.8|^3.0|^4.0
fabpot/goutte Version ^3.0
league/uri Version ^4.2
psr/log Version ^1.0

Composer command for our command line client (download client) This client runs in each environment. You don't need a specific PHP version etc. The first 20 API calls are free. Standard composer command

The package mediamonks/crawler contains the following files

Loading the files please wait ....