Download the PHP package rfussien/leboncoin-crawler without Composer
On this page you can find all versions of the php package rfussien/leboncoin-crawler. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download rfussien/leboncoin-crawler
More information about rfussien/leboncoin-crawler
Files in rfussien/leboncoin-crawler
Package leboncoin-crawler
Short Description Makes data extraction from leboncoin.fr easy
License MIT
Homepage https://github.com/rfussien/leboncoin-crawler
FAQ
Example:
In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
- Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
- To use Composer is sometimes complicated. Especially for beginners.
- Composer needs much resources. Sometimes they are not available on a simple webspace.
- If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
- Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Informations about the package leboncoin-crawler
Crawler for leboncoin.fr
This is a small crawler package for the site leboncoin.fr.
Why ?
leboncoin.fr is one of the most famous classified ads website in france. Whatever what you're looking for, it is probably there. It has a lots of ads and is very fast and simple to use.
However, the first problem comes when you need to exploit the search results in a way that the site can't help you. In fact, the search results page is pretty poor in terms of data. For exemple, it'd be so cool to get the mileage when I'm looking for a car or the surface when I'm looking for a flat.
The second problem is that saving a search is really a pain on the actual site. All the searches you want to save give one single result page. That's pretty dumb, but anyway.
And the third and last problem is that you are forced to use selected values with some criterias. For example, when I was looking for a motorcycle, I was looking for those with a bigger engine than 1200cc. The fact that the biggest value available in the input is 1000cc and because there are tons of ads with 1000cc motorcycle, it made search much more complicated. I did send an email to ask for an additional value, but I didn't get any answer (which I didn't expect anyway). So I had to change the value in the query string every single request... What a waste of time...
So for all those little reasons, I decided to write my good old web scraper to be able to extract the data from the site to anywhere (a DB, an array, a json, an api, who knows...).
Requirements
- PHP 7
- [optional] PHPUnit to execute the test suite
Install
Usage
Super easy !!!
Get the structured data from a search result page
example of output:
Get the structured data from an ad
example of output:
There are a bunch of features if you digg a bit in the sources.
Testing
Contributing
Please see CONDUCT for details.
Security
If you discover any security related issues, please email me ([email protected]) instead of using the issue tracker.
Credits
- Rémi FUSSIEN
- All Contributors
License
The MIT License (MIT). Please see License File for more information.
All versions of leboncoin-crawler with dependencies
league/uri Version ^4
nesbot/carbon Version ~1.22
php Version >=7.0
symfony/css-selector Version ~3
symfony/dom-crawler Version ~3