Download the PHP package snippetify/snippet-sniffer without Composer
On this page you can find all versions of the php package snippetify/snippet-sniffer. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download snippetify/snippet-sniffer
More information about snippetify/snippet-sniffer
Files in snippetify/snippet-sniffer
Package snippet-sniffer
Short Description Crawling and scraping web pages to extract snippets
License MIT
Homepage https://snippetify.com
Informations about the package snippet-sniffer
Snippet sniffer
Snippet sniffer allows you to extract code snippets from any websites.
What it does
This library allows you
- To get code snippets using search engine api (Google)
- To get code snippets from any web page by crawling url seeds.
How to use it
Snippet Sniffer
Providers
Providers allow you to get a stack of seeds(urls to scrape) from search engine API. Only Google search engine API is supported at this time, but you can create your own.
Add new providers to package
- Git clone the project
- Create your new class in the
Snippetify\SnippetSniffer\Providers
folder - Each provider implements
Snippetify\SnippetSniffer\Providers\ProviderInterface
- Take a look at
Snippetify\SnippetSniffer\Providers\GoogleProvider
to get you helped - Your fetch method must return an array of
Psr\Http\Message\UriInterface
- Add it in the providers stacks in the
Snippetify\SnippetSniffer\Core.php
- Write tests. Take a look at
Snippetify\SnippetSniffer\Tests\Providers\GoogleProviderTest
to get you helped - Send a pull request to us
Use your own providers
- Your provider must implement
Snippetify\SnippetSniffer\Providers\ProviderInterface
- Take a look at
Snippetify\SnippetSniffer\Providers\GoogleProvider
to get you helped - Your fetch method must return an array of
Psr\Http\Message\UriInterface
- Pass your new provider in the configuration parameter or use the
addProvider
method
Scrapers
Scrappers allow you to scrape html page and extract the snippets.
Add new scrapers to package
- Git clone the project
- Create your new class in the
Snippetify\SnippetSniffer\Scrapers
folder - Each scraper implements
Snippetify\SnippetSniffer\Scrapers\ScraperInterface
- Take a look at
Snippetify\SnippetSniffer\Scrapers\StackoverflowScraper
to get you helped - Your fetch method must return an array of
Snippetify\SnippetSniffer\Common\Snippet
- Add it in the scrapers stacks in the
Snippetify\SnippetSniffer\Core.php
- Write tests. Take a look at
Snippetify\SnippetSniffer\Tests\Scrapers\StackoverflowScraperTest
to get you helped - Send a pull request to us
Use your own scrapers
- Your scraper must implement
Snippetify\SnippetSniffer\Scrapers\ScraperInterface
- Take a look at
Snippetify\SnippetSniffer\Scrapers\StackoverflowScraper
to get you helped - Your fetch method must return an array of
Snippetify\SnippetSniffer\Common\Snippet
- Pass your new scraper in the configuration parameter or use the
addScraper
method
Snippet crawler
Snippet crawler allows you to extract all snippets from a website by crawling it.
Configuration reference
Changelog
Please see CHANGELOG for more information what has changed recently.
Testing
You must set the PROVIDER_NAME, PROVIDER_CX, PROVIDER_KEY, CRAWLER_URI, DEFAULT_SCRAPER_URI, STACKOVERFLOW_SCRAPER_URI keys in phpunit.xml file before running tests.
Important: Those links must contains at least one snippet otherwise the tests will failed. The Stackoverflow uri must be a question link with an accepted answer otherwise the tests will failed.
Contributing
Please see CONTRIBUTING for details.
Credits
License
The MIT License (MIT). Please see License File for more information.
All versions of snippet-sniffer with dependencies
guzzlehttp/psr7 Version ^1.4
fabpot/goutte Version ^4.0
spatie/crawler Version dev-master
monolog/monolog Version ^1.12|^2.0
snippetify/programming-languages Version dev-master