Download the PHP package 2dareis2do/newspaper3k-php-wrapper without Composer
On this page you can find all versions of the php package 2dareis2do/newspaper3k-php-wrapper. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download 2dareis2do/newspaper3k-php-wrapper
More information about 2dareis2do/newspaper3k-php-wrapper
Files in 2dareis2do/newspaper3k-php-wrapper
Package newspaper3k-php-wrapper
Short Description PHP wrapper for Newspaper3k Article scraping & curation
License GPL-3.0-or-later
Homepage https://github.com/2dareis2do/newspaper3k-php-wrapper
Informations about the package newspaper3k-php-wrapper
Newspaper3k PHP Wrapper
Simple php wrapper for Newspaper3/4k Article scraping and curation.
Now updated to add support for changing the current working directory, enabling you to customise your curation script per job.
Customising ArticleScraping.py
Here is an custom example of ArticleScraping.py that utilises a Playwright wrapper:
Using Newspaper3kWrapper
In this shortened example we simply pass the current working directory to the Newspaper3kWrapper.
Alternative Article Scraping Script
The path to ArticleScraping.py
can be changed by passing the cwd. Here is an example that uses the Cloudscraper library.
Features
- Multi-threaded article download framework
- News url identification
- Text extraction from html
- Top image extraction from html
- All image extraction from html
- Keyword extraction from text
- Summary extraction from text
- Author extraction from text
- Google trending terms extraction
- Works in 10+ languages (English, Chinese, German, Arabic, ...)
Get it now
Run ✅ pip3 install newspaper3k
✅
NOT ⛔ pip3 install newspaper
⛔
On python3 you must install newspaper3k
, not newspaper
. newspaper
is our python2 library.
Although installing newspaper is simple with pip <http://www.pip-installer.org/>
_, you will
run into fixable issues if you are trying to install on ubuntu.
If you are on Debian / Ubuntu, install using the following:
-
Install
pip3
command needed to installnewspaper3k
package::\$ sudo apt-get install python3-pip
-
Python development version, needed for Python.h::
\$ sudo apt-get install python-dev
-
lxml requirements::
\$ sudo apt-get install libxml2-dev libxslt-dev
-
For PIL to recognize .jpg images::
\$ sudo apt-get install libjpeg-dev zlib1g-dev libpng12-dev
NOTE: If you find problem installing libpng12-dev
, try installing libpng-dev
.
-
Download NLP related corpora::
\$ curl https://raw.githubusercontent.com/codelucas/newspaper/master/download_corpora.py | python3
-
Install the distribution via pip::
\$ pip3 install newspaper3k
If you are on OSX, install using the following, you may use both homebrew or macports:
::
$ brew install libxml2 libxslt
$ brew install libtiff libjpeg webp little-cms2
$ pip3 install newspaper3k
$ curl https://raw.githubusercontent.com/codelucas/newspaper/master/download_corpora.py | python3
Otherwise, install with the following:
NOTE: You will still most likely need to install the following libraries via your package manager
- PIL:
libjpeg-dev
zlib1g-dev
libpng12-dev
- lxml:
libxml2-dev
libxslt-dev
- Python Development version:
python-dev
::
$ pip3 install newspaper3k
$ curl https://raw.githubusercontent.com/codelucas/newspaper/master/download_corpora.py | python3
Installation
1 time download of the sentence tokenizer
After installing the NLTK package, please do install the necessary datasets/models for specific functions to work.
In particular you will need the Punkt Sentence Tokenizer.
e.g.
loads python interpreter:
or