Download the PHP package mnb/scraperkit without Composer

On this page you can find all versions of the php package mnb/scraperkit. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.

FAQ

After the download, you have to make one include require_once('vendor/autoload.php');. After that you have to import the classes with use statements.

Example:
If you use only one package a project is not needed. But if you use more then one package, without a project it is not possible to import the classes with use statements.

In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
Some PHP packages are not free to download and because of that hosted in private repositories. In this case some credentials are needed to access such packages. Please use the auth.json textarea to insert credentials, if a package is coming from a private repository. You can look here for more information.

  • Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
  • To use Composer is sometimes complicated. Especially for beginners.
  • Composer needs much resources. Sometimes they are not available on a simple webspace.
  • If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
  • Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Please rate this library. Is it a good library?

Example code of mnb/scraperkit

Informations about the package scraperkit

MNB ScraperKit v1.0.5

MNB ScraperKit is a PHP-first professional crawling and data extraction framework for safe, resumable, pipeline-based web scraping.

v1.0.5 improves common-data extraction validation. It keeps the public v1.0.x feature set stable while reducing false positives in human-name, address/location, and application/registration-number extraction on catalog/listing pages such as publisher journal indexes.

ScraperKit is designed for developers, SEO analysts, research teams, academic metadata collectors, ecommerce monitors, tender/job/government data teams, and server automation users who need safe CLI crawling, bulk jobs, resumable checkpoints, normalized records, validation, transformations, exports, and reports.

Installation

MNB ScraperKit is available on Packagist as the Composer package mnb/scraperkit.

Requirements: PHP >=8.2, Composer, and the PHP extensions listed in composer.json.

Install with:

This is the recommended install method for normal users.

Simple install

Windows CMD

Windows PowerShell

Linux/macOS/Git Bash

Verify installation

Show available commands:

On Windows CMD:

On PowerShell:

Run a basic crawl test:

On Windows CMD:

Use in a PHP file

After installing with Composer, create an index.php file in your project root.

Run it from the command line:

Or place it in your local web root and open it through your browser, for example with XAMPP:

PHP extraction rules example

You can pass extraction rules as the third argument to crawl():

A copy-ready example is available at:

Install a specific version

Install the latest stable 1.x release:

Install an exact release:

Development install from source

Use this when you want to work on MNB ScraperKit itself:

Run readiness checks:

Advanced fallback: install directly from GitHub

Most users should use:

Use the GitHub VCS method only if you need the latest main branch before a tagged release is available on Packagist, or if you are testing unreleased changes.

Windows CMD

Windows PowerShell

Linux/macOS/Git Bash

Source ZIP quick check without Composer dependencies

Source zip users can run basic diagnostics before Composer dependencies are installed. The public binary falls back to the native CLI when Symfony Console is not available:

This fallback is intended for onboarding, source-archive smoke tests, and offline QA examples. Install Composer dependencies for the full Symfony Console experience and third-party integration paths.

Troubleshooting Composer install

Error: Could not find a matching version of package mnb/scraperkit

Try clearing Composer cache and installing again:

Also check that the package is visible to Composer:

If Packagist has not updated yet, open the package page on Packagist and trigger an update, then retry the install.

Error: File "./composer.json" cannot be found

This only applies to the advanced GitHub VCS install method. Create a project first:

Then add the VCS repository and require the development branch.

Error: You have to run this command in interactive mode, or specify at least some data

Use this instead of plain composer init --no-interaction:

Error: Could not open input file: vendor/bin/mnb-scraper

The package was not installed, so the vendor binary was not created. Fix the Composer install first. On Windows after a successful install, prefer:

Quick start

Crawl one page:

Run a small safe crawl:

Run crawl plus professional pipeline:

Run the professional pipeline on an existing crawl JSON file:

What makes it different

Most PHP scraping tools focus on fetching HTML and extracting selectors. MNB ScraperKit is designed around a complete professional crawl flow:

The strongest part of the library is the professional crawl pipeline. It turns crawled pages into structured records with metadata, validation status, quality scoring, deduplication keys, failed URL handling, and export-ready output.

v1.0.5 validation hotfix focus

v1.0.5 improves the generic common-data extractor so broad catalog/index pages do not pollute structured fields. Journal and publication titles remain available as generic names, while person_names is now reserved for likely human names. Address/location extraction now avoids UI/legal text such as privacy statements and requires stronger address/location signals. Application and registration number extraction now requires real identifier structure and no longer treats word fragments such as Applicandae or Applied as numbers.

v1.0.3 patch release focus

v1.0.3 adds practical ML crawl techniques while keeping the package dependency-light and safe by default. The ML layer is deterministic and local: it learns from positive/negative URL examples and human feedback, scores candidate URLs, and creates budgeted adaptive crawl plans with exploration and diversity controls.

Highlights

Complete feature list

This section lists the main functionality available in the current V1.0.3 CLI/library release.

Package and CLI

Production readiness and hardening

Examples:

Backward compatibility policy

Academic publisher metadata crawling

V1.0.3 adds safe publisher metadata workflows for academic journal/article discovery. The default model is metadata only: prefer official APIs, public sitemaps, RSS/Atom feeds, DOI/Crossref-style metadata, and public article landing pages. Do not bypass paywalls, CAPTCHAs, authentication, or access controls.

Commands:

Included catalog:

The normalized article metadata schema covers title, subtitle, authors, DOI, normalized DOI, ISSN/eISSN, journal, publisher, volume, issue, page range, publication date, article type, abstract, URL, HTML/PDF URL, license, open-access marker, source, and quality score.

Enterprise project workspaces and access control

Example:

Security audit and compliance toolkit

Example:

Project templates and preset packs

Distributed workers and Redis queue

Example:

Redis example:

Advanced export connectors

Example:

Advanced browser sessions and authorized login workflows

Example:

Rule builder and auto-profile assistant

Example:

Evaluation, benchmarking, and training data quality

Dataset versioning and annotations

ML-ready intelligence and adaptive crawl techniques

Dashboard and local admin UI

Crawling and HTTP

Safety and responsible crawling

URL filtering and scope control

Encoding and text normalization

Parsing and extraction

Plugin system

Lightweight API and webhooks

Advanced retry, scheduling, and monitoring

Database storage layer

Browser-assisted crawling

Common data profiles

Profile schemas and extractor rules

Bulk jobs and resume

Queue and worker commands

Job manifest

Professional crawl pipeline

Validation

Transformations

Deduplication

Failure handling and retry

Export, reports, and bundles

Source connectors

Optional/future-ready modules

Package direction

Requirements

API and webhook examples

List available API routes:

Generate a Bearer token for the optional API server:

Start the optional lightweight API server on localhost:

Check the health endpoint:

Create a queued job through the API:

Create a local webhook test event without making a network request:

Send a webhook payload to an authorized HTTP/HTTPS endpoint:

Example config/webhooks.json:

Plugin system examples

List discovered plugins:

Show the bundled example plugin:

Validate a plugin before installing or publishing it:

Install a local plugin into storage/plugins/:

Minimal plugin manifest shape:

After a plugin contributes a profile, the profile is available through normal profile commands:

Profile schema and extractor rule examples

List built-in schemas:

Show the ecommerce schema:

Validate a custom schema file:

Test extraction rules against one URL:

Use a profile schema during crawl and pipeline processing:

Simple custom profile schema shape:

Export and report examples

Export pipeline records:

Export failed URLs:

Export validation issues:

Generate a professional crawl summary:

Create a portable project bundle:

Windows CMD and PowerShell crawling helpers

MNB ScraperKit includes Windows-friendly scripts for users who prefer not to type long CLI commands. They are designed for safe, low-rate crawling and local testing.

PowerShell single crawl with dry run:

PowerShell single crawl with pipeline output:

PowerShell bulk crawl from a URL list:

PowerShell URL processing with retry/backoff and Windows method ladder:

PowerShell source discovery before crawling:

CMD single crawl:

CMD bulk crawl:

CMD environment overrides:

See also:

CLI commands

Useful help commands:

Advanced retry, scheduling, and monitoring examples

Create a retry plan from a crawl output or failed URL report:

Retry only failed queue jobs that are safe to retry:

Create a local schedule that enqueues a crawl job every hour:

Use cron or Windows Task Scheduler to run due schedules periodically:

Check queue/schedule/worker health:

Database storage examples

Initialize a default local SQLite database:

Use an explicit SQLite file:

Save crawl output and pipeline records:

Export stored records:

Use MySQL/MariaDB with a PDO DSN:

Database storage is optional. Normal JSON/CSV exports continue to work without SQLite, MySQL, or any database setup.

Source connector examples

Read URLs from a sitemap and save JSON:

Export sitemap URLs as a plain TXT list for bulk crawling:

Read RSS/Atom records:

Read URLs from CSV:

Read URLs from JSON with a dot path:

Read URLs from a generic JSON API:

Export URLs from any connector JSON output:

Use --crawl on connector commands to hand the discovered URLs directly to bulk:crawl:

Professional pipeline

The pipeline converts crawled pages into normalized records that are easier to validate, deduplicate, export, and review.

Record shape:

Supported validation signals include:

Supported transformations include:

Example:

Common data profiles

ScraperKit is built for reusable extraction profiles instead of one-off scraping scripts.

Profile Typical use
academic / journal Authors, editors, affiliations, DOI, ISSN, ISBN, ORCID, publisher, article metadata, journal data, PDF links, submission links, and deadlines
conference Event names, speakers, organizers, venues, dates, registration links, CFP deadlines, and submission details
government / tender Tender numbers, notification numbers, application numbers, deadlines, document links, contacts, addresses, fees, and eligibility
ecommerce Product title, price, currency, SKU, brand, availability, images, ratings, reviews, variants, and structured data
jobs Job title, company, location, salary, experience, skills, apply link, deadline, and recruiter contact
seo Meta title, meta description, canonical URL, robots, schema, Open Graph, Twitter cards, headings, links, and sitemap hints
contact / document Emails, phones, addresses, document URLs, file metadata, and page-level contact information

Browser-assisted crawling examples

Diagnose whether a page likely needs browser fallback:

Use auto fallback during a crawl:

Force browser rendering for one crawl:

Create a queued browser fallback job:

php bin/mnb-scraper job:create --type=crawl https://example.com --profile=ecommerce --browser=auto
php bin/mnb-scraper worker:run --stop-when-empty

Browser mode is optional. Normal crawling works without Panther or Chrome. To enable the Panther adapter locally, install the optional dependency and browser driver support:

composer require symfony/panther

Queue and worker examples

Create a queued sitemap source job:

php bin/mnb-scraper job:create --source=sitemap https://example.com/sitemap.xml --profile=seo

Create a queued CSV source job:

php bin/mnb-scraper job:create --source=csv urls.csv --url-column=url --profile=ecommerce

List queued jobs:

php bin/mnb-scraper job:list

Show one queued job:

php bin/mnb-scraper job:show JOB_ID

Run one queued job manually:

php bin/mnb-scraper job:run JOB_ID

Run one worker pass and exit:

php bin/mnb-scraper worker:once

Run a worker loop for server automation:

php bin/mnb-scraper worker:run --sleep=5 --max-jobs=10 --max-runtime=3600 --memory-limit=256M

Pause, resume, cancel, and retry:

php bin/mnb-scraper job:pause JOB_ID
php bin/mnb-scraper job:resume JOB_ID
php bin/mnb-scraper job:cancel JOB_ID
php bin/mnb-scraper queue:failed
php bin/mnb-scraper queue:retry JOB_ID
php bin/mnb-scraper queue:retry-all

The queue foundation remains local and dependency-free. It is suitable for CMD, PowerShell, cron, Windows Task Scheduler, systemd, and Supervisor. Future versions can add database/Redis queue drivers without changing the crawl/pipeline core.

Job manifest and checkpoint

When --job-dir is used, ScraperKit writes a job-manifest.json file containing:

Bulk and URL-processing checkpoints include these queue groups:

This makes long jobs easier to resume, audit, and troubleshoot.

Useful pacing and safety-related options:

php bin/mnb-scraper crawl "https://example.com" \
  --max-pages=20 \
  --depth=1 \
  --delay-ms=1000 \
  --delay-jitter-ms=300 \
  --pause-after-urls=10 \
  --pause-seconds=30 \
  --cooldown-after-failures=3 \
  --cooldown-seconds=60

Safety defaults

ScraperKit is safe by default:

Use this library for public or authorized crawling, SEO audits, website diagnostics, permitted monitoring, and your own sites. Do not use it for access-control bypass, CAPTCHA bypass, paywall bypass, credential abuse, or aggressive traffic.

Export outputs

ScraperKit focuses on practical export-ready outputs:

PDF reports and richer role-based enterprise orchestration remain future upgrade areas. The current V1.0.3 release already includes CLI workflows, source connectors, exports/reports/bundles, export delivery connectors, local and distributed queue/worker commands, optional Redis queue support, optional browser-assisted crawling, API/webhooks, dashboard UI, ML-ready intelligence, dataset versioning, and annotation tools.

Windows CMD

scripts\run-crawl.cmd https://example.com 10 1
scripts\run-pipeline.cmd storage\jobs\example\crawl.json

PowerShell

.\scripts\run-crawl.ps1 -Url "https://example.com" -MaxPages 10 -Depth 1
.\scripts\run-pipeline.ps1 -Input "storage\jobs\example\crawl.json"

Source connectors

ScraperKit includes source connector commands for API/feed-first workflows:

Release package rules

This V1.0.3 package intentionally keeps documentation simple: README.md is the only project documentation file.

The release package should not include generated runtime files:

The storage/ folder is kept with .gitkeep; generated files are ignored.

Testing

Run:

php tests/run-tests.php

Optional Composer script:

composer test

Evaluation and training data quality examples

Evaluate one dataset:

php bin/mnb-scraper eval:dataset dataset_products --profile=ecommerce --format=html --output=storage/reports/products-evaluation.html

Evaluate pipeline output directly:

php bin/mnb-scraper eval:pipeline storage/jobs/job-001/pipeline/records.json --profile=ecommerce --json

Benchmark a profile against a dataset:

php bin/mnb-scraper benchmark:profile ecommerce --dataset=dataset_products --json

Measure selector/field success for a profile:

php bin/mnb-scraper eval:selectors --profile=ecommerce --dataset=dataset_products --output=storage/reports/selector-report.json

Check annotation coverage and export labels:

php bin/mnb-scraper annotation:stats dataset_products --json
php bin/mnb-scraper annotation:export dataset_products --format=jsonl --output=storage/datasets/products/annotations.jsonl

Export training-ready data:

php bin/mnb-scraper dataset:export dataset_products --format=jsonl --training-ready --training-type=classification

Dataset versioning examples

Create a dataset snapshot from pipeline records:

php bin/mnb-scraper dataset:create storage/jobs/example/pipeline/records.json --id=example_dataset

List and inspect datasets:

php bin/mnb-scraper dataset:list
php bin/mnb-scraper dataset:show example_dataset

Export normalized dataset records:

php bin/mnb-scraper dataset:export example_dataset --format=csv --output=storage/datasets/example_dataset/export.csv

Compare two dataset snapshots:

php bin/mnb-scraper dataset:diff old_dataset new_dataset --json

Initialize annotations and add review labels:

php bin/mnb-scraper annotation:init storage/datasets/example_dataset
php bin/mnb-scraper annotation:add storage/datasets/example_dataset/annotations.json --record-id=dsrec_123 --label=good --note="Ready for training"

ML-ready intelligence and adaptive crawl examples

Analyze crawl or pipeline output and export features:

php bin/mnb-scraper intelligence:analyze storage/jobs/example/crawl.json --output=storage/intelligence/features.json

Classify pages and recommend profiles:

php bin/mnb-scraper intelligence:classify storage/jobs/example/crawl.json --output=storage/intelligence/classes.json

Predict quality for pages and records:

php bin/mnb-scraper intelligence:quality storage/jobs/example/pipeline.json --output=storage/intelligence/quality.json

Prioritize URLs before crawling:

php bin/mnb-scraper intelligence:priority urls.txt --format=txt --output=priority-urls.txt

Suggest selectors from saved HTML:

php bin/mnb-scraper intelligence:selectors page.html --profile=ecommerce --output=selectors.json

List safe ML crawl strategies:

php bin/mnb-scraper ml:strategies --json

Train a lightweight crawl relevance model from reviewed examples:

php bin/mnb-scraper ml:train   --positive=examples/ml/positive-urls.txt   --negative=examples/ml/negative-urls.txt   --output=storage/ml/crawl-model.json

Score candidate URLs before crawling:

php bin/mnb-scraper ml:score examples/ml/candidate-urls.txt   --model=storage/ml/crawl-model.json   --output=storage/ml/scores.json

Create an adaptive crawl plan with crawl budget and exploration control:

php bin/mnb-scraper ml:adaptive-plan examples/ml/candidate-urls.txt   --model=storage/ml/crawl-model.json   --crawl-budget=4   --explore-ratio=0.15   --profile=academic   --output=storage/ml/adaptive-plan.json

Store feedback after reviewing crawl results:

php bin/mnb-scraper ml:feedback https://link.springer.com/article/10.1007/s007770050003   --label=relevant   --reason="article metadata page"

Export ML-ready URL features as JSONL:

php bin/mnb-scraper ml:export-training   --positive=examples/ml/positive-urls.txt   --negative=examples/ml/negative-urls.txt   --format=jsonl   --output=storage/ml/training-data.jsonl

Run the Windows helper:

powershell -ExecutionPolicy Bypass -File scripts/run-ml-adaptive-plan.ps1 -CrawlBudget 10

Project template examples

List available templates:

php bin/mnb-scraper template:list

Create an SEO audit project workspace:

php bin/mnb-scraper template:create seo-audit --output-dir=projects/seo-audit --name=seo-audit

Create an ecommerce monitoring workspace:

php bin/mnb-scraper template:create ecommerce-monitor --output-dir=projects/products --name=products

Install a preset pack with grouped profiles and workflow examples:

php bin/mnb-scraper preset:install commerce-gov-pack --output-dir=presets/commerce-gov

Validate a template before sharing it with a team:

php bin/mnb-scraper template:validate seo-audit

AI crawl intelligence and search discovery

The AI/search layer helps decide what can be crawled safely before launching a crawl. It is designed as a planning and discovery layer, not as a bypass system.

Core principles:

List AI providers:

php bin/mnb-scraper ai:providers --json

Analyze a site or saved target for crawl flexibility:

php bin/mnb-scraper ai:analyze-site https://link.springer.com/journal/777/volumes-and-issues --goal=article_metadata --provider=rule_based --output=storage/ai/springer-analysis.json

Explain the generated crawl plan:

php bin/mnb-scraper ai:explain-plan storage/ai/springer-analysis.json

List search providers:

php bin/mnb-scraper search:providers --json

Run offline/provider-backed discovery and convert results to seed URLs:

php bin/mnb-scraper search:web "Springer journal 777 article DOI" --provider=offline --output=storage/search/results.json
php bin/mnb-scraper search:discover "Springer journal 777 article DOI" --input=examples/search/springer-search-results.json --filter-domain=link.springer.com --output=storage/search/discovered.json
php bin/mnb-scraper search:to-seeds storage/search/discovered.json --filter-domain=link.springer.com --output=storage/search/seeds.txt

Recommended production flow:

search/web discovery -> search result classification -> domain filtering -> AI crawl analysis -> seed review -> low-rate crawl -> extraction recipe -> quality report

Authorized mail and webmail extraction

The mail layer is authorization-first. It is intended for user-approved exports, Gmail API/OAuth connectors, IMAP connectors, or webmail exports. It does not automate hidden login, scrape webmail UIs, store mailbox passwords, or bypass account protections.

Supported provider slots:

local_json
eml_file
gmail_api
imap
webmail_export

List mail providers and safety policy:

php bin/mnb-scraper mail:providers --json

Search an approved local mail export:

php bin/mnb-scraper mail:search "Springer" --provider=local_json --input=examples/mail/sample-authorized-mails.json --json

Extract links, PDF URLs, text, HTML, and attachment metadata:

php bin/mnb-scraper mail:extract examples/mail/sample-authorized-mails.json --extract=links,pdfs,text,attachments --query=Springer --output=storage/mail/extracted.json

Build an attachment manifest or save approved base64 attachment content from a user export:

php bin/mnb-scraper mail:attachments examples/mail/sample-authorized-mails.json --output-dir=storage/mail/attachments --json

Convert extracted mail links into crawl seeds:

php bin/mnb-scraper mail:to-seeds storage/mail/extracted.json --filter-domain=link.springer.com --output=storage/mail/seeds.txt

Recommended production flow:

authorized mailbox/export -> mail extraction -> domain filtering -> seed review -> AI crawl analysis -> extraction recipe -> dataset/export

QA smoke examples

The package includes offline-safe examples so users can verify major workflows without live credentials or crawling external websites.

Useful files:

examples/qa/command-smoke-plan.json
examples/qa/qa-workflow-notes.txt
examples/search/multi-provider-offline-results.json
examples/ai/site-analysis-targets.json
examples/mail/sample-authorized-mails.json
examples/mail/sample-authorized-message.eml
examples/mail/sample-webmail-export.json
examples/extraction/qa-components.html

Run the QA smoke helper:

scripts/run-qa-smoke.cmd

PowerShell:

./scripts/run-qa-smoke.ps1

Run the native source-zip smoke helper when you want to verify the public archive before installing Composer dependencies:

./scripts/run-native-smoke.sh

Windows CMD:

scripts\run-native-smoke.cmd

PowerShell:

./scripts/run-native-smoke.ps1

The smoke plan covers search discovery, AI analysis, mail extraction, extraction options, recipes, quality reports, and seed generation using local example data only.

License

MIT License. See LICENSE.

Dashboard usage

Start the local dashboard server:

php bin/mnb-scraper dashboard:serve

Open:

http://127.0.0.1:8788/dashboard

Build a static dashboard snapshot:

php bin/mnb-scraper dashboard:build --output=storage/dashboard/index.html

Check dashboard status from CLI:

php bin/mnb-scraper dashboard:status --json

Protect the dashboard when exposing it outside localhost:

set MNB_SCRAPERKIT_DASHBOARD_TOKEN=your-token-here
php bin/mnb-scraper dashboard:serve

Then send Authorization: Bearer your-token-here or use ?token=your-token-here for local testing.

Enterprise publisher graph crawling

MNB ScraperKit now models real academic publisher crawling as a metadata-first navigation graph:

publisher/about page -> journal/book indexes -> journal landing pages -> volumes/issues or book chapters -> article/chapter metadata pages -> normalized article records

New commands:

php bin/mnb-scraper publisher:graph springer --json
php bin/mnb-scraper publisher:enterprise-plan springer --max-journals=10 --max-books=10 --max-issues=10 --max-articles=25 --output=storage/springer-enterprise-plan.json
php bin/mnb-scraper publisher:extract-article saved-springer-article.html --publisher=springer --url=https://link.springer.com/article/10.1007/s007770050003 --output=storage/article.json

The publisher graph supports journal/book listing pages, book landing URLs, journal volume/issue tables of contents, article/chapter URLs, and detailed metadata fields such as title, article type, published date, authors, affiliation/contact metadata when public, abstract, DOI, keywords, and references. It remains metadata-only by default and does not include paywall, CAPTCHA, or access-control bypass.

Extraction options and component intelligence

v1.0.3 adds reusable extraction controls for enterprise page data extraction:

Examples:

php bin/mnb-scraper extract:types
php bin/mnb-scraper extract:components examples/extraction/sample-components.html --json
php bin/mnb-scraper extract:pagination https://link.springer.com/journals/a/1 --json
php bin/mnb-scraper extract:pagination examples/extraction/sample-pagination.html --base-url=https://example.com/search --json
php bin/mnb-scraper extract:options examples/extraction/sample-components.html --type=links,text,pagination,components,patterns,dictionary --dictionary=storage/extraction/words.json --output=storage/extraction/result.json
php bin/mnb-scraper extract:patterns examples/extraction/sample-components.html --json
php bin/mnb-scraper extract:mappings records.json --mapping=article --output=storage/mapped-records.json

Pagination extraction output

Use extract:pagination when you want only page navigation controls from one URL or saved HTML file:

php bin/mnb-scraper extract:pagination https://link.springer.com/journals/a/1 --json
php bin/mnb-scraper extract:pagination examples/extraction/sample-pagination.html --base-url=https://example.com/search --json

Typical output includes:

{
  "pagination": {
    "has_pagination": true,
    "primary_pattern": "alphabetical",
    "detected_types": ["alphabetical", "previous_next", "numbered"],
    "summary": {
      "next_url": "https://link.springer.com/journals/a/2",
      "page_numbers": [1, 2],
      "pagination_links_total": 30
    },
    "patterns": [],
    "links": []
  }
}

For Springer A-Z pages, the detector separates alphabet navigation such as A B C ... Z, page links such as 1 2 Next, and API-style URL parameters if present. Infinite scroll and JavaScript-only pagination are reported as static signals; the detector does not execute hidden JavaScript or bypass access controls.

Extraction explainability, recipes, and quality reports

The extraction layer is designed to make output reusable and explainable:

Examples:

php bin/mnb-scraper extract:recipes --json
php bin/mnb-scraper extract:recipe article.html --recipe=config/extraction/recipes/springer-article.json --output=storage/extraction/article-recipe.json
php bin/mnb-scraper extract:quality storage/extraction/article-recipe.json --required-field=title --required-field=doi --output=storage/extraction/article-quality.json

All versions of scraperkit with dependencies

PHP Build Version
Package Version
Requires php Version >=8.2
ext-dom Version *
ext-json Version *
ext-mbstring Version *
symfony/console Version ^6.4 || ^7.0
Composer command for our command line client (download client) This client runs in each environment. You don't need a specific PHP version etc. The first 20 API calls are free. Standard composer command

The package mnb/scraperkit contains the following files

Loading the files please wait ...