Download the PHP package spidra/spidra-php without Composer

Spidra PHP SDK

The official PHP SDK for Spidra that allows you to scrape pages, run browser actions, batch-process URLs, and crawl entire sites. All results come back as structured data ready to feed into your pipelines or store directly.

Requirements

• PHP >= 8.1
• Composer

Installation

Get your API key at app.spidra.io under Settings → API Keys.

Quick Start

Table of Contents

• Scraping
  • Basic scrape
  • Structured output with JSON schema
  • Geo-targeted scraping
  • Authenticated pages
  • Browser actions
  • forEach: process every element on a page
  • Manual job control
• Batch Scraping
• Crawling
• Logs
• Usage Statistics
• Error Handling

Scraping

All scrape jobs run asynchronously. The run() method submits a job and polls until it finishes. If you need more control, use submit() and get() directly.

Up to 3 URLs can be passed per request and they are processed in parallel.

Basic scrape

Structured output with JSON schema

When you need a guaranteed shape, pass a schema. The API will enforce the structure and return null for any missing fields rather than hallucinating values.

Geo-targeted scraping

Pass useProxy: true and a proxyCountry code to route the request through a specific country. Useful for geo-restricted content or localized pricing.

Supported country codes include: us, gb, de, fr, jp, au, ca, br, in, nl, sg, es, it, mx, and 40+ more. Use "global" or "eu" for regional routing.

Authenticated pages

Pass cookies as a string to scrape pages that require a login session.

Browser actions

Actions let you interact with the page before the scrape runs. They execute in order, and the scrape happens after all actions complete.

Available actions:

For selector, use a CSS selector. For value, use a plain English description and Spidra will locate the element using AI.

forEach: process every element on a page

forEach finds a set of elements on the page and processes each one individually. It is the right tool when you need to collect data from a list of items, paginate through multiple pages, or click into each item's detail page.

You don't need forEach if the data fits on a single page and is short — a plain prompt is simpler and works just as well.

Use forEach when:

• The list spans multiple pages and you need pagination
• You need to click into each item's detail page (navigate mode)
• You have 20+ items and want per-item AI extraction to stay consistent (itemPrompt)

inline mode

Read each element's content directly without navigating. Best for product cards, search results, table rows.

navigate mode

Follow each element's link to its destination page and capture content there. Best for product listings where the full detail is only on the individual page.

click mode

Click each element, capture the content that appears (a modal, drawer, or expanded section), then move on. Best for hotel room cards, FAQ accordions, or any UI where clicking reveals hidden content.

Pagination

After processing all elements on the current page, follow the next-page link and continue collecting.

maxItems applies across all pages combined. The loop stops when you hit maxItems, run out of elements, or reach maxPages.

Per-element actions

Run additional browser actions on each item after navigating or clicking into it, before the content is captured. Useful for scrolling below the fold or expanding collapsed sections.

itemPrompt vs top-level prompt

Use itemPrompt to extract fields from each item individually. Use the top-level prompt to filter, sort, or reshape the full combined output. They can be used together.

Manual job control

Use submit() and get() when you want to manage polling yourself, or fire-and-forget and check back later.

Job statuses: waiting, active, completed, failed.

Timeout option:

run() accepts a $timeout (seconds) and $pollInterval (seconds) argument:

Batch Scraping

Submit up to 50 URLs in a single request. All URLs are processed in parallel. Each URL is a plain string.

Retry failed items:

Cancel a running batch:

List past batches:

Crawling

Given a starting URL, Spidra discovers pages automatically according to your instruction and extracts structured data from each one.

Submit without waiting:

Get signed download URLs for all crawled pages:

Each page includes html_url and markdown_url pointing to S3-signed URLs that expire after 1 hour.

Re-extract with a new instruction:

Runs a new AI transformation over an existing completed crawl without re-crawling. Charges credits for the transformation only.

Crawl history and stats:

Logs

Scrape logs are stored for every job that runs through the API.

Get a single log with full extraction result:

Usage Statistics

Returns credit and request usage broken down by day or week.

Error Handling

Every API error throws a typed exception. Catch the specific class you care about or fall back to the base SpidraException.

Custom Base URL

Resources

• Spidra Documentation
• Spidra Dashboard
• Report an issue

License

MIT