Download the PHP package michaelalexeevweb/php-json-chunk without Composer

PhpJsonChunk

Read a JSON file bigger than your memory limit, one item at a time. Arrays or objects, at the root or anywhere inside. The fastest of the streaming readers compared here — a third faster than JsonMachine, at every size from ten thousand records to five million — and the only one of them whose conformance is checked against JSONTestSuite.

That is the whole idea, and it is meant literally: a 1.00 GB file of 5 400 000 records reads to the end under a 32 MB memory limit, at a 4 MB peak, in 27.5 s. Memory follows the biggest single element, never the length of the file — see what memory scales with.

Everything below is detail.

Why this one

It is the fastest of them. 100 000 records, same file, same loop — 616 ms against JsonMachine's 937 ms, and 7 252 ms for the slowest in the set. At five million records the gap stops being an abstraction: 30 seconds against 48 seconds, and 6 minutes 11 seconds for the slowest — over the same 528 MB file.

Plain json_decode() is faster still when the file fits in memory — which is the honest first question, and the comparison answers it.

Its answers match PHP's own parser. Checked against JSONTestSuite — the corpus written to break JSON parsers — over 289 documents it is meant to read. Not one verdict differs from json_decode(). No other library in this comparison publishes such a check. See Conformance.

The examples on this page run. Every PHP block below is executed by the test suite, against a document synthesised from the very keyPath it uses. A README that drifts from the code fails the build.

It reads more than a list. A root object streams as key => value; several key paths can be read in one pass; a key containing a dot is reachable; items come back as arrays or stdClass; a string or an open stream reads like a file; and a complaint names the byte it failed at.

• ✅ Stream large JSON arrays and objects in PHP
• ✅ Read item-by-item or chunk-by-chunk
• ✅ Reach nested data via keyPath, with * for "every element"
• ✅ Generators and iterators, so memory stays flat
• ✅ limit and offset without loading the whole dataset
• ✅ Optionally spill chunks to temporary files for very large workloads

Why not json_decode()?

Standard JSON parsing in PHP usually means reading the whole file into memory first and then decoding the whole document. For large JSON files and large datasets, that quickly becomes inefficient or impossible.

PhpJsonChunk solves this by streaming JSON array data and returning items or chunks incrementally.

Comparison

100 000 records, a 10 MB file. Each figure is one complete read — every record walked to the end and handed back as a PHP value, no chunking. One machine, 2026-09-06 — yours will differ, so what matters is the shape, not the milliseconds.

Read the first row before the others. If the file fits in memory, json_decode() is ten to twelve times faster than this library at every size, and you should use it. Streaming buys one thing — memory that does not grow with the file — and it is paid for in time.

The catch is that json_decode() needs about 7× the size of the file. At five million records (528 MB) it wants 3.7 GB and takes 2.9 s; PhpJsonChunk reads the same document at 0.15 MB in 29.7 s. Under the 512 MB limit many deployments run with, json_decode() stops at a file of roughly 70 MB — and then the comparison between streaming readers is the only one left.

Among those, this one is the fastest at every size measured — 34% to 38% quicker than JsonMachine, which is the next one along. It is not the thinnest: crocodile2u holds a fifteenth of the memory. What 0.15 MB buys is a decoded PHP value per item and a key path to reach it.

_{¹ salsify is a SAX parser and is not doing the same work: the listener in the benchmark counts elements without ever building one, so its time is a floor rather than a like-for-like measurement.}

Performance

Ten thousand to five million records, with charts of how memory and time actually scale: BENCHMARKS.md. The short version — memory is flat at 0.15 MB from 10 000 records to 5 000 000, and time is 5.9 ms per thousand records at every size measured.

Run it yourself; that is why the benchmark ships in the repository:

Compared against:

• PhpJsonChunk
• JsonMachine
• crocodile2u/json-streamer
• salsify/json-streaming-parser
• MAXakaWIZARD/JsonCollectionParser
• klkvsk/json-decode-stream

Install

Requirements: PHP 8.1+

Quick start

Stream a large JSON array in chunks:

Stream a nested JSON array by path:

Use * in keyPath to traverse all array items at that level:

What it reads

PhpJsonChunk reads containers — a JSON array or a JSON object:

• a root array like [{"id":1},{"id":2}], streamed item by item
• a root object like {"u1":{...},"u2":{...}}, streamed as key => value
• anything nested, reached by keyPath — data.0.items, or ['a.b'] when a key contains a dot
• * for "every element of this list": key1.*.key2.*.key3

A document that is a single string, number, boolean or null has nothing to stream, and is refused as such.

What memory actually scales with

Not the file — the largest single element. An element is scanned into a string and then decoded, and both are alive at once, so the peak lands at roughly twice the size of the biggest item in the array:

A 20 MB file of ordinary records reads at a 4 MB peak; a 20 MB file that is one enormous record does not. This is a property of reading a JSON array element at a time, not a limit you can raise — if your elements are that big, they are the unit that has to fit in memory.

Where the bytes come from

A path on the filesystem, a string already in memory, or an open stream:

The reader never seeks — it holds one block and a single pushed-back character — so a source that can be read once, in order, is enough. A source names itself in complaints, so a failure is still traceable when there is no path to point at.

A bare php:// or http:// string is still refused: pass it as a StreamSource instead, so it is clear that a stream is what you meant.

(Earlier versions of this file said wrappers were refused because the reader "seeks and re-reads within the file". It never did. The restriction was is_file(), and it is gone.)

What else it does

Objects, not only arrays

A document keyed by id — {"u1": {...}, "u2": {...}} — streams as key => value:

A keyPath may land on an object as well as on an array. Chunked, the names stay with their values.

Key paths as segments

keyPath takes a dotted string or a list of segments. The list form is how a key containing a dot is named — a domain, a version, user.name:

'*' means "every element of this list" in both forms.

Several paths in one pass

The document is read once, not once per path. The key is the path that matched, so a path matching many values appears many times.

Arrays or objects

Object keys stay strings whatever this says: a key is a name, not a value.

Stopping early

A forEach() callback that returns false stops the walk. Anything else — including nothing at all — carries on.

When something is wrong

Complaints name the byte they failed at, and it points at the START of the value that could not be read rather than wherever the scanner stopped:

API overview

count()

Returns how many entries the target container holds — items of an array, or members of an object.

read()

Returns arrays in memory. Good for smaller windows when you still want chunking.

readIterator()

Returns an Iterator of items, or chunks when chunkSize is provided.

readGenerator()

Returns a Generator of items, or chunks when chunkSize is provided. This is the most natural option for streaming large JSON files.

Convenience Methods

getFirst()

Returns the first entry of the target container — an item of an array, or the value of an object's first member.

getLast()

Returns the last entry of the target container. Reading it means walking to the end, which is what streaming costs.

getNth()

Returns the element at a specific 0-based index.

forEach()

Iterates through all elements and executes a callback for each one. Returns the total count processed.

Common options

More usage examples

When to use this library

Use PhpJsonChunk when you need to:

• stream large JSON files in PHP without loading them
• process JSON arrays or objects with generators
• read only a window of data via limit / offset
• reach a nested array or object inside a larger document
• read several parts of one document in a single pass
• keep memory flat regardless of how long the file is

What this library is not

• It is not a general-purpose JSON writer
• It is not a validator you run for its own sake — it refuses malformed input as it reads
• It is focused on reading large JSON documents, not on building them

Conformance

Checked against JSONTestSuite, the corpus written to break JSON parsers — 318 files, of which 236 have a root array and are therefore something this reader is meant to read:

Not one verdict differs from PHP's own parser, and nothing outside that scope — a root object, string or number — is accepted. The corpus is not vendored; re-run it yourself:

The script exits non-zero and names the files if any verdict disagrees.

Test

Performance test

The package includes performance checks for datasets with 10k, 30k, 50k, and 100k records in this format:

Run PHPUnit performance tests manually:

Run the benchmark runner:

If you want to keep generated dataset files in your own directory, pass the optional chunk-temp-dir CLI parameter:

You can also pass it as a separate argument:

If --chunk-temp-dir is not provided, the benchmark uses the system temporary directory and removes generated files automatically.

License

MIT