Download the PHP package awesome/crc_fast without Composer

awesome/crc_fast

Fast, hardware-accelerated CRC calculation in PHP for all known CRC-32 and CRC-64 variants using SIMD intrinsics, which can exceed 100GiB/s for CRC-32, and 50GiB/s for CRC-64, on modern systems.

It is much, much faster (crc32, crc32b, and crc32c implementations in PHP, plus adds many more variants (particularly CRC-64/NVME).

The performance gains are especially pronounced on aarch64 (Arm) systems, since PHP doesn't currently use hardware acceleration there.

CRC-64/NVME

CRC-64/NVME is in use in a variety of large-scale and mission-critical systems, software, and hardware, such as:

• The AWS S3 recommended checksum
• The Linux kernel
• The NVMe specification

CRC-32/PHP, one of PHP's special flowers 🌼

hash('crc32') in PHP is not the same as crc32() in PHP, and doesn't match the crc32 definition or output in many other programming languages and implementations (which is typically CRC-32/ISO-HDLC)

Instead, it's actually CRC-32/BZIP2 which is then byte-reversed. This extension provides a CrcFast\CRC_32_PHP algorithm const that performs the same calculation, but accelerated.

Related SIMD-accelerated PHP extensions

• simdutf PHP extension for Unicode validation and transcoding at billions of characters per second using the simdutf project.
• simdjson_plus PHP extension for parsing gigabytes of JSON per second using the simdjson project.

Requirements

Uses the crc_fast Rust package and its C-compatible shared library under the hood, so you'll need to have built and installed it. (See Usage, below).

Changes

See the change log.

Installing

Use Composer to install this library using PIE (note the Requirements above):

If you're using a non-standard installation location for the crc_fast library, you may need to specify where the crc_fast header (in include) and shared library (in lib) can be found:

Building

Like most PHP extensions, you can also build yourself:

or with a custom crc_fast location where the crc_fast header and shared library can be found:

Usage

Examples are for CRC-64/NVME, but you can use any supported algorithm variant.

Calculate CRC-64/NVME checksums:

Calculate CRC-64/NVME checksums with a Digest for intermittent / streaming / etc workloads:

Get a list of supported algorithm variants

Equivalents to PHP functions

crc32()

Calculates CRC-32/ISO-HDLC as an integer.

hash('crc32') 🌼

Calculates CRC-32/PHP as binary or hex.

hash('crc32b')

Calculates CRC-32/ISO-HDLC as binary or hex.

hash('crc32c')

Calculates CRC-32/ISCSI as binary or hex.

IDE Stubs

This extension comes with IDE stubs for use with your favorite development environment.

Tests

See the tests directory for test coverage, which also double as useful examples.

Platform support

This extension has been extensively tested on macOS and Linux, on both aarch64 and x86_64.

At Awesome we use it in production at very large scale on Linux on both Flickr and SmugMug.

This extension is not currently supported on Windows. :(

The underlying crc_fast library (same authors) builds and works on Windows, so this is likely just a build issue with creating a working config.w32 implementation. (I took a quick stab, failed, and moved on since we don't use Windows in production.)

If you want to help, please open a working PR. I'd love to merge it.

Performance

PHP already uses SIMD intrinsics for CRC-32 calculations on x86_64 but not on aarch64. Even on x86_64, this library provides considerable improvements, in addition to supporting many more variants.

Tested using the maximum settings for crc_fast for each platform, using 1MiB random payloads.

CRC-32/ISCSI and CRC-32/ISO-HDLC

CRC-32/PHP 🌼

CRC-64/NVME

Note that PHP has no native equivalent.

License

cfc-fast is dual-licensed under

• Apache 2.0 license (http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (https://opensource.org/licenses/MIT)