Download the PHP package ferry-ai/php-inference without Composer

On this page you can find all versions of the php package ferry-ai/php-inference. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.

FAQ

After the download, you have to make one include require_once('vendor/autoload.php');. After that you have to import the classes with use statements.

Example:
If you use only one package a project is not needed. But if you use more then one package, without a project it is not possible to import the classes with use statements.

In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
Some PHP packages are not free to download and because of that hosted in private repositories. In this case some credentials are needed to access such packages. Please use the auth.json textarea to insert credentials, if a package is coming from a private repository. You can look here for more information.

  • Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
  • To use Composer is sometimes complicated. Especially for beginners.
  • Composer needs much resources. Sometimes they are not available on a simple webspace.
  • If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
  • Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Please rate this library. Is it a good library?

Example code of ferry-ai/php-inference

Informations about the package php-inference

FerryAI — native AI inference for PHP

Run ONNX, GGUF, and RubixML models directly in PHP — no Python, no HTTP microservices, no Docker sidecars. One API, full FFI bridge to native engines. Inference-only. PHP 8.3+.

CI Version PHP Tests

Status: early release (v0.1.1). The public API is stabilizing and may change before 1.0 — pin a version and skim the CHANGELOG when upgrading. Code quality is production-grade (PHPStan level 8, Psalm level 3, 793 tests green on Windows + Linux).

Contents

Quick example

Embeddings & vector search — semantic RAG in 8 lines:

Chat & streaming — local LLM in 3 lines:

Why FerryAI

FerryAI Python sidecar
Deployment One PHP process. composer require Python runtime + HTTP server + process manager
Latency Zero-copy FFI → sub-ms overhead HTTP round-trip per inference
Memory Shared weights across workers (shmop) Duplicated per process
Debugging PHP stack traces, xdebug Cross-process tracing
Structured output JSON Schema → GBNF grammar, guaranteed valid JSON Prompt engineering + regex hacks
Model cache Built-in HuggingFace download + LRU cache + SHA-256 verify Manual pip + custom scripts
Type safety PHPStan level 8 + Psalm level 3 mypy (optional)
Streaming Native PHP Generator + SSE/NDJSON PSR-7 response Flask/FastAPI streaming boilerplate

FerryAI loads native shared libraries (onnxruntime.dll, llama.dll) directly via PHP FFI — the same C APIs that Python uses. No subprocess, no shell_exec, no Python. Tokenizers, vector search and tensor math all run in pure PHP when native equivalents are unavailable.

Backends

Backend Drives Highlights
ONNX Runtime embed() similarity() classify() moderate() Any .onnx model. CPU + CUDA/ROCm/DirectML/OpenVINO GPU. Auto-fallback to CPU when GPU deps are missing. All-MiniLM-L6-v2 → 384d vectors.
llama.cpp chat() stream() streamResponse() Real LLM chat & token-by-token streaming. Runs on CPU and CUDA GPU (Windows + Linux). Samplers: greedy, top-k, top-p, GBNF grammar. JSON Schema → GBNF for guaranteed structured output. ChatFormatter with 5 message templates.
CPU Native predict() + tensor ops Pure-PHP tensor math (matmul, transpose, reshape, slice). Optional RubixML .rbm tabular inference. Always available, no native deps.

LLM in detail

Path Support
AI::chat() / AI::stream() (CPU) ✅ real chat via LlamaBackend + ferry_llama wrapper, Windows and Linux
AI::chat() / AI::stream() (GPU, CUDA) ✅ layer offload via GGML_CUDA=ON build
Safetensors→GGUF models (e.g. Qwen3-0.6B) ✅ one-time convert_hf_to_gguf.py, then native inference
ONNX embeddings (GPU, CUDA) ✅ CUDA provider auto-detected, silent CPU fallback

Configure the wrapper via FERRY_AI_LLAMA_WRAPPER (or FERRY_AI_LLAMA_LIB), add that dir to PATH. Sampling is per-request: temperature: 0 → greedy, > 0 → top-p; force one with ['sampler' => 'top_k'] or supply a ['grammar' => '<gbnf>'] / JSON Schema. Build steps: docs/DOCUMENTATION.md · native/llama-wrapper/README.md. Run: examples/03-chat.php · examples/04-streaming.php · examples/09-grammar.php.

Vector store

Two interchangeable backends behind the same VectorStore contract — pick per environment:

Backend Search Best for
SQLite Brute-force, or native KNN via sqlite-vec (vec0 ANN) when available Dev, demos, embedded, single-file
PostgreSQL + pgvector Native <=> / <-> / <#>, HNSW / IVFFlat indexes Production, large collections, concurrency

SQLite transparently uses sqlite-vec (vec0 virtual tables) for native KNN on PHP 8.4+, and falls back to pure-PHP brute-force otherwise — filters always work. examples/21-postgres-vector.php · examples/23-sqlite-vec.php.

Observability & model pool

Instrumentation lives at the facade layer (backends stay isolated). Off by default — zero overhead when disabled:

AI::warmup([...]) preloads models into a memory-bounded LRU ModelPool; classify() / moderate() / predict() / chat() reuse pooled instances. Opt into cross-worker weight sharing via ext-shmop. Downloads retry transient failures. examples/22-observability.php.

Install

Base requirements: PHP 8.3+, ext-ffi, ext-json, ext-hash, ext-fileinfo.

After install — run the diagnostic to see what's available:

Everything else is optional and on-demand — install only what a feature needs. FerryAI degrades gracefully (pure-PHP fallback or a clear "not available" message) when a native library or model is missing.

Dependencies

What you need for each capability. Full source list with versions: docs/SOURCES.md.

Capability PHP side Native artifact Config
ONNX (embeddings, classification) ext-ffi ONNX Runtime lib FERRY_AI_MODEL_DIR or backends.embedding.model_path
LLM chat / streaming ext-ffi llama.cpp + ferry_llama wrapper FERRY_AI_LLAMA_DIR / FERRY_AI_LLAMA_LIB
GPU (ONNX CUDA / llama.cpp) — CUDA Toolkit + cuDNN for ONNX device: 'cuda' + GPU-enabled build
Vector store (SQLite) ext-pdo_sqlite (bundled) — works out of the box
Vector ANN (sqlite-vec) ext-pdo_sqlite vec0.{dll,so,dylib} FERRY_AI_VEC_EXTENSION_LIB
Vector store (PostgreSQL) ext-pdo_pgsql PostgreSQL + pgvector FERRY_AI_VECTOR_DRIVER=pgsql
Model Hub / HuggingFace ext-curl, ext-zip, ext-sodium — FERRY_AI_MODEL_CACHE
CPU tabular ML (RubixML) rubix/ml (isolated) .rbm estimator FERRY_AI_RUBIXML_AUTOLOAD
Native tokenizer (optional) ext-ffi tokenizers-cpp lib FERRY_AI_TOKENIZERS_LIB
Shared weights (workers) ext-shmop — model_pool.shared_memory=true

GPU setup guide (CUDA/cuDNN/curand/cufft + llama.cpp build): docs/DOCUMENTATION.md → Quick Start → GPU setup. GPU→CPU fallback is automatic and silent.

Capabilities

Inference

Capability Description
AI::embed() / AI::similarity() Text → vector, cosine similarity. 4 pooling strategies (mean, cls, eos, max). Batch embedding.
AI::chat() / AI::stream() LLM chat & token-by-token streaming. Samplers: greedy, top-k, top-p, GBNF grammar.
AI::streamResponse() PSR-7 SSE/NDJSON streaming HTTP response for web apps.
AI::classify() Run classification .onnx models (or CPU-native fallback).
AI::moderate() Content moderation with per-category scores and a flagged boolean.
AI::predict() CPU-native tabular prediction via pure-PHP tensor ops or RubixML .rbm models.

Structured generation

Capability Description
GBNF grammar Constrain LLM output to a formal grammar. Guaranteed valid JSON, enum values, DSLs.
JSON Schema → GBNF Pass a JSON Schema object as grammar — auto-converted to GBNF. No prompt engineering needed.

Vector store

Capability Description
SQLite store CRUD, brute-force KNN, metadata filtering. Optional sqlite-vec (vec0) native ANN.
PostgreSQL + pgvector Native <=> / <-> / <#>, HNSW/IVFFlat indexes. Metadata filtering.
AI::pipeline() Composable Generator-based pipeline with 8 built-in stages: chunk, tokenize, embed, classify, normalize, filter, store, transform.

Model management

Capability Description
Model Hub Download from HuggingFace with progress. SHA-256 + Ed25519 signature verification. LRU cache with size limits.
Model pool Memory-bounded LRU eviction. Opt-in cross-worker weight sharing via ext-shmop.
AI::warmup() Preload models into the pool so first inference is instant.
Auto GPU→CPU fallback ONNX silently retries on CPU when GPU providers are missing (incomplete CUDA installs).

Concurrency

Capability Description
FiberPipeline Pipeline with cooperative concurrency and wall-clock timeout support.
AsyncInference runAsync() / runParallel() — run multiple inferences concurrently via PHP Fibers.

Developer experience

Capability Description
ferry-ai check Full environment diagnostic: PHP, extensions, backends, cache — with --json mode.
ferry-ai models:download / chat CLI model management and single-turn chat.
Pure-PHP tokenizers BPE and WordPiece tokenizers with zero native dependencies.
Pure-PHP tensor math matmul, transpose, reshape, slice — always available.
Framework adapters Thin Laravel ServiceProvider and Symfony Bundle included.

Platform
Windows ✅ Unit + integration (ONNX, llama.cpp, SQLite, PostgreSQL)
Linux ✅ Unit + integration (all backends including CUDA)
macOS ✅ Supported (CI-targeted, not yet in active integration matrix)

Packages

Testing

Examples

examples/ — 26 standalone scripts covering every capability: embedding, tokenizer, chat, streaming, RAG, pipeline, SQLite + sqlite-vec & PostgreSQL/pgvector, grammar-constrained generation, model hub, profiling, async fibers, model pool, observability, retry, CPU tensor math + RubixML, benchmarks, Laravel, Symfony.

Documentation

Start here: docs/DOCUMENTATION.md — definitive single-file reference (architecture, facade API, contracts, GPU setup).

Guides: getting-started · configuration · llama.cpp · vector store · model hub · safetensors → GGUF · streaming · deployment · Symfony · API reference · CHANGELOG

Document Purpose
docs/TECHNICAL_SPECIFICATION.md Architecture
docs/FILE_TREE.md Complete file map
docs/INTERFACE_CONTRACTS.md Interface signatures
docs/SOURCES.md External stack reference
docs/README.md Full navigator

Contributing & license

All versions of php-inference with dependencies

PHP Build Version
Package Version
Requires php Version >=8.3
ext-ffi Version *
ext-fileinfo Version *
ext-hash Version *
ext-json Version *
ankane/onnxruntime Version ^0.3.0
psr/http-factory Version ^1.1
psr/http-message Version ^1.1 || ^2.0
Composer command for our command line client (download client) This client runs in each environment. You don't need a specific PHP version etc. The first 20 API calls are free. Standard composer command

The package ferry-ai/php-inference contains the following files

Loading the files please wait ...