Download the PHP package displace/ai-toolkit without Composer
On this page you can find all versions of the php package displace/ai-toolkit. It is possible to download/install these versions without Composer. Possible dependencies are resolved automatically.
Download displace/ai-toolkit
More information about displace/ai-toolkit
Files in displace/ai-toolkit
Package ai-toolkit
Short Description Pure-PHP utilities for local-first AI pipelines: text chunkers, packed-vector math, MRL truncation.
License MIT
Homepage https://github.com/DisplaceTech/ai-toolkit
FAQ
Example:
In general, it is recommended to use always a project to download your libraries. In an application normally there is more than one library needed.
- Some hosting areas are not accessible by a terminal or SSH. Then it is not possible to use Composer.
- To use Composer is sometimes complicated. Especially for beginners.
- Composer needs much resources. Sometimes they are not available on a simple webspace.
- If you are using private repositories you don't need to share your credentials. You can set up everything on our site and then you provide a simple download link to your team member.
- Simplify your Composer build process. Use our own command line tool to download the vendor folder as binary. This makes your build process faster and you don't need to expose your credentials for private repositories.
Informations about the package ai-toolkit
ai-toolkit
Pure-PHP utilities for local-first AI pipelines.
Text chunkers, packed-vector math, Matryoshka truncation — boring on purpose.
What is ai-toolkit?
The plumbing every embed-and-search pipeline needs and no one wants to rewrite: split documents into chunks, move vectors around in the packed float32 format, slice Matryoshka embeddings down to size. Pure PHP, no extension required, no framework, no model downloads.
It pairs with ext-infer (embeddings), ext-turbovec (vector search), and the ai-contracts interfaces — but depends on none of them.
Chunkers
Three strategies behind one Chunker interface, sized in characters
(multibyte-aware):
| Chunker | Strategy | Reach for it when |
|---|---|---|
RecursiveCharacterChunker |
Split on paragraphs first, then lines, sentences, words; hard-cut last | Prose, markdown, HTML-stripped text — the default |
SentenceChunker |
Pack whole sentences; never splits mid-sentence; overlap counted in sentences | Transcripts and clean copy where sentence integrity matters |
FixedSizeChunker |
Sliding character window, verbatim slices | Logs, minified or OCR text with no useful structure |
Packed-vector math
Vectors in the Displace stack travel as packed little-endian float32
binary strings — the output of pack('g*', ...$floats), batches by
plain concatenation. Packed is the pure-PHP companion for glue code
and tests:
Matryoshka (MRL) truncation
MRL-trained embedding models (Qwen3-Embedding, ...) pack the most important information into the leading coordinates, so a prefix of the vector is itself a usable embedding. Trade recall for a smaller index by slicing — no re-embedding required:
Only valid for MRL-trained models; truncating an ordinary embedding just throws information away.
On the hot path
Packed is for correctness, not speed — it's pure PHP. When
ext-turbovec is loaded, prefer its native Displace\Vector\Vectors
pack/unpack and let the index do the scoring; when you only need
similarity between a handful of vectors, Packed is plenty.
Deliberately out of scope
Token-based chunking — token counts are model-specific and need a tokenizer; character budgets are model-independent and close enough for retrieval (rule of thumb: ~4 characters per English token) · embedding generation — that's ext-infer or your API client · vector storage and ANN search — that's ext-turbovec or your database · document loaders/parsers (PDF, HTML, ...) — bring your own text · an orchestration framework — chains and agents belong to the frameworks.
License
MIT © 2026 Eric Mann / Displace Technologies
All versions of ai-toolkit with dependencies
ext-mbstring Version *