Libraries tagged by tokenize

Bkash Payment Integration for Laravel.

Go to Download

Library emulating the PHP internal reflection using just the tokenized source code.

Go to Download

A Naive Bayesian classification library for PHP with support for different tokenizers to optimize string classification.

Go to Download

Flexible, strategy-based text chunking for Laravel with support for character, token, sentence, and markdown-aware splitting. Perfect for AI/LLM applications and RAG systems.

Go to Download

A simple library for make Lexer and Parsers to build a language

Go to Download

This project implements a simple lexer.

Go to Download

PHP BPE Text Encoder/Decoder for GPT-2 / GPT-3. Based on coderevolutionplugins/gpt-3-encoder-php

Go to Download

A PHP implementation of Complement Naive Bayes and SVM statistical classifiers, including a structure for building other classifier, multiple data sources and multiple caching backends

Go to Download

Tokenizer of YAML

Go to Download

Hippo is an open-source tool that helps PHP programmers adhere to certain a given convention.

Go to Download

Travers - PHP array traversing via tokenized paths

Go to Download

Custom Bundle Extension (Custom Tokenizers) for Tonics Template System

Go to Download

Modernized, composer compatible version of Tim Whitlock's JTokenizer PHP Javascript parser and tokenizer library.

Go to Download

Polyfill for php extension Tokenizer

Go to Download

A tokenizer for PHP

Go to Download