Libraries tagged by tokenizer
h4kuna/latte-php-tokenizer
17901 Downloads
Transform macro args to array.
simpletools/functionparser
20620 Downloads
Function parser for PHP functions, methods, and closures
phplrt/lexer-contracts
146255 Downloads
A set of phplrt lexer abstractions
hexydec/tokenise
8962 Downloads
A program for performing lexical analysis, written in PHP
camspiers/statistical-classifier
36991 Downloads
A PHP implementation of Complement Naive Bayes and SVM statistical classifiers, including a structure for building other classifier, multiple data sources and multiple caching backends
rahul900day/gpt-3-encoder
29930 Downloads
GPT-3-Encoder is a PHP implementation of OpenAI's original python encoder/decoder.
herrera-io/annotations
127553 Downloads
A tokenizer for Doctrine annotations.
mistralys/jtokenizer
6764 Downloads
Modernized, composer compatible version of Tim Whitlock's JTokenizer PHP Javascript parser and tokenizer library.
ricwein/tokenizer
1129 Downloads
configurable string tokenizer
mathematicator-core/tokenizer
4664 Downloads
Math Tokenizer
tsukasa/php-tokenizer
23085 Downloads
Polyfill for php extension Tokenizer
ride/lib-tokenizer
7489 Downloads
Tokenizer library of the Ride framework.
kerasai/behat-tokenizer
20315 Downloads
Token replacement for Behat test suite.
turanjanin/serbian-language-tools
2050 Downloads
Set of tools for tokenization, transliteration and diacritic restoration of a text written in Serbian language.
tacoberu/bnf
2745 Downloads
Framework for building BNF like parsers.