Libraries tagged by tokeniser
ltd-beget/sphinx-configuration-tokenizer
21169 Downloads
tokenize sphinx configuration and that's all, folks.
infinityloop-dev/graphpinator-tokenizer
34517 Downloads
Tokenizer subproject for GraPHPinator: GraphQL server implementation for PHP.
fi1a/tokenizer
8638 Downloads
Лексический анализатор строк (tokenizer). Разбор входной последовательности символов на распознанные группы (лексемы).
artemeon/tokenizer
73081 Downloads
Tokenizer for php
kevintweber/html-tokenizer
47765 Downloads
Will tokenize HTML.
h4kuna/latte-php-tokenizer
18206 Downloads
Transform macro args to array.
simpletools/functionparser
21241 Downloads
Function parser for PHP functions, methods, and closures
phplrt/lexer-contracts
147013 Downloads
A set of phplrt lexer abstractions
rahul900day/gpt-3-encoder
30148 Downloads
GPT-3-Encoder is a PHP implementation of OpenAI's original python encoder/decoder.
herrera-io/annotations
127564 Downloads
A tokenizer for Doctrine annotations.
mistralys/jtokenizer
6969 Downloads
Modernized, composer compatible version of Tim Whitlock's JTokenizer PHP Javascript parser and tokenizer library.
ze/tokenizer-gpt3
1553 Downloads
PHP package for Byte Pair Encoding (BPE) used by GPT-3.
tsukasa/php-tokenizer
23158 Downloads
Polyfill for php extension Tokenizer
ride/lib-tokenizer
7497 Downloads
Tokenizer library of the Ride framework.
ricwein/tokenizer
1153 Downloads
configurable string tokenizer