Libraries tagged by tokenzie
guttedgarden/tiktoken
13238 Downloads
PHP 7.4 version of tiktoken
rahul900day/gpt-3-encoder
29309 Downloads
GPT-3-Encoder is a PHP implementation of OpenAI's original python encoder/decoder.
strscan/strscan
132239 Downloads
Simple string tokenizer for lexical scanning operations
tsukasa/php-tokenizer
23001 Downloads
Polyfill for php extension Tokenizer
phootwork/tokenizer
209000 Downloads
Tokenizer for PHP code
mathematicator-core/tokenizer
4663 Downloads
Math Tokenizer
ltd-beget/sphinx-configuration-tokenizer
20510 Downloads
tokenize sphinx configuration and that's all, folks.
fi1a/tokenizer
6320 Downloads
Лексический анализатор строк (tokenizer). Разбор входной последовательности символов на распознанные группы (лексемы).
artemeon/tokenizer
71125 Downloads
Tokenizer for php
kevintweber/html-tokenizer
47572 Downloads
Will tokenize HTML.
h4kuna/latte-php-tokenizer
17527 Downloads
Transform macro args to array.
simpletools/functionparser
19215 Downloads
Function parser for PHP functions, methods, and closures
phplrt/lexer-contracts
144951 Downloads
A set of phplrt lexer abstractions
hexydec/tokenise
8719 Downloads
A program for performing lexical analysis, written in PHP
camspiers/statistical-classifier
36990 Downloads
A PHP implementation of Complement Naive Bayes and SVM statistical classifiers, including a structure for building other classifier, multiple data sources and multiple caching backends