Libraries tagged by tokeniser
instaclick/php-code-sniffer
209792 Downloads
(fork) PHP_CodeSniffer tokenises PHP, JavaScript and CSS files and detects violations of a defined set of coding standards.
hostnet/phpcs-tool
162087 Downloads
PHP_CodeSniffer tokenises PHP, JavaScript and CSS files and detects violations of a defined set of coding standards. Hostnet version
phootwork/tokenizer
216237 Downloads
Tokenizer for PHP code
neuron-core/tokenizer
885 Downloads
High-Performance Tokenizer implementation in PHP
mathematicator-core/tokenizer
5029 Downloads
Math Tokenizer
infinityloop-dev/graphpinator-tokenizer
42678 Downloads
Tokenizer subproject for GraPHPinator: GraphQL server implementation for PHP.
fi1a/tokenizer
16515 Downloads
Лексический анализатор строк (tokenizer). Разбор входной последовательности символов на распознанные группы (лексемы).
artemeon/tokenizer
87248 Downloads
Tokenizer for php
aisk/mistral-tokenizer
17974 Downloads
PHP implementation of Mistral's tokenizers
h4kuna/latte-php-tokenizer
25345 Downloads
Transform macro args to array.
simpletools/functionparser
48043 Downloads
Function parser for PHP functions, methods, and closures
phplrt/lexer-contracts
176684 Downloads
A set of phplrt lexer abstractions
guttedgarden/tiktoken
22898 Downloads
PHP 7.4 version of tiktoken
globus-studio/php-obfuscator
83 Downloads
A token-based PHP source code obfuscator with full support for PHP 8.3 through 8.5.
frolaxhq/laravel-llm-tokenkit
748 Downloads
Token estimation, cost calculation & context-window utilities for LLM-powered Laravel apps