Libraries tagged by tokeniser
hippophp/tokenizer
1101 Downloads
Wrapper around token_get_all
funivan/php-tokenizer
38436 Downloads
Wrapper around token_get_all. Easy to extract and modify php code.
sastrawi/tokenizer
946 Downloads
PHP library that allows you to tokenize Bahasa Indonesia.
vanderlee/comprehend
14 Downloads
Framework for building BNF LR(1) parsers
teepluss/wordbreaker
24 Downloads
Word breaker is Thai wording separator for using with fulltext software.
spec-doc/parser-contracts
44 Downloads
A set of interfaces for the parser for the specificator
rudl/logger
2378 Downloads
Rudl UDP cluster logging library (PSR4 compliant)
phpwatch/phptoken-polyfill
17 Downloads
A polyfill for PHP 8.0's PhpToken class
patchwork/php-parser
26 Downloads
Fast and extensible PHP code analysis and transformation
dbeurive/lexer
2282 Downloads
This project implements a simple lexer.
crazzy501/gpt-3-encoder-php
295 Downloads
PHP BPE Text Encoder/Decoder for GPT-2 / GPT-3. Based on coderevolutionplugins/gpt-3-encoder-php
chrismcmacken/phptools
3558 Downloads
Small utilities that assist in developing applications
robo47/php-manipulator
8 Downloads
Manipulation of php code, automated refactoring based on phps tokenizer
ecomdev/magento-translate-tool
6 Downloads
Magento translation tool that utilizes PHP Tokenizer in order to generate csv files from PHP, XML, PHTML code
yuubit/tokenizer
401 Downloads
PHP Library to tokenize various inputs.