Libraries tagged by tokenizer
brad-jones/superclosure
236 Downloads
Doing interesting things with closures like serialization.
undercoder/magenizer
3 Downloads
Implements Iterator interface over a String for extract data with an array of tokens. The name comes from Mage and tokenizer :)
thomasbachem/php-short-array-syntax-converter
10785 Downloads
Command-line script to convert PHP's array() syntax to PHP 5.4's short array syntax [] using PHP's built-in tokenizer.
phpbsb/php-short-array-syntax-converter
14 Downloads
Command-line script to convert or revert PHP's array() syntax to PHP 5.4's short array syntax [] using PHP's built-in tokenizer.
nbgrp/singlea-jwt-bundle
13 Downloads
The SingleA tokenizer JWT implementation
kickupx/waiting-for
38 Downloads
WaitingFor based tokenizer
f2/code-parser
3 Downloads
A forgiving code tokenizer for tokenizing most programming languages such as Javascript, TypeScript, Python, C, C++, PHP, Forth and so on. A nice basis for making a compiler or DSL.
squizlabs/php_codesniffer
275708342 Downloads
PHP_CodeSniffer tokenizes PHP, JavaScript and CSS files and detects violations of a defined set of coding standards.
phpcsstandards/php_codesniffer
143360 Downloads
PHP_CodeSniffer tokenizes PHP, JavaScript and CSS files and detects violations of a defined set of coding standards.
imanghafoori/laravel-tokenize-login
64527 Downloads
Helps you generate and send a random token to have passwordless login.
mehrab-wj/tiktoken-php
18292 Downloads
a clone of python tiktoken but for PHP! fast BPE tokeniser for use with OpenAI's models.
karim007/laravel-bkash-tokenize
4964 Downloads
This is bKash tokenize payment gateway for laravel
xp-framework/tokenize
91897 Downloads
Text tokenizing
muffin/tokenize
14695 Downloads
Security Tokens
mikefrancis/tokenizr
861 Downloads
PHP library to generate unique, URL friendly tokens