Libraries tagged by tokenize
ltd-beget/sphinx-configuration-tokenizer
16175 Downloads
tokenize sphinx configuration and that's all, folks.
tsukasa/php-tokenizer
21026 Downloads
Polyfill for php extension Tokenizer
ride/lib-tokenizer
6649 Downloads
Tokenizer library of the Ride framework.
phootwork/tokenizer
205970 Downloads
Tokenizer for PHP code
infinityloop-dev/graphpinator-tokenizer
19462 Downloads
Tokenizer subproject for GraPHPinator: GraphQL server implementation for PHP.
funivan/php-tokenizer
38433 Downloads
Wrapper around token_get_all. Easy to extract and modify php code.
artemeon/tokenizer
50578 Downloads
Tokenizer for php
h4kuna/latte-php-tokenizer
11908 Downloads
Transform macro args to array.
simpletools/functionparser
4295 Downloads
Function parser for PHP functions, methods, and closures
hexydec/tokenise
5951 Downloads
A program for performing lexical analysis, written in PHP
guttedgarden/tiktoken
9508 Downloads
PHP 7.4 version of tiktoken
apicart/fql
9349 Downloads
Filter Query Language
udan11/sql-parser
907 Downloads
A validating SQL lexer and parser with a focus on MySQL dialect.
rahul900day/gpt-3-encoder
15814 Downloads
GPT-3-Encoder is a PHP implementation of OpenAI's original python encoder/decoder.
herrera-io/annotations
126838 Downloads
A tokenizer for Doctrine annotations.