Libraries tagged by tokenize
rahul900day/gpt-3-encoder
27880 Downloads
GPT-3-Encoder is a PHP implementation of OpenAI's original python encoder/decoder.
strscan/strscan
131392 Downloads
Simple string tokenizer for lexical scanning operations
ltd-beget/sphinx-configuration-tokenizer
20131 Downloads
tokenize sphinx configuration and that's all, folks.
tsukasa/php-tokenizer
22882 Downloads
Polyfill for php extension Tokenizer
phootwork/tokenizer
208742 Downloads
Tokenizer for PHP code
mathematicator-core/tokenizer
4573 Downloads
Math Tokenizer
artemeon/tokenizer
70110 Downloads
Tokenizer for php
kevintweber/html-tokenizer
47404 Downloads
Will tokenize HTML.
h4kuna/latte-php-tokenizer
17108 Downloads
Transform macro args to array.
simpletools/functionparser
17864 Downloads
Function parser for PHP functions, methods, and closures
phplrt/lexer-contracts
143875 Downloads
A set of phplrt lexer abstractions
hexydec/tokenise
8503 Downloads
A program for performing lexical analysis, written in PHP
herrera-io/annotations
127455 Downloads
A tokenizer for Doctrine annotations.
mwstake/mediawiki-lib-nodes
19326 Downloads
Provides an infrastructure to tokenize content into nodes
mistralys/jtokenizer
6412 Downloads
Modernized, composer compatible version of Tim Whitlock's JTokenizer PHP Javascript parser and tokenizer library.