Libraries tagged by tokeniser

Tokenizer for php

Go to Download

Will tokenize HTML.

Go to Download

Transform macro args to array.

Go to Download

Framework for building BNF like parsers.

Go to Download

Function parser for PHP functions, methods, and closures

Go to Download

A set of phplrt lexer abstractions

Go to Download

PHP 7.4 version of tiktoken

Go to Download

A PHP implementation of Complement Naive Bayes and SVM statistical classifiers, including a structure for building other classifier, multiple data sources and multiple caching backends

Go to Download

Filter Query Language

Go to Download

A validating SQL lexer and parser with a focus on MySQL dialect.

Go to Download

GPT-3-Encoder is a PHP implementation of OpenAI's original python encoder/decoder.

Go to Download

A tokenizer for Doctrine annotations.

Go to Download

Modernized, composer compatible version of Tim Whitlock's JTokenizer PHP Javascript parser and tokenizer library.

Go to Download

PHP_CodeSniffer tokenises PHP, JavaScript and CSS files and detects violations of a defined set of coding standards. Hostnet version

Go to Download

Tokenizer library of the Ride framework.

Go to Download