Libraries tagged by unigram

webrek/tokenizers

0 Favers
0 Downloads

Native byte-level BPE (tiktoken-compatible: cl100k_base/o200k_base) plus WordPiece and Unigram tokenizers for PHP 8.3+, with a process-shared vocab cache. Loads HuggingFace tokenizer.json models and counts Claude/Gemini tokens via their APIs.

Go to Download

marcelorobson001/phpmac-morpho

0 Favers
5 Downloads

Tagger sequential PHP que utiliza BD Sqlite com Unigramas e Bigramas do Corpus Mac-morpho.

Go to Download

marcelorobson001/phpflorestasintatica

0 Favers
5 Downloads

tagger do Corpus Floresta Sintática com Bigramas e Unigramas, construido para taggear frases.

Go to Download