Libraries tagged by cl100k_base

rajentrivedi/tokenizer-x

53 Favers
22383 Downloads

TokenizerX calculates required tokens for given prompt

danny50610/bpe-tokeniser

4 Favers
22611 Downloads

PHP port for openai/tiktoken (most)