PHP code example of sastrawi / tokenizer

1. Go to this page and download the library: Download sastrawi/tokenizer library. Choose the download type require.

2. Extract the ZIP file and open the index.php.

3. Add this code to the index.php.

    
        
<?php
require_once('vendor/autoload.php');

/* Start to develop here. Best regards https://php-download.com/ */

sastrawi / tokenizer example snippets




// demo.php

// include composer autoloader
nizerFactory();
$tokenizer = $tokenizerFactory->createDefaultTokenizer();

$tokens = $tokenizer->tokenize('Saya membeli barang seharga Rp 5.000 di Jl. Prof. Soepomo no. 67.');

var_dump($tokens);

bash
php composer.phar

sh
$ echo Saya sedang belajar NLP Bahasa Indonesia. | php vendor/bin/sastrawi-tokenize