Download the PHP package eftec/documentstoreone without Composer

DocumentStoreOne

A document store for PHP that allows multiples concurrencies. It is a minimalist alternative to MongoDB or CouchDB without the overhead of installing a new service.

It also works as a small footprint database.

[]() []() []() []() []() []() []()

• DocumentStoreOne
  • Key features
  • Test
  • Concurrency test
  • Usage
  • Methods
  • Constructor($baseFolder,$collection,$strategy=DocumentStoreOne::DSO_AUTO,$server="",$serializeStrategy = false,$keyEncryption = '')
  • isCollection($collection)
  • collection($collection)
  • autoSerialize($value=true,$strategy='php')
  • createCollection($collection)
  • insertOrUpdate($id,$document,[$tries=-1])
  • insert($id,$document,[$tries=-1])
  • update($id,$document,[$tries=-1])
  • get($id,[$tries=-1],$default=false)
  • getFiltered($id,[$tries=-1],$default=false,$condition=[],$reindex=true)
  • public function appendValue($name,$addValue,$tries=-1)
  • getNextSequence($name="seq",$tries=-1,$init=1,$interval=1,$reserveAdditional=0)
  • getSequencePHP()
  • ifExist($id,[$tries=-1])
  • delete($id,[$tries=-1])
  • select($mask="*")
  • copy($idorigin,$iddestination,[$tries=-1])
  • rename($idorigin,$iddestination,[$tries=-1])
  • fixCast (util class)
  • DocumentStoreOne Fields
  • MapReduce
  • Limits
• Strategy of Serialization
  • NONE
  • PHP
  • PHP_ARRAY
  • JSON_ARRAY and JSON_OBJECT
• Control of Error
• Working with CSV
• Version list
  • Pending

Key features

• Single key based.
• Fast. However, it's not an alternative to a relational database. It's optimized to store a moderated number documents instead of millions of rows.
• Allows multiple concurrences by locking and unlocking a document. If the document is locked then, it retries until the document is unlocked or fails after a number of retries.
• One single class with no dependencies.
• Automatic unlock document locked (by default, every 2 minutes if the file was left locked).
• It could use MapReduce See example

Test

In average, an SMB generates 100 invoices per month. So, let's say that an SMB generates 12000 invoices per decade.

Testing generating 12000 invoices with customer, details (around 1-5 lines per detail) and date on an i7/ssd/16gb/windows 64bits.

• Store 12000 invoices 45.303 seconds (reserving a sequence range)
• Store 12000 invoices 73.203 seconds (reading a sequence for every new invoice)
• Store 12000 invoices 49.0286 seconds (reserving a sequence range and using igbinary)
• Reading all invoices 60.2332 seconds. (only reading)
• MapReduce all invoices per customers 64.0569 seconds.
• MapReduce all invoices per customers 32.9869 seconds (igbinary)
• Reading all invoices from a customer 0.3 seconds. (including render the result, see image)
• Adding a new invoice without recalculating all the MapReduce 0.011 seconds.

Concurrency test

A test with 100 concurrent test (write and read), 10 times.

Usage

Methods

Constructor($baseFolder,$collection,$strategy=DocumentStoreOne::DSO_AUTO,$server="",$serializeStrategy = false,$keyEncryption = '')

It creates the DocumentStoreOne instance.

• $baseFolder: should be a folder
• $collection: (a subfolder) is optional.
• $strategy: It is the strategy used to determine if the file is in use or not.

• $server: It is used by REDIS. You can set the server used by the strategy.
• $serializeStrategy: If false then it does not serialize the information.

Examples:

Benchmark how much time (in seconds) it takes to add 100 inserts.

isCollection($collection)

Returns true if collection is valid (a sub-folder).

collection($collection)

It sets the current collection

This command could be nested.

Note, it doesn't validate if the collection is correct or exists. You must use isCollection() to verify if it's right.

autoSerialize($value=true,$strategy='php')

It sets if we want to auto serialize the information, and we set how it is serialized. You can also set using the constructor.

createCollection($collection)

It creates a collection (a new folder inside the base folder). It returns false if the operation fails; otherwise it returns true

insertOrUpdate($id,$document,[$tries=-1])

inserts a new document (string) in the $id indicated. If the document exists, then it's updated.
$tries indicates the number of tries. The default value is -1 (default number of attempts).

If the document is locked then it retries until it is available or after a "nth" number of tries (by default it's 100 tries that equivalent to 10 seconds)

It's faster than insert or update.

insert($id,$document,[$tries=-1])

Inserts a new document (string) in the $id indicated. If the document exists, then it returns false.
$tries indicates the number of tries. The default value is -1 (default number of attempts).

If the document is locked then it retries until it is available or after a "nth" number of tries (by default it's 100 tries that equivalent to 10 seconds)

update($id,$document,[$tries=-1])

Update a document (string) in the $id indicated. If the document doesn't exist, then it returns false
$tries indicates the number of tries. The default value is -1 (default number of attempts).

If the document is locked then it retries until it is available or after a "nth" number of tries (by default it's 100 tries that equivales to 10 seconds)

get($id,[$tries=-1],$default=false)

It reads the document $id. If the document doesn't exist, or it's unable to read it, then it returns false.
$tries indicates the number of tries. The default value is -1 (default number of attempts).

If the document is locked then it retries until it is available or after a "nth" number of tries (by default it's 100 tries that equivalent to 10 seconds)

getFiltered($id,[$tries=-1],$default=false,$condition=[],$reindex=true)

It reads the document $id filtered. If the document doesn't exist, or it's unable to read it, then it returns false.
$tries indicates the number of tries. The default value is -1 (default number of attempts).

If the document is locked then it retries until it is available or after a "nth" number of tries (by default it's 100 tries that equivalent to 10 seconds)

public function appendValue($name,$addValue,$tries=-1)

It adds a value to a document with name $name. The new value is added, so it avoids to create the whole document. It is useful, for example, for a log file.

a) If the value doesn't exist, then it's created with $addValue. Otherwise, it will return true
b) If the value exists, then $addValue is added, and it'll return true
c) Otherwise, it will return false

getNextSequence($name="seq",$tries=-1,$init=1,$interval=1,$reserveAdditional=0)

It reads or generates a new sequence.

a) If the sequence exists, then it's incremented by $interval and this value is returned.
b) If the sequence doesn't exist, then it's created with $init, and this value is returned. c) If the library is unable to create a sequence, unable to lock or the sequence exists but, it's unable to read, then it returns false

You could peek a sequence with $id=get('genseq_') however it's not recommended.

If the sequence is corrupt then it's reset to $init

If you need to reserve a list of sequences, you could use $reserveAdditional

getSequencePHP()

It returns a unique sequence (64bit integer) based on time, a random value and a serverId.

The chances of collision (a generation of the same value) is 1/4095 (per two operations executed every 0.0001 second).

ifExist($id,[$tries=-1])

It checks if the document $id exists. It returns true if the document exists. Otherwise, it returns false.
$tries indicates the number of tries. The default value is -1 (default number of tries).

The validation only happens if the document is fully unlocked.

If the document is locked then it retries until it is available or after a "nth" number of tries (by default it's 100 tries that equivales to 10 seconds)

delete($id,[$tries=-1])

It deletes the document $id. If the document doesn't exist, or it's unable to delete, then it returns false.
$tries indicates the number of tries. The default value is -1 (default number of tries).

If the document is locked then it retries until it is available or after a "nth" number of tries (by default it's 100 tries that equivales to 10 seconds)

select($mask="*")

It returns all the IDs stored on a collection.

It includes locked documents.

copy($idorigin,$iddestination,[$tries=-1])

Copy the document $idorigin in $iddestination

If the document destination exists then its replaced

rename($idorigin,$iddestination,[$tries=-1])

Rename the document $idorigin as $iddestination

If the document destination exists then the operation fails.

fixCast (util class)

It converts a stdclass to a specific class.

It doesn't work with members that are array of objects. The array is kept as stdclass.

DocumentStoreOne Fields

The next fields are public, and they could be changed during runtime

Example:

MapReduce

It could be done manually. The system allows to store a pre-calculated value that could be easily accesses (instead of read all values).

Let's say the next exercise, we have a list of purchases

John purchased 3 products with the code 33. The products 33 costs $23.3 per unit.

Question, how much every customer paid?.

It's a simple exercise, it's more suitable for a relational database (select * from purchases inner join products). However, if the document is long or complex to store in the database then it's here where a document store shines.

Since it's done on code then it's possible to create a hybrid system (relational database+store+memory cache)

Limits

• Keys should be of the type A-a,0-9. In windows, keys are not case-sensitive.
• The limit of documents that a collection could hold is based on the document system used. NTFS allows 2 million of documents per collection.

Strategy of Serialization

Let's say we want to serialize the next information:

NONE

The values are not serialized, so it is not possible to serialize an object, array or other structure. It only works with strings.

How values are stored

How values are returned

PHP

The serialization of PHP is one of the faster way to serialize and de-serialize, and it always returns the same value with the same structure (classes, array, fields)

However, the value stored could be long.

How the values are stored:

How the values are returned:

PHP_ARRAY

This serialization generates a PHP code. This code is verbose however, it has some nice features:

• It could be cached by PHP's OPcache.
• It's fast to load.

How the values are stored:

How the values are returned:

JSON_ARRAY and JSON_OBJECT

Both methods work with JSON for the serialization and de-serialization but the first on returns always an associative array while the other could return an object (stdClass)

Pro:

• JSON is fast (but not as fast a PHP's serialization)
• JSON is compatible across different platforms.
• JSON uses fewer space than PHP?s serialization.

Cons:

• It is a big slower than PHP's serialization
• The result could vary, and it could return a different structure (objects are always returned as stdClass)

How the values are stored:

How the values are returned:

Control of Error

By default, this library throws errors when an error or exception happens. Some methods allow to avoid throwing errors but most of them could throw an error.

The errors are try/catch catch-ables.

Or you could also use to avoid throwing an exception:

Working with CSV

You can work with CSV as follows:

Version list

• 1.27 2027-07-19
  • Fixed a problem with unlock (folder strategy), when the folder does not exist anymore.
• 1.26 2024-02-13
  • composer.json ig-binary is suggested, not required
• 1.25.1 2023-06-04
  • fixed a bug in the constructor. Now it generates the folders if they don't exist.
• 1.25 2023-06-04
  • added DocumentStoreOne::isRelativePath()
  • now you can specify a relative path in the constructor
• 1.24 2022-06-29
  • deleteCollection() deletes the collection including it's content.
• 1.23 2022-03-20
  • [new] It allows to obtain an instance (if any) of DocumentStoreOne using the static method DocumentStoreOne::instance()
• 1.22.1 2022-03-12
  • getTimeStamp() Fixed: returns a warning if the file does not exist.
• 1.22 2022-03-12
  • added setTimeStamp()
• 1.21 2022-02-07
  • compatibility with PHP 7.2 and higher. This library is not compatible with PHP 5.6 anymore, but you can use an old version of the library.
  • Tested the compatibility with PHP 8.1
  • [added] method noThrowOnError()
• 1.20 2021-12-11
  • add igbinary
• 1.19 2021-12-08
  • [added] more controls over the errors.
• 1.18 2021-12-08
  • [added] csv as serialization strategy
  • Some optimizations
  • Memcache is removed.
• 1.16.2 2020-09-20
  • getTimeStamp() throws an exception when the file doesn't exist. Now it returns false.
• 1.16 2020-09-20
  • new method getTimeStamp()
• 1.15 2020-09-13
  • method get() now unlocks a document correctly (using method php_array)
  • method appendValue() is more efficient with json_object,json_array, and it works with php_array.
  • method appendValue() now generates an array of values.

• 1.14 2020-09-13

  • Fixed composer.json. However, the previous composer.json poisoned installations, so it removed all the previous version from packagist.
  • Maybe you should delete "composer.lock" and the folder vendor\efted\documentstoreone and runs composer update.

    [RuntimeException] Could not load package eftec/documentstoreone in repo.packagist.org: [UnexpectedValueException] Could not parse version constraint ^5.6.: Invalid version string "^5.6."

• 1.13 2020-07-12
  • method appendValue() now serializes information and works with most method but php_array.
• 1.12 2020-04-18
  • method get() has a default value
  • method unlock() removed the argument $forced
  • new method getFiltered()
• 1.11 2019-10-23
  • new method setObjectIndex() It sets the default index field for insertObject() and insertOrUpdateObject()
  • new method insertObject()
  • new method insertOrUpdateObject()
  • method select() now could return a list of indexes of a list of documents
• 1.10 2019-08-30 Some cleaning. Added getSequencePHP() and field nodeId
• 1.9 2019-02-10 Unlock now tries to unlock. Manuallock field is not used anymore.
• 1.8 2018-02-03 field neverLock (for fast access a read only database) also phpunit
• 1.7.3 2018-02-03 Updated composer.json
• 1.7.1 2018-10-20 Removed an incorrect echo on lock()
• 1.7 2018-10-20 Added key encryption (optional)
• 1.6 2018-10-19
• • Reduced the default time from 30 seconds to 10 seconds because usually PHP is configured to a timeout of 30 seconds.
• • Method ifExist locks the resource and never releases. Now it releases as expected.
• 1.5 2018-10-13 Maintenance update. Fixed the automatic strategy
• 1.4 2018-08-26 function rename
• 1.3 2018-08-15 Added strategy of lock.
• 1.2 2018-08-12 Small fixes.
• 1.1 2018-08-12 Changed schema with collection.
• 1.0 2018-08-11 first version

Pending

• Transactional (allows to commit or rollback a multiple step transaction). It's in evaluation.
• Different strategy of lock (folder,redis and apcu)
• Msgpack and igbinary