Download the PHP package matchory/elasticsearch without Composer

Laravel Elasticsearch Integration

This is a fork of the excellent library by @basemkhirat, who sadly seems to have abandoned it by now.
As we rely on this library quite heavily, we will attempt to keep it up to date and compatible with newer Laravel and Elasticsearch versions.

Changes in this fork:

• [x] Support for Elasticsearch 7.10 and newer
• [x] Support for PHP 7.3 and newer (PHP 8 included!)
• [x] Broadened support for Laravel libraries, allowing you to use it with almost all versions of Laravel
• [x] Type hints in all supported places, giving confidence in all parameters
• [x] Docblock annotations for advanced autocompletion, extensive inline documentation
• [x] Clean separation of connection management into a ConnectionManager class, while preserving backwards compatibility
• [x] Support for most Eloquent model behaviour (see below)
• [x] Removed dependencies on Laravel internals

If you're interested in contributing, please submit a PR or open an issue!

Features:

• Fluent Elasticsearch query builder with an elegant syntax
• Elasticsearch models inspired by Laravel's Eloquent
• Index management using simple artisan commands
• Limited support for the Lumen framework
• Can be used as a Laravel Scout driver
• Parallel usage of multiple Elasticsearch connections
• Built-in pagination based on Laravel Pagination
• Caching queries using a caching layer based on laravel cache.

Table of Contents

• Requirements
• Installation
  • Install package using composer
    • Laravel Installation
    • Lumen Installation
  • Generic app installation
• Configuration (Laravel & Lumen)
• Artisan commands (Laravel & Lumen)
  • es:indices:list: List all indices on server
  • es:indices:create: Create indices defined in config/es.php
  • es:indices:update: Update indices defined in config/es.php
  • es:indices:drop: Drop index
  • Reindexing data (with zero downtime)
• Usage as a Laravel Scout driver
• Elasticsearch models
  • Index Names
  • Connection Names
  • Mapping type
  • Default Attribute Values
  • Retrieving Models
  • Adding additional constraints
  • Collections
  • Chunking Results
  • Retrieving individual Models
  • Not Found Exceptions
  • Inserting and Updating Models
    • Inserts
    • Updates
    • Examining Attribute Changes
    • Mass Assignment
    • Allowing Mass Assignment
    • Upserts
    • Deleting An Existing Model By Its ID
  • Query Scopes
    • Global Scopes
    • Local Scopes
    • Dynamic Scopes
  • Comparing Models
  • Events
  • Replicating Models
  • Mutators and Casting
• Usage as a query builder
• Releases
• Authors
• Bugs, Suggestions and Contributions
• License

Requirements

• PHP >= 7.3
  See Travis CI Builds.
• laravel/laravel >= 5. or laravel/lumen >= 5. or any other application using composer

Installation

This section describes the installation process for all supported application types.

Install package using composer

Whether you're using Laravel, Lumen or another framework, start by installing the package using composer:

Laravel Installation

If you have package autodiscovery disabled, add the service provider and facade to your config/app.php:

Lastly, publish the service provider to your configuration directory:

Lumen Installation

After installing the package from composer, add package service provider in bootstrap/app.php:

Copy the package config directory at vendor/matchory/elasticsearch/src/config/ to your project root folder alongside with your app/ directory:

If you haven't already, make Lumen work with facades by uncommenting this line in bootstrap/app.php:

If you don't want to enable facades in Lumen, you can access the query builder using app("elasticsearch"):

Generic app installation

You can install package with any composer-based application. While we can't provide general instructions, the following example should give you an idea of how it works:

Configuration (Laravel & Lumen)

After publishing the service provider, a configuration file has been created at config/es.php. Here, you can add one or more Elasticsearch connections, with multiple servers each. Take a look at the following example:

If you'd like to use Elasticsearch with Laravel Scout, you can find the scout specific settings in config/scout.php.

Artisan commands (Laravel & Lumen)

With the artisan commands included with this package, you can create or update settings, mappings and aliases. Note that all commands use the default connection by default. You can change this by passing the --connection <your_connection_name> option.

The following commands are available:

es:indices:list: List all indices on server

es:indices:create: Create indices defined in config/es.php

Note that creating operation skips the index if exists.

es:indices:update: Update indices defined in config/es.php

Note that updating operation updates indices setting, aliases and mapping and doesn't delete the indexed data.

es:indices:drop: Drop index

Be careful when using this command, as you will lose your index data!
Running drop command with --force option will skip all confirmation messages.

Reindexing data (with zero downtime)

First, why reindexing?
Changing index mapping doesn't reflect without data reindexing, otherwise your search results will not work on the right way.
To avoid down time, your application should work with index alias not index name.
The index alias is a constant name that application should work with to avoid change index names.

Assume that we want to change mapping for my_index, this is how to do that:

1. Add alias as example my_index_alias to my_index configuration and make sure your application is working with it.

2. Update index with command:

3. Create a new index as example my_new_index with your new mapping in configuration file.

4. Reindex data from my_index into my_new_index with command:

5. Remove my_index_alias alias from my_index and add it to my_new_index in configuration file and update with command:

Usage as a Laravel Scout driver

First, follow Laravel Scout installation.
All you have to do is updating the following lines in config/scout.php:

Have a look at Laravel Scout documentation, too!

Elasticsearch models

Each index type has a corresponding "Model" which is used to interact with that type. Models allow you to query for data in your types or indices, as well as insert new documents into the type. Elasticsearch Models mimic Eloquent models as closely as possible: You can use model events, route bindings, advanced attribute methods and more. If there is any Eloquent functionality you're missing, open an issue, and we'll be happy to add it!.

Supported features:

• Attributes
• Events
• Route bindings
• Global and Local Query Scopes
• Replicating models

A minimal model might look like this:

Index Names

This model is not specifically bound to any index and will simply use the index configured for the given Elasticsearch connection. To specifically target an index, you may define an index property on the model:

Connection Names

By default, all Elasticsearch models will use the default connection that's configured for your application. If you would like to specify a different connection that should be used when interacting with a particular model, you should define a $connection property on the model:

Mapping type

If you're still using mapping types, you may add a type property to your model to indicate the mapping _type to be used for queries.

Mapping Types are deprecated:
Please note that Elastic has deprecated mapping types and will remove them in the next major release. You should not rely on them to continue working.

Default Attribute Values

By default, a newly instantiated model instance will not contain any attribute values. If you would like to define the default values for some of your model's attributes, you may define an attributes property on your model:

Retrieving Models

Once you have created a model and its associated index type, you are ready to start retrieving data from your index. You can think of your Elasticsearch model as a powerful query builder allowing you to fluently query the index associated with the model. The model's all method will retrieve all the documents from the model's associated Elasticsearch index:

Adding additional constraints

The all method will return all the results in the model's index. However, since each Elasticsearch model serves as a query builder, you may add additional constraints to queries, and then invoke the get() method to retrieve the results:

Collections

As we have seen, Elasticsearch methods like all and get retrieve multiple documents from the index. However, these methods don't return a plain PHP array. Instead, an instance of Matchory\Elasticsearch\Collection is returned.

The Elasticsearch Collection class extends Laravel's base Illuminate\Support\Collection class, which provides a variety of helpful methods for interacting with data collections. For example, the reject method may be used to remove models from a collection based on the results of an invoked closure:

In addition to the methods provided by Laravel's base collection class, the Elasticsearch collection class provides a few extra methods that are specifically intended for interacting with collections of Elasticsearch models:

Result Meta data

Elasticsearch provides a few additional fields in addition to the hits of a query, like the total result amount, or the query execution time. The Elasticsearch collection provides getters for these properties:

Iterating

Since all of Laravel's collections implement PHP's iterable interfaces, you may loop over collections as if they were an array:

Chunking Results

Elasticsearch indices can grow quite huge. Your application may run out of memory if you would attempt to load tens of thousands of Elasticsearch documents via the all or get methods without an upper bound. Therefore, the default amount of documents fetched is set to 10. To change this, use the take method:

Retrieving individual Models

In addition to retrieving all the documents matching a given query, you may also retrieve single documents using the find, first, or firstWhere methods. Instead of returning a collection of models, these methods return a single model instance:

$posts = Post::ofType('news')->get(); php ES::type("my_type")->where("views", ">", 150)->get(); php ES::type("my_type")->where("views", ">=", 150)->get(); php ES::type("my_type")->where("views", "<", 150)->get(); php ES::type("my_type")->where("views", "<=", 150)->get(); php ES::type("my_type")->where("title", "like", "foo")->get(); php ES::type("my_type")->where("hobbies", "exists", true)->get();

or

ES::type("my_type")->whereExists("hobbies", true)->get(); php ES::type("my_type")->whereIn("id", [100, 150])->get(); php ES::type("my_type")->whereBetween("id", 100, 150)->get();

or

ES::type("my_type")->whereBetween("id", [100, 150])->get(); php ES::type("my_type")->whereNot("status", "published")->get();

or

ES::type("my_type")->whereNot("status", "=", "published")->get(); php ES::type("my_type")->whereNot("views", ">", 150)->get(); php ES::type("my_type")->whereNot("views", ">=", 150)->get(); php ES::type("my_type")->whereNot("views", "<", 150)->get(); php ES::type("my_type")->whereNot("views", "<=", 150)->get(); php ES::type("my_type")->whereNot("title", "like", "foo")->get(); php ES::type("my_type")->whereNot("hobbies", "exists", true)->get();

or

ES::type("my_type")->whereExists("hobbies", true)->get(); php ES::type("my_type")->whereNotIn("id", [100, 150])->get(); php ES::type("my_type")->whereNotBetween("id", 100, 150)->get();

or

ES::type("my_type")->whereNotBetween("id", [100, 150])->get(); php ES::type("my_type")->distance("location", ["lat" => -33.8688197, "lon" => 151.20929550000005], "10km")->get();

or

ES::type("my_type")->distance("location", "-33.8688197,151.20929550000005", "10km")->get();

or

ES::type("my_type")->distance("location", [151.20929550000005, -33.8688197], "10km")->get();
php ES::type("my_type")->body([ "query" => [ "bool" => [ "must" => [ [ "match" => [ "address" => "mill" ] ], [ "match" => [ "address" => "lane" ] ] ] ] ] ])->get();

Note that you can mix between query builder and array queries.

The query builder will will be merged with the array query.

ES::type("my_type")->body([ "_source" => ["content"]

"query" => [
     "bool" => [
         "must" => [
             [ "match" => [ "address" => "mill" ] ]
         ]
     ]
],

"sort" => [
    "_score"
]

])->select("name")->orderBy("created_at", "desc")->take(10)->skip(5)->get();

The result query will be

/ Array ( [index] => my_index [type] => my_type [body] => Array ( [_source] => Array ( [0] => content [1] => name ) [query] => Array ( [bool] => Array ( [must] => Array ( [0] => Array ( [match] => Array ( [address] => mill ) ) ) ) ) [sort] => Array ( [0] => _score [1] => Array ( [created_at] => desc ) ) ) [from] => 5 [size] => 10 [client] => Array ( [ignore] => Array ( ) ) ) / php ES::type("my_type")->search("hello")->get();

search with Boost = 2

ES::type("my_type")->search("hello", 2)->get();

search within specific fields with different weights

ES::type("my_type")->search("hello", function($search){ $search->boost(2)->fields(["title" => 2, "content" => 1]) })->get(); php $doc = ES::type("my_type")->highlight("title")->search("hello")->first();

Multiple fields Highlighting is allowed.

$doc = ES::type("my_type")->highlight("title", "content")->search("hello")->first();

Return all highlights as array using $doc->getHighlights() method.

$doc->getHighlights();

Also you can return only highlights of specific field.

$doc->getHighlights("title"); php ES::type("my_type")->search("hello")->first(); php ES::type("my_type")->search("hello")->count(); php

These queries are suitable for large amount of data.

A scrolled search allows you to do an initial search and to keep pulling batches of results

from Elasticsearch until there are no more results left.

It’s a bit like a cursor in a traditional database

$documents = ES::type("my_type")->search("hello") ->scroll("2m") ->take(1000) ->get();

Response will contain a hashed code scroll_id will be used to get the next result by running

$documents = ES::type("my_type")->search("hello") ->scroll("2m") ->scrollID("DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAFMFlJQOEtTdnJIUklhcU1FX2VqS0EwZncAAAAAAAABSxZSUDhLU3ZySFJJYXFNRV9laktBMGZ3AAAAAAAAAU4WUlA4S1N2ckhSSWFxTUVfZWpLQTBmdwAAAAAAAAFPFlJQOEtTdnJIUklhcU1FX2VqS0EwZncAAAAAAAABTRZSUDhLU3ZySFJJYXFNRV9laktBMGZ3") ->get();

And so on ...

Note that you don't need to write the query parameters in every scroll. All you need the scroll_id and query scroll time.

To clear scroll_id

ES::type("my_type")->scrollID("DnF1ZXJ5VGhlbkZldGNoBQAAAAAAAAFMFlJQOEtTdnJIUklhcU1FX2VqS0EwZncAAAAAAAABSxZSUDhLU3ZySFJJYXFNRV9laktBMGZ3AAAAAAAAAU4WUlA4S1N2ckhSSWFxTUVfZWpLQTBmdwAAAAAAAAFPFlJQOEtTdnJIUklhcU1FX2VqS0EwZncAAAAAAAABTRZSUDhLU3ZySFJJYXFNRV9laktBMGZ3") ->clear(); php $documents = ES::type("my_type")->search("hello")->paginate(5);

Getting pagination links

$documents->links();

Bootstrap 4 pagination

$documents->links("bootstrap-4");

Simple bootstrap 4 pagination

$documents->links("simple-bootstrap-4");

Simple pagination

$documents->links("simple-default"); php $documents->count() $documents->currentPage() $documents->firstItem() $documents->hasMorePages() $documents->lastItem() $documents->lastPage() $documents->nextPageUrl() $documents->perPage() $documents->previousPageUrl() $documents->total() $documents->url($page) php ES::type("my_type")->search("hello")->where("views", ">", 150)->toArray(); php ES::type("my_type")->search("hello")->where("views", ">", 150)->response(); php ES::type("my_type")->ignore(404, 500)->id(5)->first(); php ES::type("my_type")->search("hello")->remember(10)->get();

Specify a custom cache key

ES::type("my_type")->search("hello")->remember(10, "last_documents")->get();

Caching using other available driver

ES::type("my_type")->search("hello")->cacheDriver("redis")->remember(10, "last_documents")->get();

Caching with cache key prefix

ES::type("my_type")->search("hello")->cacheDriver("redis")->cachePrefix("docs")->remember(10, "last_documents")->get(); php ES::raw()->search([ "index" => "my_index", "type" => "my_type", "body" => [ "query" => [ "bool" => [ "must" => [ [ "match" => [ "address" => "mill" ] ], [ "match" => [ "address" => "lane" ] ] ] ] ] ] ]); php ES::type("my_type")->id(3)->insert([ "title" => "Test document", "content" => "Sample content" ]);

A new document will be inserted with _id = 3.

[id is optional] if not specified, a unique hash key will be generated.

php

Main query

ES::index("my_index")->type("my_type")->bulk(function ($bulk){

# Sub queries
$bulk->index("my_index_1")->type("my_type_1")->id(10)->insert(["title" => "Test document 1","content" => "Sample content 1"]);
$bulk->index("my_index_2")->id(11)->insert(["title" => "Test document 2","content" => "Sample content 2"]);
$bulk->id(12)->insert(["title" => "Test document 3", "content" => "Sample content 3"]);

});

Notes from the above query:

As index and type names are required for insertion, Index and type names are extendable. This means that:

If index() is not specified in subquery:

-- The builder will get index name from the main query.

-- if index is not specified in main query, the builder will get index name from configuration file.

And

If type() is not specified in subquery:

-- The builder will get type name from the main query.

you can use old bulk code style using multidimensional array of [id => data] pairs

ES::type("my_type")->bulk([

10 => [
    "title" => "Test document 1",
    "content" => "Sample content 1"
],

11 => [
    "title" => "Test document 2",
    "content" => "Sample content 2"
]

]);

The two given documents will be inserted with its associated ids

php ES::type("my_type")->id(3)->update([ "title" => "Test document", "content" => "sample content" ]);

Document has _id = 3 will be updated.

[id is required]

php

Bulk update

ES::type("my_type")->bulk(function ($bulk){ $bulk->id(10)->update(["title" => "Test document 1","content" => "Sample content 1"]); $bulk->id(11)->update(["title" => "Test document 2","content" => "Sample content 2"]); }); php ES::type("my_type")->id(3)->increment("views");

Document has _id = 3 will be incremented by 1.

ES::type("my_type")->id(3)->increment("views", 3);

Document has _id = 3 will be incremented by 3.

[id is required]

php ES::type("my_type")->id(3)->decrement("views");

Document has _id = 3 will be decremented by 1.

ES::type("my_type")->id(3)->decrement("views", 3);

Document has _id = 3 will be decremented by 3.

[id is required]

php

increment field by script

ES::type("my_type")->id(3)->script( "ctx._source.$field += params.count", ["count" => 1] );

add php tag to tags array list

ES::type("my_type")->id(3)->script( "ctx._source.tags.add(params.tag)", ["tag" => "php"] );

delete the doc if the tags field contain mongodb, otherwise it does nothing (noop)

ES::type("my_type")->id(3)->script( "if (ctx._source.tags.contains(params.tag)) { ctx.op = 'delete' } else { ctx.op = 'none' }", ["tag" => "mongodb"] ); php ES::type("my_type")->id(3)->delete();

Document has _id = 3 will be deleted.

[id is required]

php

Bulk delete

ES::type("my_type")->bulk(function ($bulk){ $bulk->id(10)->delete(); $bulk->id(11)->delete(); });

Releases
--------
See the [release page](https://github.com/matchory/elasticsearch/releases).

Authors
-------
[Basem Khirat](http://basemkhirat.com) - [[email protected]](mailto:[email protected]) - [@basemkhirat](https://twitter.com/basemkhirat)  
[Moritz Friedrich](https://www.matchory.com) - [[email protected]](mailto:[email protected])

Bugs, Suggestions and Contributions
-----------------------------------
Thanks to [everyone](https://github.com/basemkhirat/elasticsearch/graphs/contributors) who has contributed to the original project and
[everyone else](https://github.com/matchory/elasticsearch/graphs/contributors) who has contributed to this fork!  
Please use [Github](https://github.com/matchory/elasticsearch) for reporting bugs, and making comments or suggestions.

If you're interested in helping out, the most pressing issues would be modernizing the query builder to provide better support for Elasticsearch features as
well as completing the test suite!

License
-------
MIT

`Have a happy searching..`