Download the PHP package neoslive/hybridsearch without Composer

HybridSearch

HybridSearch is a powerful realtime search engine written in Javascript/AngularJS based on an intelligent indexing mechanism powered by Google Firebase (noSQL). Hybrid stands for the innovative way of streaming search results. Every search request delivers small and preselected data blocks and then they are processed client side for calculation the search result. Whenever data of the source application (Neos CMS, Magento, etc.) was changed, HybridSearch updates the search index und performs a live update of current search results - that's why we call it realtime search engine. The search engine was invented by Michael Egli in 2016 and it's a free open source software. Special thanks to Oliver Nightingale (lunr.js) and Wei Song (elasticlunr.js).

Features

• Search as you type (autocomplete and autocorrect were done in background).
• Realtime binding (search index and results view).
• Analyzes Google Analytics Reports for better semantics (optionally).
• Creates search index from rendered and structured data, not just from raw data.
• No configuration needed while indexing.
• Provides a javascript framework for creating fantastic user experience.
• Minimal, but powerful configurations (relevance boosting, filtering, distincts, facets, etc.)
• High performance, no Server-side utilization while searching and very low cpu usage while indexing.
• Supports up to 100'000 concurrency search request. To reach 100,000 concurrent search requests, you usually first need to accumulate tens of millions of daily actives

Demo / Show case

• https://www.phlu.ch (Neos)

Implementations

HybridSearch comes with default integration for Neos CMS. Other implementations can be done very easy. Default Implementation for Magento is coming soon.

Installation with Neos

Backend

Minimal configuration

composer require neoslive/hybridsearch

Create new a firebase project for free. Then you need a database token. Open your firebase project settings and create new database secret/token (see service accounts > Database secrets).

Add to your flow Settings.yaml (minimal configuration)

Optimal configuration

Example of Settings.yaml

If you are using optimal configuration, then you need also something like following settings.

TypoScript (root.ts)

NodeTypes.yaml

Indexing your data

Run flow command for initial indexing your Neos Site

php ./flow hybridsearch:createfullindex

For better semantic you should use optimal configuration and don't index all of your node types. The indexer is rendering every node as a standalone front-end view and not simple raw data, so initially it takes time. But the result is magic.

If you have set "Realtime: true", then all further changes are done automatically as an intelligent background job asynchronously and without any performance impacts while editing in Neos backend.

It's recommended to execute full index creating from time to time. Just create a cronjob like this one:

1 1 * * 1 FLOW_CONTEXT=Production php -d memory_limit=8096M /home/www-data/flow hybridsearch:createfullindex >/dev/null 2>&1

So, while your index is creating you should not waiting and do the frontend integration.

Front-End

Add the following javascript files to your layout/template.

Create an Angular Controller.

Create an HTML search page.

HybridSearch own JavaScript Framework

HybridSearch provides a javascript framework for creating fantastic user experience. Create robust search-as-you-type, lists, facet-filters, etc. in minutes - and everything is wired to the realtime index without writing one line of code.

Initializing

$HybridsearchObject (define the search request)

This is the main request object, where all methods are called that are defining the search request.

Configuration methods

This methods are influencing the behavior of the search result.

setNodeTypeLabels

For each resulting nodeType the search engine creates a group name by the nodeType. So you can easy create facetted search results, etc.. With the method setNodeTypeLabels you are able to combine several nodeType groups into one.

Arguments

Returns

{HybridsearchObject} the search object

Examples

• setNodeTypeLabels({'vendor-package-contact': 'Persons'},{'vendor-package-customers': 'Persons'}) group contact and customers into one group called "Persons".

setPropertiesBoost

Adjust the relevance score calculation for each nodeType property. Higher values means higher relevance score for matching properties.

Arguments

Returns

{HybridsearchObject} the search object

Examples

• setPropertiesBoost({'vendor-package-contact-name': 50},{'vendor-package-contact-lastname': 50}) query matchings in lastname are higher scored in

setParentNodeTypeBoostFactor

Adjust the relevance score calculation for each node by parent node type. The given boost value is the factor that is applied to the relavence score of each node.

Arguments

Returns

{HybridsearchObject} the search object

Examples

• setParentNodeTypeBoostFactor({'typo3-neos-contentcollection1': 1.8,'typo3-neos-contentcollection2': 1.5}) nodes in parent node contentcollection1 are higher scored than nodes in contentcollection2

Filtering methods

This methods are restricting the search result.

setQuery

Sets a query for the search. If scope is given, then threat the param as reference to the scopes variable and watch automatically for any changes. If param scope is undefined then search for the string itself.

Arguments

Returns

{HybridsearchObject} the search object

setNodeType

Sets the nodeType (example 'vendor-package-contact'). If scope is given, then threat the param as reference to the scopes variable and watch automatically for any changes. If param scope is undefined then search for the string itself. NodeTypes are equivalent to Neos NodeTypes, but all special chars a replaced with '-' and everything is lowercase.

Arguments

Returns

{HybridsearchObject} the search object

addPropertyFilter

Adds a property filter that restrict the search result. If scope is given, then threat the param as reference to the scopes variable and watch automatically for any changes. If param scope is undefined then search for the string itself.

Arguments

Returns

{HybridsearchObject} the search object

Examples

• addPropertyFilter('persons.name',['michael','peter']) finds persons with name 'michael' or 'peter'
• addPropertyFilter('persons.name',{'michael': true,'peter': false}) finds persons with name 'michael' but not 'peter'
• addPropertyFilter('persons.*.email','[email protected]') finds persons with email '[email protected]' in primary or secondary address.

setNodePath

Sets the node path (or uri segment) that is the entry point searching for. If scope is given, then threat the param as reference to the scopes variable and watch automatically for any changes. If param scope is undefined then search for the string itself. NodeTypes are equivalent to Neos NodeTypes, but all special chars a replaced with '-' and everything is lowercase.

Arguments

Returns

{HybridsearchObject} the search object

Examples

• setNodePath('/company/about/') get only search results, that are under the uri path '/company/about/'

Preprocessing methods

This methods are called before the search filters are applied.

addNodesByIdentifier

Add nodes to local search index by given node identifiers (uuid). If the there are no further restrictions (filters), then the added nodes are added to the search result.

Arguments

Returns

{HybridsearchObject} the search object

Examples

• addNodesByIdentifier(['64021902-b7d9-11e6-80f5-76304dec7eb7','57978ed3-6e41-48ce-8421-22d80beacef0']) add the nodes to local search index.

Postprocessing methods

This methods are called after the search result was calculated.

setOrderBy

Sets orderings for the result. You can order the result in template view, but then you cannot use the limit/paginate method. Orderings are configured per nodeType or nodeTypeLabel. Ordering cannot be DESC because its an internal behavior of complex search algorithm.

Arguments

Returns

{HybridsearchObject} the search object

Examples

• setOrderBy({'vendor-package-contact': ['name','lastname']}) sorts result by name and lastname for contact nodes
• setOrderBy({'*': 'title'}) sorts result by title for all other nodes

setGroupedBy

Group the result by given identifiers. With this groupedBy function you can filter out duplicates from the search result. GroupedBy are configured per nodeType or nodeTypeLabel.

Arguments

Returns

{HybridsearchObject} the search object

Examples

• setGroupedBy({'vendor-package-contact': ['name','lastname']}) group result by name and lastname for contact nodes. So only persons with different fullnames were applied to the search result.
• setGroupedBy({'*': 'title'}) group result by title for all other nodes

Runtime methods

This methods are called not more than once while initializing.

$watch

Watching the {HybridsearchResultsObject} for any changes.

Returns

{HybridsearchObject} the search object

Examples

• $watch(function(data) {console.log(data);}) The function argument 'data' contains the current search result.

$bind

Bind the {HybridsearchResultsObject} to given scope variable.

Arguments

Returns

{HybridsearchObject} the search object

Examples

• $bind('result',$scope) The search result is automatically applied to the angular scope variable 'result', that you can use in your controller.

$HybridsearchResultsObject (define the search result)

This is the main result object, where all methods are called that are presenting the search result.

Runtime methods

isLoading

Check if search is in progress. Return false when current search is done.

Returns

{boolean} if search is done or not

Examples

• $scope.status = isLoading() ? 'please wait' : countAll()+' results found'

Counter methods

The counter methods delivers statistics about current search result.

count

Get number of nodes in current result (excluding turbo nodes)

Returns

{Integer} number of nodes excluding turbonodes

Examples

• <div>Results {{result.count()}}</div> where result is scope variable with binding to HybridsearchResultsObject (see $bind)

countAll

Get number of nodes in current result (including turbo nodes)

Returns

{Integer} number of nodes including turbo nodes

countTurboNodes

Get number of turbo nodes nodes in current result

Returns

{Integer} number of turbo nodes

countByNodeType

Get number of nodes in current result by node type

Arguments

Returns

{Integer} number of nodes

countByNodeTypeLabel

Get number of nodes in current result by node type label

Arguments

Returns

{Integer} number of nodes

getDistinctCount

Get number of different values (distinct) of current search result.

Arguments

Returns

{Integer} number of different values

Getter methods

The getter methods delivers the result as collection {HybridsearchResultsNode}.

getNodes

Get result collection (excluding turbo nodes)

Arguments

Returns

{Array} Collection of HybridsearchResultsNode

Examples

• <ul><li data-ng-repeat="node in result.getNodes()">{{node.getProperty('title')}}</li></ul> where result is scope variable with binding to HybridsearchResultsObject (see $bind)

getTurboNodes

Get result collection (only turbo nodes)

Arguments

Returns

{Array} Collection of HybridsearchResultsNode

getNodesByNodeType

Get result collection by node type

Arguments

Returns

{Array} Collection of HybridsearchResultsNode

getNodesByNodeTypeLabel

Get result collection by node type label

Arguments

Returns

{Array} Collection of HybridsearchResultsNode

getDistinct

Get distinct values of given property

Arguments

Returns

{Object.value} Value {Object.count} Number of nodes containing this value.

getGrouped

Get alle nodes from current search result a grouped object.

Returns

{HybridsearchResultsGroupObject} Result groups

Examples

Where result is scope variable with binding to HybridsearchResultsObject (see $bind)

$HybridsearchResultsGroupObject (define a group of result)

This object is representing a grouped result.

count

Get number of items in group object

Returns

{Integer} number of items

getItems

Get group items as array

Returns

{array} collection of {HybridsearchResultsDataObject}

$HybridsearchResultsDataObject (define a set of results)

This object is representing a group of result items.

count

Get number of items in this group

Returns

{Integer} number of items

getLabel

Get groups label

Returns

{String} label of group

getNodes

Get result collection (excluding turbo nodes)

Arguments

Returns

{Array} Collection of HybridsearchResultsNode

getProperty

Get value of first node in the group by given property. This is useful for getting values for grouped objects.

Arguments

Returns

{Mixed} Properties value

$HybridsearchResultsNode (define a single result item/node)

This object is representing a single result item.

Getter methods

The getter methods delivers all informations about the item.

getNodeType

Get the node type.

Returns

{String} node type

getScore

Get the relevance score of the items.

Returns

{integer} relevance score (highest value is highest relevance)

isTurboNode

Check if its a turbo node or a normal node.

Returns

{boolean} true if is a turbo node, false is default node

getBreadcrumb

Get breadcrumb as HTML code (if its a document node).

Returns

{String} HTML

getUrl

Get Url (if its a document node).

Returns

{String} url

getPreview

Get rendered preview of node.

Arguments

Returns

{String} preview content

getSortingIndex

Get sorting index of node.

Returns

{Integer} sorting index value

getProperty

Get value by given property.

Arguments

Returns

{Mixed} Properties value

Examples

• {{node.getProperty('name')}} return the name of mixed node types
• {{node.getProperty('vendor-package-contact-name')}} return only the name of node type vendor-package-contact
• {{node.getProperty('primaryAdress.email')}} return primary email
• {{node.getProperty('*.email')}} return all email addresses

getDocumentNode

Get the nearest document node.

Returns

{HybridsearchResultsNode} HybridsearchResultsNode containing breadcrumb and url

Examples