Advanced Configuration of the Liferay Elasticsearch Connector
The Elasticsearch connection is configured using a configuration file or via System Settings.
The Elasticsearch connector has a lot of configuration options out of the box; most Elasticsearch settings can be configured by a similarly or identically named Liferay setting (e.g., httpSSLEnabled). If you need an Elasticsearch setting without a dedicated Liferay setting, add the configuration options you need using the advanced settings. These are most often used for adding or overriding Elasticsearch settings and mappings.
- Adding Index Configurations
- Adding Type Mappings
- Overriding Type Mappings
- Adding Configurations to the Development Mode Elasticsearch
If something is configurable for Elasticsearch, it’s configurable using the Elasticsearch connector.
Adding Settings and Mappings to the Liferay Elasticsearch Connector
Think of the available configuration options as being divided into two groups: the most common ones that are easily configured, and more complex configurations requiring entering YAML or JSON.
Adding Index Configurations
Requires Reindex
The additionalIndexConfigurations configuration defines extra settings (in JSON or YAML) that are applied to each company index (i.e., each Liferay Virtual Instance’s index) when it’s created. For example, you can create custom analyzers and filters using this setting. For a complete list of available settings, see the Elasticsearch reference.
Here’s an example that shows how to configure analysis that can be applied to a field or a dynamic template (see below for an example application to a dynamic template).
{
"analysis": {
"analyzer": {
"kuromoji_liferay_custom": {
"filter": [
"cjk_width",
"kuromoji_baseform",
"pos_filter"
],
"tokenizer": "kuromoji_tokenizer"
}
},
"filter": {
"pos_filter": {
"type": "kuromoji_part_of_speech"
}
}
}
}
Some index settings are given dedicated properties in the Elasticsearch Connector. Do not configure the same property in both the dedicated setting and in additionalIndexConfigurations.
Adding Type Mappings
Requires Reindex
additionalTypeMappings defines extra mappings for indexing data into each company and system indexes (i.e., each Liferay Virtual Instance’s index). These are applied when the index is created. Add the mappings using JSON syntax. For more information see here and here. Use additionalTypeMappings for new field (properties) mappings and new dynamic templates, but don’t try to override existing mappings. If any of the mappings set here overlap with existing mappings, index creation fails. Use overrideTypeMappings to replace default mappings.
As with dynamic templates, you can add sub-field mappings to Liferay’s type mapping. These are referred to as properties in Elasticsearch.
To add a property, use this JSON syntax:
{
"properties": {
"fooName": {
"index": "true",
"store": "true",
"type": "keyword"
}
}
}
For Liferay 7.4 U80 or earlier, you must include the LiferayDocumentType declaration at the beginning of the JSON file.
To see that your additional mappings have been added to the Liferay mappings, use curl to access this URL after saving your additions and reindexing:
curl http://[HOST]:[ES_PORT]/liferay-[COMPANY_ID]/_mapping?pretty
Here’s what it would look like for an Elasticsearch instance running on localhost:9200, with a Liferay Company ID of 20116:
curl http://localhost:9200/liferay-20116/_mapping?pretty
In the above URL, liferay-20116 is the index name. Including it indicates that you want to see the mappings that were used to create the index with that name.
See here for more details on Elasticsearch’s field data types.
The above example shows how a fooName field might be added to Liferay’s type mapping. Because fooName is not an existing property in the mapping, it works fine. If you try to override an existing property mapping, index creation fails. Instead use the overrideTypeMappings setting to override properties in the mapping.
Overriding Type Mappings
Requires Reindex
Use overrideTypeMappings to override Liferay’s default type mappings and exert control over how data is indexed into the company and system indexes. This is an advanced feature that should be used only if strictly necessary. If you set this value, the default mappings in Liferay’s source code (for example, liferay-type-mappings.json) are ignored entirely, so include the whole mappings definition in this property, not only the segment you’re modifying.
To make a modification, find the entire list of the current mappings being used to create the index by navigating to the URL
http://[HOST]:[ES_PORT]/liferay-[COMPANY_ID]/_mapping?pretty
Copy the contents in as the value of this property (either into System Settings or your OSGi configuration file). Leave the opening curly brace {, but delete lines 2 and 3 entirely (the line with the index name and the line with mappings):
"liferay-[COMPANY_ID]": {
"mappings" : {
Then, from the end of the mappings, delete the concluding two curly braces.
}
}
Now modify whatever mappings you’d like. The changes take effect once you save the changes and trigger a reindex from Server Administration.
Here’s a partial example, showing a dynamic template that uses the analysis configuration from additionalIndexConfigurations to analyze all string fields that end with _ja. You’d include this with all the other default mappings, replacing the provided template_ja with this custom one:
{
"dynamic_templates": [
{
"template_ja": {
"mapping": {
"analyzer": "kuromoji_liferay_custom",
"index": "analyzed",
"store": "true",
"term_vector": "with_positions_offsets",
"type": "string"
},
"match": "\\w+_ja\\b|\\w+_ja_[A-Z]{2}\\b",
"match_mapping_type": "string",
"match_pattern": "regex"
}
...
}
]
}
For Liferay 7.4 U80 or earlier, you must include the LiferayDocumentType declaration at the beginning of the JSON file.
Adding Configurations to the Development Mode Elasticsearch
Use the Additional Configurations (additionalConfigurations) field to define extra settings (in YAML) for the embedded or sidecar Elasticsearch instance. This is only useful for testing environments. Any node settings normally set in elasticsearch.yml can be declared here. See the Elasticsearch documentation for a description of all possible node settings.
Multi-line YAML Configurations
If you configure the settings from the last section using an OSGi configuration file, you might find yourself needing to write YAML snippets that span multiple lines. Append each line with \n\, like this:
additionalConfigurations=\
cluster.routing.allocation.disk.threshold_enabled: false\n\
cluster.service.slow_task_logging_threshold: 600s\n\
index.indexing.slowlog.threshold.index.warn: 600s\n\
index.search.slowlog.threshold.fetch.warn: 600s\n\
index.search.slowlog.threshold.query.warn: 600s\n\
monitor.jvm.gc.old.warn: 600s\n\
monitor.jvm.gc.young.warn: 600s
From simple configurations to overriding existing type mappings, Elasticsearch and Liferay’s connector to Elasticsearch are configurable.