oo

Connecting to Elasticsearch

After setting up Elasticsearch, connect it to Liferay using the Liferay Connector to Elasticsearch. Connection steps depend on the connector you’re configuring. See Available Liferay Elasticsearch Connectors for reference.

Notable installation and configuration procedure differences are presented here.

Important
  • The Elasticsearch 7 system settings entry and corresponding configuration file are also used for configuring the connection to Elasticsearch 8.
  • Stop each Liferay server node before configuring the connection.

If you’re on Liferay 7.2, skip to Liferay 7.2: Installing Elasticsearch 7 Connector.

Configuring the Connector

You can configure the Elasticsearch connector for Liferay 7.3+ with a configuration file:

com.liferay.portal.search.elasticsearch7.configuration.ElasticsearchConfiguration.config

This same file can be used to connect Liferay 7.3 and 7.4 with Elasticsearch 7 or 8. See the Search Engine Compatibility Matrix to find your available Elasticsearch versions.

To deploy it, place it in [Liferay Home]/osgi/configs/.

For Docker,

docker cp ~/path/to/com.liferay.portal.search.elasticsearch7.configuration.ElasticsearchConfiguration.config [container]:/mnt/liferay/files

Alternatively, you can configure the connector in the user interface. In the Global Menu (Global Menu), go to Control Panel → System Settings and open the Search category. The entry is called Elasticsearch 7.

A simple Liferay 7.3+ connector configuration enables production mode (productionModeEnabled="true") and sets the URL to each Elasticsearch node (networkHostAddresses=["http://es-node:9200"]).

  1. Create the following configuration file:

    com.liferay.portal.search.elasticsearch7.configuration.ElasticsearchConfiguration.config
    
  2. Specify the configuration properties in the .config file. Here’s an example that includes security properties commented out (note that you’d need to use https network host addresses when encryption is enabled):

    # In CE/DXP 7.3+, productionModeEnabled replaces operationMode (deprecated):
    productionModeEnabled=B"true"
    networkHostAddresses=["http://es-node1:9200","http://es-node3:9201","http://es-node3:9202"]
    # In CE/DXP 7.3+ the security settings are included in the ElasticsearchConfiguration
    # In CE/DXP 7.2 the security settings go in com.liferay.portal.search.elasticsearch7.configuration.XPackSecurityConfiguration.config
    # Authentication
    #authenticationEnabled=B"true"
    #username="elastic"
    #password="liferay"
    
    # TLS/SSL
    #networkHostAddresses=["https://es-node1:9200","https://es-node3:9201","https://es-node3:9202"]
    #httpSSLEnabled=B"true"
    #truststoreType="pkcs12"
    #trustStorePath="/PATH/TO/elastic-nodes.p12"
    #trustStorePassword="liferay"
    
    # Highly recommended for all non-prodcution usage (e.g., practice, tests, diagnostics):
    #logExceptionsOnly="false"
    
  3. Place the .config file in your [Liferay Home]/osgi/configs folder.

tip

The connectors contain many configuration settings. See the Elasticsearch Connector Settings for their definitions. Most of the configurations correspond to settings available in Elasticsearch.

To refer to Elasticsearch servers by name, map each Elasticsearch server name to its IP address in your DNS or your Liferay server’s /etc/hosts file.

tip

The network host address format is http[s]://[host name]:[port]. If you’re using a Liferay Docker container, you can use --add-host [host name]:[IP address] options with your docker run command to map a host name to each Elasticsearch server IP address. The port is defined in the Elasticsearch container’s docker run command as the first value of the -p 1234:5678 option (it’s 1234 in this case). If you’re running a local test environment without HTTPS enabled, all the addresses can be http://localhost:port. See Docker’s documentation for more details.

Liferay 7.2: Installing Elasticsearch 7 Connector

Stop the Elasticsearch 6 Connector

On Liferay 7.2, the bundled connector application and APIs are for Elasticsearch 6. These must be disabled before installing the Elasticsearch 7 connector.

  1. Create a file called

    com.liferay.portal.bundle.blacklist.internal.BundleBlacklistConfiguration.config
    
  2. Add this content to the file:

    blacklistBundleSymbolicNames=[ \
    	"com.liferay.portal.search.elasticsearch6.api", \
    	"com.liferay.portal.search.elasticsearch6.impl", \
    	"com.liferay.portal.search.elasticsearch6.spi", \
    	"com.liferay.portal.search.elasticsearch6.xpack.security.impl", \
    	"Liferay Connector to X-Pack Security [Elastic Stack 6.x] - Impl", \
    	"Liferay Enterprise Search Security - Impl.lpkg" \
    ]
    
  3. Place the file in your [Liferay Home]/osgi/configs folder.

    When you start the Liferay server (not yet), Liferay reads this file and blocks the declared bundles from starting.

    tip

    Docker: Liferay Home and other important folders of a Liferay installation are accessed in a Docker container at /mnt/liferay as described here. You can use docker cp /path/to/local/file [container_name]:/mnt/liferay/files/osgi/configs to place configuration files into the container. Later, you can use docker cp to deploy the Liferay Connector to Elasticsearch 7 LPKG file.

Install the Elasticsearch 7 Connector

  1. Download the Liferay Connector to Elasticsearch 7.

    Make sure the connector corresponds to your Elasticsearch version. Note, the client libraries in the connector can be for an older version of Elasticsearch (e.g., 7.17.1) even though the connector application supports a newer version (e.g., 7.18.0). Liferay tests the connector with every minor Elasticsearch version and creates new update connector versions when needed. As always, consult the Search Engine Compatibility Matrix for connector compatibility.

  2. Install the LPKG by placing it in the folder

    [Liferay Home]/deploy
    

    For Docker,

    docker cp ~/path/to/Liferay\ Connector\ to\ Elasticsearch.lpkg [container]:/mnt/liferay/deploy
    

    When you start the Liferay server (not yet), Liferay deploys the LPKG.

You’re ready to configure the connector.

Configure the Connector for Liferay 7.2

  1. Create the following Elasticsearch configuration file:

    com.liferay.portal.search.elasticsearch7.configuration.ElasticsearchConfiguration.config
    
  2. Specify the configuration properties in the .config file. Here’s an example that enables remote operation mode, sets the transport address for each Elasticsearch node, and identifies the connection you’re configuring:

    # com.liferay.portal.search.elasticsearch7.configuration.ElasticsearchConfiguration.config
    operationMode="REMOTE"
    transportAddresses="ip.of.elasticsearch.node:9300"
    # Highly recommended for all non-production usage (e.g., practice, tests, diagnostics):
    #logExceptionsOnly="false"
    
  3. Deploy the configuration by placing the .config file in your [Liferay Home]/osgi/configs folder.

You’re ready to start Liferay.

Start Liferay and Re-Index

If Elasticsearch is installed and running, start Liferay. In the Control Panel, navigate to Configuration → Search and verify the Elasticsearch connection is active.

An active connection is displayed in the Search administrative panel.

Re-index the search indexes and spell check indexes. Invoke both of these actions in the Index Actions tab of Control Panel → Configuration → Search.

On Liferay 7.3+, Re-index the Workflow Metrics indexes from the Workflow Metrics Settings window:

  1. From the Global Menu (Applications Menu) navigate to Applications → Workflow Metrics.

  2. Open the Settings window from the App Options menu (App Options).

  3. Click Reindex All. Repeat this operation for each Virtual Instance in your system.

note

If you use Elasticsearch indexes for primary data storage (storing data not backed by a database) you can bring that data into your new Elasticsearch cluster using the snapshot and restore approach. In 7.2 and 7.3, Liferay’s own Search Tuning indexes for Result Rankings and Synonyms are primary storage indexes. In 7.4, these indexes are backed by database tables.

In Liferay 7.4 U45+/GA45+ and Liferay DXP 7.3 U14+, enhanced logging is enabled so you can know when the re-index starts and finishes for system and company indexes. For example,

2023-01-09 20:44:17.730 INFO  [liferay/background_task-2][ReindexPortalBackgroundTaskExecutor:57] Start reindexing company 0
2023-01-09 20:44:53.677 INFO  [liferay/background_task-2][ReindexPortalBackgroundTaskExecutor:76] Finished reindexing company 0
2023-01-09 20:44:53.677 INFO  [liferay/background_task-2][ReindexPortalBackgroundTaskExecutor:57] Start reindexing company 20097
2023-01-09 20:45:00.986 INFO  [liferay/background_task-2][ReindexPortalBackgroundTaskExecutor:76] Finished reindexing company 20097

Now Liferay is indexing content into your remote Elasticsearch installation.

Available Liferay Elasticsearch Connectors

The bundled connector to Elasticsearch is not always the best choice for your installation. It’s important to understand the differences between the connectors you can use to communicate with Elasticsearch:

Liferay CE/DXP Version Name Availability Communication Protocol Supports Secure Connection Operation Modes
Liferay 7.4, all varieties Liferay Connector to Elasticsearch 7 Bundled HTTP Sidecar / Remote (Production)*
CE 7.3 GA4+, DXP 7.3 GA1+ Liferay (CE) Connector to Elasticsearch 7 Bundled HTTP Sidecar / Remote (Production)*
CE 7.2, DXP 7.2 Liferay Connector (CE) to Elasticsearch 6 Bundled Transport ✔** (requires LES) Embedded / Remote
CE 7.2, DXP 7.2 Liferay Connector (CE) to Elasticsearch 7 (v3.x) Marketplace: CE, DXP Transport Embedded / Remote

* The connector configuration’s Operation Mode (operationMode) setting is deprecated and replaced with Production Mode Enabled (productionModeEnabled).

** Through the Liferay Enterprise Search Security application.

For detailed compatibility information, including the compatible Elasticsearch versions and required patch levels, see the Search Engine Compatibility Matrix.

What’s Next

Now that Liferay is connected to Elasticsearch, you can start using Elasticsearch. For production, you should secure communication between your Liferay and Elasticsearch servers. See Securing Elasticsearch for more information.

Capability:
Deployment Approach: