Submit Feedback

Using the Sidecar or Embedded Elasticsearch

The Liferay Tomcat bundles installed via a ZIP file or Docker image include an Elasticsearch node that starts with Liferay. The node in Liferay 7.3+ and 7.4+ runs in a separate JVM as a sidecar server and the node in Liferay 7.2 is embedded in the Liferay server.

If you’re running Liferay locally, the Elasticsearch server is accessible at these URLs:

http://localhost:9201 on Liferay 7.3+ and 7.4+
http://localhost:9200 on Liferay 7.2

Here’s example sidecar server output:

{
  "name" : "liferay",
  "cluster_name" : "LiferayElasticsearchCluster",
  "cluster_uuid" : "aDsZ5VWTQ96-lT-7WmcoHg",
  "version" : {
    "number" : "[VERSION]",
    "build_flavor" : "unknown",
    "build_type" : "unknown",
    "build_hash" : "e3b0c3d3c5c130e1dc6d567d6baef1c73eeb2059",
    "build_date" : "2023-07-20T05:33:33.690180787Z",
    "build_snapshot" : false,
    "lucene_version" : "8.11.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

In addition, the Liferay log prints information about the running sidecar server. For example,

2024-08-28 14:27:30.450 INFO  [SystemExecutorServiceUtil-12][Sidecar:100] Sidecar Elasticsearch [VERSION] liferay_sidecar started at 127.0.0.1:9201

The sidecar version is updated regularly. To check the version without a running Liferay, see the sidecar.version file in the Liferay source code. Replace the git tag (i.e., 7.4.3.120-ga120) to match your version.

While the bundled Elasticsearch servers are convenient for development and testing, neither is suitable for production.

Note

While it’s not a supported production configuration, installing Kibana to monitor the bundled Elasticsearch server is useful during development and testing.

On Liferay 7.3, install the OSS only Kibana build.
On Liferay DXP 7.4 U17+ and Liferay Portal 7.4 GA17+, install the free version of Kibana.

See the troubleshooting section if you encounter errors with the setup.

You wouldn’t run an embedded database like HSQL in production, and you shouldn’t run the bundled Elasticsearch server in production either. Instead, run Elasticsearch in remote mode, as a standalone server or cluster of server nodes.

Important

The search tuning apps Synonym Sets and Result Rankings used the search index for primary data storage on Liferay 7.2 and 7.3. No data for these apps was stored in the Liferay database. Therefore, if you have Synonym Sets or Result Rankings configured while using the sidecar or embedded Elasticsearch in Liferay 7.2 and 7.3, switching to a remote Elasticsearch server and reindexing does not restore those configurations. Instead you must manually bring the Synonym Sets and Result Rankings into the remote Elasticsearch cluster. See the Upgrade Guide for details on using Elastic’s Snapshot and Restore feature to preserve these indexes.

Bundled Elasticsearch Server Use cases

Here are common uses for the default Elasticsearch server (sidecar and embedded):

Testing your custom search and indexing code
Developing search queries by running queries directly on Elasticsearch through Kibana
Testing the search tuning functionality
Exploring and configuring the search widgets

App Server Differences

An Elasticsearch sidecar server is included with the Tomcat bundles and Docker images in Liferay 7.4 (DXP and Portal), Liferay DXP 7.3, and Liferay Portal 7.3 GA4+. There are some key differences if you’re installing the Liferay WAR onto any supported application server.

In the table below, the notation 7.3+ includes Liferay 7.4 (DXP and Portal).

Liferay DXP Flavor	Default Elasticsearch	Pre-Installed	Requires Manual Intervention
Tomcat bundle: 7.3+	Sidecar	✔	✘
Tomcat: 7.3+	Sidecar	✘	✘ (auto-downloaded)
Docker: 7.3+	Sidecar	✔	✘
JBoss: 7.3+	Sidecar	✘	✘ (auto-downloaded)
Wildfly: 7.3+	Sidecar	✘	✘ (auto-downloaded)
WebSphere: 7.3+	Sidecar	✘	✔
Weblogic: 7.3+	Sidecar	✘	✔
All flavors: 7.2/7.3 GA3-	Embedded	✔	✘

If you downloaded a bundle for an application server besides Tomcat, when you start the server an Elasticsearch distribution is downloaded on-the-fly and started as a sidecar server.

Installation instructions for Liferay on the WebSphere and Weblogic application servers include directions for manually providing the Elasticsearch archives required for the sidecar server to be initialized.

Important

The bundled Elasticsearch server is useful for development and testing purposes and must not be used in production. See Installing Elasticsearch to learn about installing a remote search engine.

Embedded Versus Sidecar

Here’s a comparison between the embedded and sidecar Elasticsearch servers.

EMBEDDED	SIDECAR
Cannot configure a secure connection	Cannot configure a secure connection
Uses an OSS version of Elasticsearch	Liferay 7.3 uses an OSS version of Elasticsearch Liferay DXP 7.4 U17+ and Liferay Portal 7.4 GA17+ no longer use the OSS Elasticsearch
Runs at http://localhost:9200	Runs at http://localhost:9201
Pre-Installed on all Liferay distributions	Not Always Pre-Installed
Not supported for production	Not supported for production
No special steps required for any app server	Some app servers require additional steps

Troubleshooting the Sidecar/Embedded Elasticsearch

These errors indicate problems with the Elasticsearch Sidecar installation at startup:

2022-03-02 17:32:22.952 ERROR [main][ElasticsearchConnectionManager:81] Elasticsearch sidecar could not be started. Search will be unavailable. Manual installation of Elasticsearch and activation of remote mode is recommended.

2022-03-04 10:41:13.249 ERROR [com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarMainProcessCallable@3da0921a[-Xms1g -Xmx1g -XX:+AlwaysPreTouch -Des.path.conf=/home/tibusz/liferay/bundles/master/tomcat-9.0.56/temp/sidecar5219314832153809763/config -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dio.netty.allocator.type=unpooled -Dio.netty.allocator.numDirectArenas=0 -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/home/tibusz/liferay/bundles/master/tomcat-9.0.56/temp/sidecar5219314832153809763 -Djava.security.policy=jar:file:/home/tibusz/liferay/bundles/master/osgi/state/org.eclipse.osgi/257/0/bundleFile!/META-INF/sidecar.policy -Djna.nosys=true]-][Sidecar:198] Abort subprocess piping
java.io.InvalidClassException: org.elasticsearch.ElasticsearchException; local class incompatible: stream classdesc serialVersionUID = 4568865521165719982, local class serialVersionUID = -7656133510021522625
	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:689) ~[?:?]

2022-03-04 10:41:30.191 ERROR [main][SidecarManager:93] bundle com.liferay.portal.search.elasticsearch7.impl:6.0.34 (257)[com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarManager(973)] : The activate method has thrown an exception
java.util.concurrent.CancellationException: null
	at java.util.concurrent.FutureTask.report(FutureTask.java:121) ~[?:?]
	at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
	at com.liferay.portal.search.elasticsearch7.internal.sidecar.Sidecar._waitForPublishedAddress(Sidecar.java:591) ~[?:?]

2022-03-04 10:41:30.162 WARN  [com.liferay.portal.search.elasticsearch7.internal.sidecar.SidecarMainProcessCallable@3da0921a[-Xms1g -Xmx1g -XX:+AlwaysPreTouch -Des.path.conf=/home/tibusz/liferay/bundles/master/tomcat-9.0.56/temp/sidecar5219314832153809763/config -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Dio.netty.allocator.type=unpooled -Dio.netty.allocator.numDirectArenas=0 -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/home/tibusz/liferay/bundles/master/tomcat-9.0.56/temp/sidecar5219314832153809763 -Djava.security.policy=jar:file:/home/tibusz/liferay/bundles/master/osgi/state/org.eclipse.osgi/257/0/bundleFile!/META-INF/sidecar.policy -Djna.nosys=true]-][Sidecar:632] Sidecar Elasticsearch process is aborted
java.util.concurrent.ExecutionException: com.liferay.petra.process.TerminationProcessException: Subprocess terminated with exit code 130
	at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[?:?]
	at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:?]
	at com.liferay.portal.search.elasticsearch7.internal.sidecar.Sidecar$RestartFutureListener.complete(Sidecar.java:628) [bundleFile:?]
	at com.liferay.petra.concurrent.DefaultNoticeableFuture$OnceFutureListener.complete(DefaultNoticeableFuture.java:111) [com.liferay.petra.concurrent.jar:?]
	at com.liferay.petra.concurrent.DefaultNoticeableFuture.done(DefaultNoticeableFuture.java:85) [com.liferay.petra.concurrent.jar:?]
	at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:381) [?:?]
	at java.util.concurrent.FutureTask.setException(FutureTask.java:250) [?:?]
	at com.liferay.petra.concurrent.DefaultNoticeableFuture.setException(DefaultNoticeableFuture.java:79) [com.liferay.petra.concurrent.jar:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:269) [?:?]
	at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: com.liferay.petra.process.TerminationProcessException: Subprocess terminated with exit code 130
	at com.liferay.petra.process.local.LocalProcessExecutor$SubprocessReactor.call(LocalProcessExecutor.java:309) ~[com.liferay.petra.process.jar:?]
	at com.liferay.petra.process.local.LocalProcessExecutor$SubprocessReactor.call(LocalProcessExecutor.java:161) ~[com.liferay.petra.process.jar:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]

Caused by: org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: Could not load codec 'Lucene87'.  Did you forget to add lucene-backward-codecs.jar?

To resolve errors like these,

Stop Liferay.
Delete [Liferay Home]/elasticsearch-sidecar/ or [Liferay Home]/elasticsearch7. This folder holds the Sidecar Elasticsearch runtime.
Delete [Liferay Home]/data/elasticsearch7. This folder holds the index data.
Restart Liferay.
The Portal downloads and extracts the Elasticsearch runtime at startup on select App Servers (e.g, Tomcat).

Capability:

Resource Type:

Official Documentation

Feature:

Search Engine Installation and Maintenance

Deployment Approach:

Liferay Self-Hosted