Issue
- I'm not 100% sure on how to reproduce/trigger it, but it seems that when there is active traffic to the cluster, there is a fair chance that one of the nodes will deadlock during startup of the portal. We see this happen almost every deployment on production, however on dev, test & acc it almost never happens (probably because these have much less traffic during the deployments).
-
Today however I did manage to reproduce it sometimes on our dev servers, using our smoketest to put some traffic on the system, and then restarting one of the nodes in the cluster.
The result is that the startup hangs around the time where it tries to load the company configurations...
Company properties can be overwrritten by setting the environment variable LIFERAY_PROPS_BY_COMPANY_3145002
Loading file:/home/liferay/liferay-dxp-2024.q3.13/portal-ext.properties for company ID 3145002 and web ID msus
Company properties can be overwrritten by setting the environment variable LIFERAY_PROPS_BY_COMPANY_3145002
Loading jar:file:/home/liferay/liferay-dxp-2024.q3.13/tomcat/webapps/ROOT/WEB-INF/shielded-container-lib/portal-impl.jar!/portal-developer.properties for company ID 3145002 and web ID msus
Company properties can be overwrritten by setting the environment variable LIFERAY_PROPS_BY_COMPANY_3145002 - Then nothing after this (we have 10 companies); a restart can fix it, but may simply give the same issue (on production the solution is generally to restart both nodes at the same time, which is annoying because it gives outage that we try to prevent by having the cluster in the first place).
Environment
- 2024.q3.13
Resolution
- As described on https://liferay.atlassian.net/browse/LPD-42612 , this might be happening if you use the deprecated (since 7.3) feature described in https://help.liferay.com/hc/en-us/articles/360018421231-Using-Portal-Properties-for-Separate-Liferay-Instances
- if so, please try to not use this feature, remove the -Dcompany-id-properties=true JVM option and the company-specific properties files