Note: please note that Liferay has renamed its Liferay Experience
Could offerings to Liferay SaaS (formerly LXC) and
Liferay PaaS (formerly LXC-SM).
Issue
- Whenever we deploy a build (even if the image versions are not changed), we notice an "error killing pod" in the status logs of a service. For example, this is what is seen with the search service:
Mar 22 16:01:24.000 [search-0] Stopping container search
Mar 22 16:06:25.000 [search-0] error killing pod: failed to "KillContainer" for "search" with KillContainerError: "rpc error: code = Unknown desc = operation timeout: context deadline exceeded"
Mar 22 16:06:37.000 [search-0] Pulling image... - The deployment does appear successful, but there seems to be a five minute gap where the new container is not created (see above), until after this error is observed
Environment
- Liferay PaaS
Resolution
- This behavior is resolved via the fixes created for the internal tracking ticket LCD-10441. To apply this fix, deploy a build using the minimum image versions described in the 3/11/2021 Service Update 2021.10.1, and any future deployments will no longer encounter this error or delay.
Additional Information
- This error was encountered due to the containers not being able to fully complete their shutdown processes, which led to Liferay PaaS keeping those containers active until the grace period was surpassed (5 minutes, in this case). The fix allows the containers to complete all shutdown processes, without needing to wait for the termination grace period to be exceeded.
- Additional information regarding deployment and logs can be found in the following articles: