"Too Many Open Files" error in log

Issue

"Too Many Open Files" error is reported in the log.

For example:

2019-05-06 09:41:20,142 SEVERE [org.apache.tomcat.util.net.NioEndpoint] Socket accept failed
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.apache.tomcat.util.net.NioEndpoint$Acceptor.run(NioEndpoint.java:682)
at java.lang.Thread.run(Thread.java:748)

02/09/2019 23:45:52,531 ERROR [liferay/lcs_status-1][LCSLicenseManager:?] Unable to read LCSState java.io.FileNotFoundException: /opt/liferay-dxp-digital-enterprise-7.0-sp2/data/license/server/lcsState (Too many open files) 
at java.io.RandomAccessFile.open0(Native Method) 
at java.io.RandomAccessFile.open(RandomAccessFile.java:316) 
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:243) 
at com.liferay.portal.util.FileImpl.getBytes(FileImpl.java:545) 
at com.liferay.portal.kernel.util.FileUtil.getBytes(FileUtil.java:241) 
at com.liferay.portal.ee.license.LCSLicenseManager.f(Unknown Source) 
at com.liferay.portal.ee.license.LCSLicenseManager.a(Unknown Source) 
at com.liferay.portal.ee.license.messaging.LicenseMessageListener.doReceive(Unknown Source) 
at com.liferay.portal.kernel.messaging.BaseMessageListener.receive(BaseMessageListener.java:26) 
at com.liferay.portal.kernel.messaging.InvokerMessageListener.receive(InvokerMessageListener.java:74) 
at com.liferay.portal.kernel.messaging.ParallelDestination$1.run(ParallelDestination.java:52) 
at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask._runTask(ThreadPoolExecutor.java:756) 
at com.liferay.portal.kernel.concurrent.ThreadPoolExecutor$WorkerTask.run(ThreadPoolExecutor.java:667) 
at java.lang.Thread.run(Thread.java:745)

Note: The second LCSLicenseManager error may affect the license registration.

Environment

All Liferay versions

Resolution

This resolution requires altering operating system configurations and should only be implemented at the discretion of your team. Liferay Support will not be able to assist with providing instructions or parameters specific to your operating system.
The error is due to the operating system's open files limits. To resolve the error, increase the OS level open files limits. For example:
```
  ulimit -n 65535
```
To check the limit of how many open files you can have, execute:
```
ulimit -n
```
If increasing the ulimit does not resolve the issue, you may also need to increase the open file limits at the service level as well. An example of this on CentOS 7 and Red Hat 7 can be found at this third-party website: https://www.grumpyland.com/blog/231/changing-limits-for-services-with-centos-7-rhel-7-systemd/
If your Liferay runs on Docker, it's necessary to adapt the solution, since inside a Docker container it's not possible in general to override file limits with the command ulimit. You can either:
- pass the flag --ulimit when running docker run, like --ulimit nofile=1000:1000, or
- add an option in docker-compose, like
```
ulimits:
      nofile:
        soft: 1000
        hard: 1000
```