Issue
- When we upload a large number of files in a hierarchical directory structure, it takes a lot of time even if the file sizes are small.
Environment
- Liferay DXP 7.3
Resolution
-
When a User uploads many files, Liferay is processing the files one by one.
The three main parts of the upload process are:
1) Uploading it to S3 store
2) Saves the metadata in the database
3) Indexes the entry
Once it is done then it starts to upload the next file.
- Starting the indexing in bulk only after uploading all the images may sound like a better option, but the current functionality is intended to work like this, and usually, these approaches need careful planning by our Product Teams, that can also consider the limitations of the framework that Liferay uses.
-
There is some documentation regarding ElasticSearch performance, these related articles that could be useful when tuning it:
- https://help.liferay.com/hc/en-us/articles/360029031651-Preparing-to-Install-Elasticsearch
- https://help.liferay.com/hc/en-us/articles/360018175691-Tuning-and-Scaling-Elasticsearch
- https://learn.liferay.com/dxp/latest/en/using-search/installing-and-upgrading-a-search-engine/elasticsearch/getting-started-with-elasticsearch.html