Issue
- Would like to find document and media files faster during item selection.
- Providing part of the file name in the search term does not return the correct results or return nothing.
- Using wildcard at the beginning of a search term dos not provide any extra usability
Environment
- Liferay DXP
Resolution
- During a search, the Search engine performs text analysis to tokenize the keywords being searched for and attempts to match it with tokens belonging to indexed documents. Liferay uses the standard language analyzer for each language to do this analysis. (For the English locale, the English analyzer is used.) The analyzer has a list of "stopwords" that it looks for to break tokens up, for example a “-” in a word will split that token into 2 separate tokens.
-
What does it look like in action - with an example scenario:
In this case “_” is not a "stopword", which means it has been tokenized together as “icon_checkBoux” which doesn’t match the searched tokens “check” or “Boux”.
To see how an analyzer parses tokens, you can use a command like this:POST _analyze
{
"analyzer": "english",
"text": "icon_checkbox"
}
## which returns
{
"tokens" : [
{
"token" : "icon_checkbox",
"start_offset" : 0,
"end_offset" : 13,
"type" : "<ALPHANUM>",
"position" : 0
}
]
} - What does it look like in action - with the "default" use case:
Note that using other separates such as “-” or “ “ would create separate tokens which would match with “check”.
POST _analyze
{
"analyzer": "english",
"text": "icon-checkbox"
}
## which returns
{
"tokens" : [
{
"token" : "icon",
"start_offset" : 0,
"end_offset" : 4,
"type" : "<ALPHANUM>",
"position" : 0
},
{
"token" : "checkbox",
"start_offset" : 5,
"end_offset" : 13,
"type" : "<ALPHANUM>",
"position" : 1
}
]
} -
There is a way to mitigate this behavior (besides changing the file naming scheme), by customizing the Elasticsearch analyzer’s "stopwords". Here are the basic steps to achieve this:
1. Identify the language and analyzer:
> Determine the languages for which you want to modify the stopwords.
> Locate the corresponding analyzer in Elasticsearch's configuration. You can find this information by examining the Field Mappings.
2. Customize the analyzer following Elasticsearch Documentation
> Create a custom analyzer that includes a modified stop filter:
>> Define a new stop filter with a stopwords_path property.
>> Incorporate this filter into your custom analyzer's configuration.
3. Apply the custom analyzer in Liferay by Overriding the Mappings and changing the default analyzers to the new custom analyzers.
4. Initiate a full reindex to apply the new analyzer to your content.
Additional Information
-
The use of wildcard searches has been removed from Liferay by
LPS-130375: Discontinue "two star" wildcard search queries, and it’s related tickets. - Additional information on this topic: No results are returned when searching for a specific term because it is a Elasticsearch stopword