Unable to read the content for custom_robots.txt file
How To articles are not official guidelines or officially supported documentation. They are community-contributed content and may not always reflect the latest updates to Liferay DXP. We welcome your feedback to improve How To articles!
While we make every effort to ensure this Knowledge Base is accurate, it may not always reflect the most recent updates or official guidelines.We appreciate your understanding and encourage you to reach out with any feedback or concerns.
Legacy Article
You are viewing an article from our legacy "FastTrack"
publication program, made available for informational purposes. Articles
in this program were published without a requirement for independent
editing or verification and are provided"as is" without
guarantee.
Before using any information from this article, independently verify its
suitability for your situation and project.
Resolution
Why can that error message be thrown?
- That exception is thrown in RobotsUtil.java class. Either for
robots.txt without sitemap, or with sitemap.
- In the first case (without sitemap):
- It means that
layoutset is null when reaching RobotsUtil.java#L40
- Therefore, it tries to get the file set in
robots.txt.without.sitemap property.
- If the file cannot be found, it prints the aforementioned error message.
- In the second case (with sitemap):
- It means that
layoutset is not null in RobotsUtil.java#L40
- It retrieves the
robots.txt with sitemap here:
- From the
layoutset's settings, if they have been already modified.
- Otherwise, from the file indicated by the
robots.txt.with.sitemap property.
- If the file cannot be found, it prints the aforementioned error message.
Why is (or is not) layoutset null in RobotsUtil.getRobots() method?
Where should I place my custom robots.txt file?
- If you want to leverage
robots.txt.without.sitemap or robots.txt.with.sitemap properties, take into account that Liferay tries to get those files using the ClassLoader as base path. (see RobotsUtil.java#L43).
- For example, when using Tomcat, your custom
robots.txt file should be placed in tomcat/webapps/ROOT/WEB-INF/classes/ (or a relative path from there).
Did this article resolve your issue ?