Issue
- Any kind of update transactions do not work and users are not able to create/update web content, files, etc, in the portal.
- There are several errors in the Liferay Service's log about
java.sql.SQLException: The MySQL server is running with the --read-only option so it cannot execute this statement
Example:Apr 28 09:26:22.890 build-1546 [liferay-7b56457799-c9gtg] org.quartz.JobPersistenceException: Failure updating scheduler state when checking-in: The MySQL server is running with the --read-only option so it cannot execute this statement [See nested exception: java.sql.SQLException: The MySQL server is running with the --read-only option so it cannot execute this statement] at org.quartz.impl.jdbcjobstore.JobStoreSupport.clusterCheckIn(JobStoreSupport.java:3393) at org.quartz.impl.jdbcjobstore.JobStoreSupport.doCheckin(JobStoreSupport.java:3240) at org.quartz.impl.jdbcjobstore.JobStoreSupport$ClusterManager.manage(JobStoreSupport.java:3857) at org.quartz.impl.jdbcjobstore.JobStoreSupport$ClusterManager.run(JobStoreSupport.java:3894) Caused by: java.sql.SQLException: The MySQL server is running with the --read-only option so it cannot execute this statement at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:129) at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:955) at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1094) at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdateInternal(ClientPreparedStatement.java:1042) at com.mysql.cj.jdbc.ClientPreparedStatement.executeLargeUpdate(ClientPreparedStatement.java:1345) at com.mysql.cj.jdbc.ClientPreparedStatement.executeUpdate(ClientPreparedStatement.java:1027) at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61) at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java) at org.quartz.impl.jdbcjobstore.StdJDBCDelegate.updateSchedulerState(StdJDBCDelegate.java:2930) at org.quartz.impl.jdbcjobstore.JobStoreSupport.clusterCheckIn(JobStoreSupport.java:3387) ... 3 more
Environment
- Liferay Cloud projects
Resolution
The read-only error experienced with your DB is related to an issue with the legacy HA mode for Cloud SQL. Cloud SQL checks the availability of the master instance using different checkpoints to know if the instance is healthy and in particular, one of them is the writability of the disk; in case the master is not ok because of the writability, the traffic will be routed to the failover replica causing the state of the DB to be "read-only.
The long-term solution is to set up an HA mode that does not use an explicit failover replica.
The new HA replication mode has some added benefits such as increased replication performance and will have no cost implications for our customers
The Liferay team will coordinate with customers to provide a detailed path to perform this migration. Meanwhile, should the behavior happen the recommended workaround is to restart the Liferay service.
Additional Information
Please, contact the Support Service opening a ticket in the Help Center.
Check the incident's updates in https://status.liferay.cloud