Tuning Your Auto-Scaling Settings

Auto-scaling ensures your services have the appropriate amount of instances at different levels of usage. You should, however, monitor your environment’s usage and observe the behavior so you can tune your settings for better results.

Tuning your auto-scaling settings starts with configuring your target average utilization values.

Auto-Scaling Behavior

Your service upscales or downscales based on criteria relative to your configured target utilization value, when these criteria are met:

1. The desired number of instances changes up or down. This number is calculated as currentInstances x (currentUtilization / targetUtilization), rounded up to the next integer.

   A separate value is calculated using both the CPU and memory target utilization values. The higher number of desired instances is used, to ensure both CPU and memory requirements are met for your usage.

2. The difference between the current utilization and the target utilization must be outside the tolerance window of 10%.

   To prevent rapidly scaling up and down, the ratio of currentUtilization / targetUtilization must be ≤ 0.9 to trigger a downscale, and it must be ≥ 1.1 to trigger an upscale.

Once both of these conditions are sustained for 5 minutes, the service upscales or downscales to the appropriate number of instances.

Configuring Target Average Utilization

System administrators can specify a target average utilization. This value is the average of memory and CPU usage across Liferay DXP services. That value threshold must be crossed before auto-scaling is triggered.

For example, if three service instances utilize 70%, 90%, and 95% of memory, respectively, the average memory utilization is 85%. If the target average utilization is set to 90, no upscaling is needed; upscaling in this situation only occurs when the average memory utilization exceeds the target.

Note

The default target utilization is 80%. Services that use a high amount of memory (such as the liferay service) can surpass this target utilization value quickly, even immediately after deployment.

The total available memory is specified by the memory property in LCP.json, as referenced in Configuration via LCP.json.

Specify the target average utilization in the autoscale property of the service’s LCP.json:

"autoscale": {
  "cpu": 80,
  "memory": 90
}

Balance your target average utilization according to your application’s specific needs for the most efficient auto-scaling. For example, this configuration heavily prioritizes CPU usage:

"autoscale": {
  "cpu": 60,
  "memory": 95
}

If the autoscale property isn’t set, the target average utilization defaults to 80 for both CPU and memory utilization.

JVM-Based Auto-Scaling

In most cases, tuning your target utilization value is sufficient to promote desirable auto-scaling behavior. However, some environments may not downscale as expected because unused memory is not actively reallocated from the JVM. This can happen when the operating system prefers to keep memory in the page cache to optimize the application’s performance.

In these instances, you may see better auto-scaling results with JVM-based scaling, rather than the default scaling behavior. This graph shows an example of this behavior: the green line shows the JVM’s heap usage as garbage collection occurs, and the blue line shows the total memory allocated to the application in the service’s container.

JVM-based scaling follows the same auto-scaling behavior, but it respects the usage of the JVM’s heap memory instead of how much memory is allocated by the operating system. Your environment may perform better with JVM-based scaling if

• Your application has configured garbage collection to actively free up memory.

• Your application allocates large amounts of memory by default, so it naturally stays close to auto-scaling thresholds.

• Your application’s total memory allocation remains high even in periods of low traffic.

Enable JVM-based auto-scaling in your LCP.json file’s autoscale object:

1. Set the memory property to a high value (e.g., 1000).

2. Add a prometheus.googleapis.com|jvm_memory_pool_bytes_used|gauge property, and set it to a value proportional to the total bytes in the JVM’s heap.

For example, this LCP.json configuration sets a JVM heap usage threshold of about 80% of 12 GB:

"autoscale": {
    "cpu": 90,
    "memory": 1000,
    "prometheus.googleapis.com|jvm_memory_pool_bytes_used|gauge": "10307921510"
}

The extremely high memory threshold effectively disables the standard mechanism of auto-scaling based on container metrics, in favor of JVM-based scaling.

Instead, the service uses the configured number of bytes as a target memory threshold, which gives auto-scaling a more accurate measurement of your service’s memory usage for some applications.

Alternative Scaling Metrics

Alternative metric configurations are available if your environment’s performance aligns better with other metrics. Add one of these values to autoscale in LCP.json to add a usage threshold (as a real value, not a percentage):

prometheus.googleapis.com|catalina_globalrequestprocessor_processingtime|unknown

prometheus.googleapis.com|catalina_globalrequestprocessor_requestcount|unknown

prometheus.googleapis.com|catalina_threadpool_currentthreadcount|unknown

prometheus.googleapis.com|catalina_threadpool_currentthreadsbusy|unknown

prometheus.googleapis.com|com_zaxxer_hikari_pool_hikaripool_{1,2,3...}_activeconnections|unknown

prometheus.googleapis.com|com_zaxxer_hikari_pool_hikaripool_{1,2,3...}_idleconnections|unknown

prometheus.googleapis.com|java_lang_threading_threadcount|unknown

prometheus.googleapis.com|jvm_buffer_pool_used_bytes|gauge

prometheus.googleapis.com|jvm_classes_currently_loaded|gauge

prometheus.googleapis.com|jvm_gc_collection_seconds_count|summary

prometheus.googleapis.com|jvm_memory_pool_used_bytes|gauge

prometheus.googleapis.com|jvm_memory_used_bytes|gauge

prometheus.googleapis.com|jvm_threads_current|gauge

prometheus.googleapis.com|jvm_threads_peak|gauge

prometheus.googleapis.com|jvm_threads_state|gauge

prometheus.googleapis.com|process_cpu_seconds_total|counter

prometheus.googleapis.com|process_open_fds|gauge

prometheus.googleapis.com|process_resident_memory_bytes|gauge

If necessary, set the memory or cpu metrics to high percentages (e.g., 1000) to functionally disable them in favor of your custom metrics.

Related Topics

• Scaling the Liferay Service