CNE AWS Ready: Troubleshooting

Liferay Cloud Native Experience (CNE) deployments involve multiple components, including AWS infrastructure, Kubernetes, GitOps workflows, and Terraform/OpenTofu configurations. Issues can occur if resources become inconsistent or are modified outside the expected workflow.

Below is how to resolve common possible problems with deploying or managing CNE AWS Ready environments.

Terraform Errors After Deleting AWS Resources Manually

Infrastructure managed through infrastructure-as-code (IaC) tools such as Terraform or OpenTofu should remain the source of truth and control for the resources it creates. When resources are modified or deleted manually in the AWS console, the IaC state becomes inconsistent with the actual infrastructure.

When this happens, Terraform/OpenTofu may fail because it still expects those resources to exist.

For example, if an EKS cluster is deleted manually, Terraform may return an error like this:

Error: Get "http://localhost/apis/storage.k8s.io/v1/storageclasses/gp3": dial tcp [::1]:80: connect: connection refused

with kubernetes_storage_class_v1.gp3_storage_class,
on eks.tf line 203, in resource "kubernetes_storage_class_v1" "gp3_storage_class":
203: resource "kubernetes_storage_class_v1" "gp3_storage_class"
...

This occurs because the Terraform state still contains references to Kubernetes resources that no longer exist.

1. Navigate to the EKS Terraform directory:

cd liferay-cne-bootstrap/cloud/terraform/aws/eks

1. Authenticate with AWS:

aws sso login

1. Attempt to destroy the infrastructure:

terraform destroy -var-file="../../../cloud/scripts/global_terraform.tfvars"

If the command succeeds, the issue is resolved.

1. If the command fails, check whether Terraform still tracks Kubernetes resources:

terraform state list | grep kubernetes

1. If resources appear, remove them from the Terraform state:

terraform state rm [resource_name]

1. Repeat the same process in the following directories:

bootstrap/cloud/terraform/aws/gitops/platform

bootstrap/cloud/terraform/aws/gitops/resources

After removing the invalid state entries, Terraform should complete successfully.

Grafana Workspace Provisioning Fails

During bootstrap, AWS Managed Grafana (AMG) provisioning can fail with errors similar to:

Error: creating Grafana Workspace: operation error grafana: CreateWorkspace, ...
lookup grafana.<region>.amazonaws.com ... no such host

This is an intermittent issue and may depend on the selected AWS region.

1. Re-run the bootstrap script:

   ./cloud/scripts/setup_aws.sh ./config.json ./cloud/scripts/versions_aws.tfvars
   

2. If the issue persists, disable observability in your configuration:

   "observability_config": {
      "enabled": false
   }
   

Disabling observability skips Grafana provisioning and allows the bootstrap process to complete.

Bootstrap Script Requires Two Arguments

Recent versions of the bootstrap script require two arguments.

If you see an error like:

Usage: ./setup_aws.sh <configuration-json-file> <versions-tfvars-file>

Run the script with both required files:

./cloud/scripts/setup_aws.sh ./config.json ./cloud/scripts/versions_aws.tfvars

• config.json defines the deployment configuration.
• versions_aws.tfvars defines infrastructure component versions.

Partial Infrastructure Provisioning After Failure

If the bootstrap process fails, some AWS resources may still be created.

This can result in

• Incomplete infrastructure
• Terraform state inconsistencies
• Subsequent bootstrap failures

1. Re-run the bootstrap script to resume provisioning:

   ./cloud/scripts/setup_aws.sh ./config.json ./cloud/scripts/versions_aws.tfvars
   

2. If issues persist, clean up the environment before retrying.

Cannot Access Argo CD

If you cannot access Argo CD at http://localhost:8080, the bootstrap process may not have completed successfully.

Argo CD becomes available only after

• Infrastructure provisioning completes
• Kubernetes resources are deployed
• GitOps synchronization finishes

1. Verify the bootstrap script completed without errors.

2. If the script failed, resolve the error and re-run the bootstrap process.