Skip to main content

Documentation Index

Fetch the complete documentation index at: https://ona.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

Network misconfigurations are the most common cause of issues. See access requirements first.

​
CloudFormation stack fails

Symptoms: ROLLBACK_COMPLETE or ROLLBACK_IN_PROGRESS with errors like Parameter validation failed: parameter value for EC2RunnerInstancesSubnet does not exist. Fix: Ensure you select a VPC, at least one availability zone, and subnets across multiple AZs.

​
Runner task fails

Symptoms:
  • CREATE_FAILED with ECS Deployment Circuit Breaker was triggered
  • ResourceInitializationError in task logs
  • Cannot pull images or access AWS services
Fix:
  • Verify VPC has Internet Gateway or NAT Gateway
  • Update route tables (public → IGW, private → NAT)
  • For private subnets, add VPC endpoints for Secrets Manager, S3, ECR
  • Check security groups allow outbound HTTPS

​
Instance type not available

Symptoms: Error like “m6i.xlarge is not available in us-east-1e” Fix:

​
Unexpected costs

Symptoms: Unexpected AWS charges, or continued billing after deleting a runner. Fix:
  • See managing costs to identify resources
  • After deleting a runner, verify the CloudFormation stack is fully deleted
  • Check for residual EC2 instances or EBS volumes and delete manually

​
SSM access blocked

Symptoms:
  • Environments fail with AWS account policy blocks ssm:SendCommand
  • Runner marked as degraded
  • Slow startup (cache credentials can’t refresh)
Cause: Service Control Policies (SCPs) blocking SSM access. The runner needs ssm:SendCommand and ssm:GetCommandInvocation permissions. Fix: Request your AWS administrator add an exception for the runner’s IAM role: 
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": ["ssm:SendCommand", "ssm:GetCommandInvocation"],
    "Resource": ["arn:aws:ec2:*:*:instance/*", "arn:aws:ssm:*:*:command/*"]
  }]
}

​
Prebuilds fail due to policy restrictions

Symptoms:
  • Prebuilds fail with AWS Service Control Policy blocks ec2:CreateSnapshot
  • Prebuilds fail with AWS IAM policy does not allow ec2:CreateSnapshot
  • Similar errors for ec2:RegisterImage, ec2:DescribeSnapshots, or ec2:DescribeImages
Cause: Prebuilds require creating EBS snapshots and AMIs. These operations can be blocked by:
  1. Service Control Policies (SCPs) - Organization-level policies that deny EC2 snapshot/AMI actions
  2. IAM policies - The runner’s IAM role is missing required permissions (outdated CloudFormation stack)

​
Fix for SCP restrictions

Request your AWS administrator to allow these actions for the runner’s IAM role in the SCP: 
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Action": [
      "ec2:CreateSnapshot",
      "ec2:RegisterImage",
      "ec2:DescribeSnapshots",
      "ec2:DescribeImages",
      "ec2:DeleteSnapshot",
      "ec2:DeregisterImage"
    ],
    "Resource": "*"
  }]
}

​
Fix for IAM policy restrictions

Update your CloudFormation stack to the latest version. The latest stack template includes all required IAM permissions for prebuilds.

​
Custom CA certificate issues

The ca-trust-init container in the Runner’s ECS task logs can help diagnose CA issues. Check its Status (it should show STOPPED with exit code 0) and its Logs for errors and warnings.

​
Environment stops shortly after starting

Symptoms: Environment starts but stops within seconds. No error is visible in the Ona dashboard. Cause: The environment instance failed to download or parse the CA bundle. The instance shuts down when this fails. Fix:
  • Verify the CA bundle source (S3 bucket or HTTPS URL) is accessible from the runner’s VPC
  • For S3 URLs, ensure the bucket name starts with gitpod- — the runner’s IAM role only has access to buckets matching gitpod-*
  • Confirm the S3 object exists and the path in the CustomCATrustBundle parameter is correct
  • Verify the CA bundle is valid PEM — the -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- delimiters must each be on their own line
  • Verify the CA bundle does not contain invalid content, such as displaying metadata before -----BEGIN CERTIFICATE-----

​
User data is limited to 16384 bytes

Symptoms: Environment creation fails with "User data is limited to 16384 bytes" Cause: The CustomCATrustBundle CloudFormation parameter uses an SSM dynamic reference ({{resolve:ssm:...}}), which embeds the full PEM certificate content into EC2 user data. CA bundles with multiple certificates exceed the 16 KB AWS limit. Fix: Switch to an S3 URL for the trust bundle:
  1. Create an S3 bucket with a name starting with gitpod- (e.g. gitpod-myorg)
  2. Upload your CA bundle to s3://gitpod-myorg/shared/ca-bundle.pem
  3. Update the CustomCATrustBundle CloudFormation parameter to s3://gitpod-myorg/shared/ca-bundle.pem
  4. Update the CloudFormation stack
See Custom CA Certificate for details on all supported formats.

​
CA not trusted in devcontainer builds

Symptoms: Devcontainer image builds or feature installs fail with TLS certificate errors, even though the custom CA works for other operations. Cause: Custom CA certificates are applied to the runner and environment host but not injected into devcontainer build phases. Docker builds run in an isolated context that does not inherit the host’s CA trust store. Fix: Add your CA certificates directly to your devcontainer image: 
COPY my-ca-bundle.crt /usr/local/share/ca-certificates/
RUN update-ca-certificates

​
Network connectivity issues

Checklist:
  • Security groups: port 29222 (SSH), outbound HTTPS, port 22999 (internal)
  • Route tables: public subnets → IGW, private subnets → NAT
  • Network ACLs: not blocking required traffic
  • DNS: VPC DNS resolution enabled, can resolve app.gitpod.io
Test connectivity: 
# Health endpoint (should return 200)
curl -v https://<your-domain>/_health

# Required endpoints
curl -I https://app.gitpod.io
curl -I https://public.ecr.aws

​
Restart runner after network changes

After changing security groups, route tables, or VPC endpoints, restart the runner: Console: ECS console → Clusters → your cluster → Services → Update → check Force new deployment CLI: 
aws ecs update-service --cluster YOUR_CLUSTER_NAME --service YOUR_SERVICE_NAME --force-new-deployment
Verify: Check runner shows “Connected” in Settings → Runners, then test creating an environment.

​
Getting help

Use the support chat (bubble icon in bottom-right). Include:
  • Runner ID and version (from Settings → Runners → ... menu)
  • CloudFormation stack name and region
  • Runner logs from CloudWatch (ECS task logs)