> ## Documentation Index
> Fetch the complete documentation index at: https://ona.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Networking

The AWS runner runs as a Fargate service in your VPC. The ECS cluster contains three services: the runner orchestrator, a proxy for routing traffic to environments, and AWS ADOT for metrics export (when configured). Each Fargate task gets its own elastic network interface (ENI) in your subnets, so the subnets must provide a path to the AWS services the runner depends on.

## Required AWS service endpoints

The runner must reach the following AWS services over HTTPS (port 443):

| Service             | Endpoint                                | Purpose                                          |
| ------------------- | --------------------------------------- | ------------------------------------------------ |
| **Secrets Manager** | `secretsmanager.<region>.amazonaws.com` | Retrieve runner token and configuration secrets  |
| **CloudWatch Logs** | `logs.<region>.amazonaws.com`           | Ship runner and container logs                   |
| **ECR API**         | `api.ecr.<region>.amazonaws.com`        | Authenticate and authorize container image pulls |
| **ECR Docker**      | `*.dkr.ecr.<region>.amazonaws.com`      | Pull runner container images                     |
| **S3**              | `s3.<region>.amazonaws.com`             | Download ECR image layers                        |

Replace `<region>` with your AWS region.

<Warning>
  These connections must be **direct TCP**. The runner verifies reachability by dialing each endpoint directly, bypassing any HTTP proxy. If any endpoint is unreachable, the runner reports a degraded network status.
</Warning>

## Connectivity options

Choose the method that fits your network architecture.

### NAT gateway

Route traffic from private subnets to AWS service public endpoints via a NAT gateway in a public subnet.

**When to use:** Private subnets that already have a NAT gateway for general internet access.

No additional configuration is needed. As long as the Fargate task subnets have a route to a NAT gateway, the runner can reach AWS service endpoints over their public addresses.

### Internet gateway (with public IP)

Assign a public IP directly to the Fargate task and route traffic through an internet gateway.

**When to use:** Public subnets without NAT gateway, or development/test environments where simplicity is preferred.

To enable this, set the **Assign Public IP** CloudFormation parameter to `true` when deploying the runner stack. ECS assigns a public IP to each Fargate task ENI when this parameter is enabled.

<Warning>
  If your Fargate task subnets use an internet gateway but you do **not** enable public IP assignment, the tasks will have no outbound connectivity and the runner will fail to start.
</Warning>

### VPC endpoints (AWS PrivateLink)

Route traffic to AWS services over private IPs within your VPC, without traversing the public internet.

**When to use:** Private subnets with no internet access, or when security policy prohibits public internet egress.

See [VPC endpoints reference](#vpc-endpoints-reference) below for the full list of endpoints and creation instructions.

<Info>
  **Cost consideration**: VPC endpoints incur additional AWS charges (\~\$7.20/month per endpoint per AZ plus data processing fees). See [AWS PrivateLink pricing](https://aws.amazon.com/privatelink/pricing/) for details.
</Info>

### Transit gateway / VPC peering / VPN

Route traffic to AWS service endpoints through a transit gateway, VPC peering connection, or VPN tunnel to a VPC that has access to the services.

**When to use:** Hub-and-spoke network architectures where AWS service access is centralized in a shared-services VPC.

Ensure the route tables on the Fargate task subnets include routes to the AWS service endpoint IP ranges through the transit gateway or peering connection.

## Automatic connectivity checks

The runner automatically checks connectivity to all required endpoints on startup and every 5 minutes. Results are reported to the management plane as a network readiness status.

Each endpoint check:

1. Resolves the hostname via DNS
2. Opens a direct TCP connection on port 443

The report classifies each endpoint's access type:

| Access type        | Meaning                                                                       |
| ------------------ | ----------------------------------------------------------------------------- |
| `vpc_endpoint`     | DNS resolved to private IPs. Traffic stays within the VPC                     |
| `nat_gateway`      | DNS resolved to public IPs, task has no public IP. Traffic routes through NAT |
| `internet_gateway` | DNS resolved to public IPs, task has a public IP. Traffic routes through IGW  |

You can view the readiness status in the runner details on the Ona dashboard.

## VPC endpoints reference

If your network does not allow public internet egress, create VPC endpoints to give the runner infrastructure private access to AWS services. The table below lists all endpoints required by the runner stack, including those used by the ECS control plane, CloudFormation, environment VMs, and the runner services themselves.

### Prerequisites

Before creating VPC endpoints, ensure:

* Your VPC has both **DNS hostnames** and **DNS resolution** enabled
* You have the VPC ID, subnet IDs, and security group IDs ready
* You are creating endpoints in the same region as your runner deployment

You can enable DNS settings in the VPC Console under Actions > Edit VPC settings.

### Required interface endpoints

Replace `<region>` with your AWS region.

| Service                    | VPC Endpoint Service Name                     |
| -------------------------- | --------------------------------------------- |
| **EC2**                    | `com.amazonaws.<region>.ec2`                  |
| **ECR API**                | `com.amazonaws.<region>.ecr.api`              |
| **ECR Docker**             | `com.amazonaws.<region>.ecr.dkr`              |
| **SSM**                    | `com.amazonaws.<region>.ssm`                  |
| **STS**                    | `com.amazonaws.<region>.sts`                  |
| **CloudFormation**         | `com.amazonaws.<region>.cloudformation`       |
| **Secrets Manager**        | `com.amazonaws.<region>.secretsmanager`       |
| **ECS**                    | `com.amazonaws.<region>.ecs`                  |
| **ECS Agent**              | `com.amazonaws.<region>.ecs-agent`            |
| **ECS Telemetry**          | `com.amazonaws.<region>.ecs-telemetry`        |
| **SSM Messages**           | `com.amazonaws.<region>.ssmmessages`          |
| **EC2 Messages**           | `com.amazonaws.<region>.ec2messages`          |
| **CloudWatch Logs**        | `com.amazonaws.<region>.logs`                 |
| **IAM**                    | `com.amazonaws.<region>.iam`                  |
| **Elastic Load Balancing** | `com.amazonaws.<region>.elasticloadbalancing` |
| **ACM**                    | `com.amazonaws.<region>.acm`                  |

<Note>
  IAM is a global service, but its VPC endpoint is created as a regional interface endpoint. See the [AWS documentation on creating an IAM VPC endpoint](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_iam_vpc_endpoint_create.html) for setup instructions.
</Note>

### Required gateway endpoints

| Service      | VPC Endpoint Service Name         |
| ------------ | --------------------------------- |
| **S3**       | `com.amazonaws.<region>.s3`       |
| **DynamoDB** | `com.amazonaws.<region>.dynamodb` |

We recommend the S3 **gateway endpoint** because it is free of charge. Gateway endpoints are configured on route tables rather than ENIs. If you need DNS-based private resolution for S3 (e.g. for compliance reasons), use an interface endpoint instead.

<Note>
  Gateway endpoints (S3, DynamoDB) are configured differently than interface endpoints. See the [AWS documentation on gateway endpoints](https://docs.aws.amazon.com/vpc/latest/privatelink/gateway-endpoints.html) for details.
</Note>

### Creating VPC endpoints

#### Using AWS CLI

```bash theme={null}
# Replace vpc-xxx, subnet-xxx, and sg-xxx with your actual IDs
aws ec2 create-vpc-endpoint \
    --vpc-id vpc-xxx \
    --service-name com.amazonaws.vpce.us-east-1.vpce-svc-08de744d433e60ff2 \
    --vpc-endpoint-type Interface \
    --subnet-ids subnet-xxx subnet-xxx \
    --security-group-ids sg-xxx \
    --private-dns-enabled \
    --service-region us-east-1
```

For more CLI options and examples, see the [AWS CLI reference for create-vpc-endpoint](https://docs.aws.amazon.com/cli/latest/reference/ec2/create-vpc-endpoint.html).

#### Using AWS PowerShell

```powershell theme={null}
# Replace vpc-xxx, subnet-xxx, and sg-xxx with your actual IDs
New-EC2VpcEndpoint `
    -VpcId "vpc-xxx" `
    -ServiceName "com.amazonaws.vpce.us-east-1.vpce-svc-08de744d433e60ff2" `
    -VpcEndpointType "Interface" `
    -SubnetId @("subnet-xxx", "subnet-xxx") `
    -SecurityGroupId @("sg-xxx") `
    -PrivateDnsEnabled $true `
    -ServiceRegion "us-east-1"
```

For more PowerShell options and examples, see the [AWS PowerShell reference for New-EC2VpcEndpoint](https://docs.aws.amazon.com/powershell/v4/reference/items/New-EC2VpcEndpoint.html).

#### Using AWS Management Console

1. **Navigate to VPC Console**
   * Open the [AWS VPC Console](https://console.aws.amazon.com/vpc)
   * Select "Endpoints" from the left navigation

2. **Create Endpoint**
   * Click "Create endpoint"
   * **Name tag** (optional): Enter a descriptive name like `ona-management-plane`
   * **Service type**: Select "Endpoint services that use NLBs and GWLBs"

3. **Service Settings**
   * **Service name**: Enter `com.amazonaws.vpce.us-east-1.vpce-svc-08de744d433e60ff2`
   * **Enable Cross Region endpoint**: Check this box to connect across regions (Note: For **us-east-1** region, leave this unchecked)
   * **Region**: Select us-east-1 as the region
   * Click "Verify service" to confirm the service is available

<img src="https://mintcdn.com/gitpod-13c83c2b/-CSjSbIr1VxJFl5t/images/docs/flex/runners/vpc-endpoint-service-settings.png?fit=max&auto=format&n=-CSjSbIr1VxJFl5t&q=85&s=2c3dbf893bc0f494785b017425908043" alt="AWS VPC endpoint creation showing service name verification and cross-region endpoint settings" width="3320" height="1404" data-path="images/docs/flex/runners/vpc-endpoint-service-settings.png" />

4. **Network Settings**
   * **VPC**: Select the same VPC where your runner is deployed
   * **Enable DNS name**: Check this box to enable private DNS names (this allows `app.gitpod.io` to resolve to the endpoint)
   * **Subnets**: Choose private subnets in available AZs (can be the same subnets as your EC2 runner instances)
   * **Security groups**: Select or create a security group that allows HTTPS traffic (port 443, inbound). Include the runner security group and the security group for environment access (you can find these in the Output section of the Runner CloudFormation template).

<img src="https://mintcdn.com/gitpod-13c83c2b/-CSjSbIr1VxJFl5t/images/docs/flex/runners/vpc-endpoint-network-settings.png?fit=max&auto=format&n=-CSjSbIr1VxJFl5t&q=85&s=e98546503a21bd0ac70021d05a1c9ad9" alt="AWS VPC endpoint network configuration showing VPC, subnet, and DNS name settings" width="3320" height="1002" data-path="images/docs/flex/runners/vpc-endpoint-network-settings.png" />

5. **Create and Verify**
   * Click "Create endpoint"
   * Wait for the endpoint status to become "Available"

For detailed instructions on creating and managing VPC endpoints, see the [AWS documentation on creating interface endpoints](https://docs.aws.amazon.com/vpc/latest/privatelink/create-interface-endpoint.html).

#### Key configuration settings

When creating interface endpoints:

* **Enable private DNS names**: This allows AWS service hostnames to resolve to your VPC endpoint's private IP addresses
* **Subnets**: Select subnets in multiple availability zones for high availability
* **Security groups**: Configure to allow HTTPS (443) traffic from your runner subnets

<Note>
  **us-east-1 region specific**: When creating VPC endpoints in the us-east-1 region, disable cross-region support. This is due to availability zone mapping differences between AWS accounts that can prevent endpoint creation. The endpoint only needs to exist in one AZ within your VPC. Clients in other AZs can still reach it via DNS.
</Note>

### Security group configuration

Your VPC endpoint security group needs these rules:

**Inbound Rules:**

* **Type**: HTTPS (443)
* **Source**: Your runner subnets CIDR or the security group attached to your runner instances
* **Description**: Allow runner access to AWS services

**Outbound Rules:**

* **Type**: All traffic
* **Destination**: 0.0.0.0/0 (or keep the default outbound rule)

## Troubleshooting

<Accordion title="Runner fails to start or reports degraded status">
  1. **Check subnet route tables:** Verify the subnets assigned to the Fargate service have routes to the required endpoints (via NAT gateway, internet gateway, transit gateway, or VPC endpoints).

  2. **Check security groups:** The ECS task security group must allow **outbound HTTPS (port 443)** to the AWS service endpoints. The default `AllowAllOutbound` rule covers this.

  3. **Check VPC endpoint configuration** (if using PrivateLink):
     * Ensure **private DNS** is enabled on interface endpoints
     * Ensure the endpoint security group allows inbound HTTPS from the Fargate task subnets
     * If using an S3 gateway endpoint, ensure it is associated with the correct route tables

  4. **Check public IP assignment** (if using internet gateway):
     * Verify the [`AssignPublicIp` CloudFormation parameter](/ona/runners/aws/setup#network-configuration) is set to `true`
     * Verify the subnet has an internet gateway attached and a route to `0.0.0.0/0` via the IGW

  5. **DNS resolution:** From a test instance in the same subnet, verify endpoints resolve correctly:

     ```bash theme={null}
     nslookup secretsmanager.<region>.amazonaws.com
     nslookup logs.<region>.amazonaws.com
     nslookup api.ecr.<region>.amazonaws.com
     ```

     If using VPC interface endpoints, these should resolve to private IPs within your VPC CIDR.

       <Note>
         If using an S3 **gateway endpoint** (recommended), `nslookup s3.<region>.amazonaws.com` will still resolve to public IPs. This is expected. Gateway endpoints route traffic via route table entries, not DNS overrides. Verify connectivity by checking that the gateway endpoint is associated with the correct route tables. If using an S3 **interface endpoint** instead, DNS should resolve to private IPs.
       </Note>
</Accordion>

<Accordion title="VPC endpoint issues">
  * **DNS resolution fails**: Verify DNS hostnames and DNS resolution are enabled on your VPC
  * **Connection timeouts**: Check security group rules allow HTTPS (443) from runner subnets
  * **Endpoint creation fails in us-east-1**: Disable cross-region support when creating the endpoint
  * **Service unavailable**: Ensure the endpoint is in "Available" state and private DNS is enabled

  For additional troubleshooting, see the [AWS PrivateLink troubleshooting guide](https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-troubleshooting.html).
</Accordion>
