Route53 Resolver Endpoint Hash Function Prevents Multiple IPs per Subnet When IP Address is Auto-Assigned

Terraform and AWS Provider Version

Terraform v1.9.8
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v6.7.0

Tested across multiple versions:
- AWS Provider v5.0.0 through v6.7.0 (all affected)
- Terraform v1.1.9 through v1.9.8 (all affected)

Affected Resource(s) or Data Source(s)

• aws_route53_resolver_endpoint

Expected Behavior

AWS Route53 Resolver endpoints should support up to 6 IP addresses per endpoint (quotas can be increased), including multiple IP addresses per subnet with auto-assignment. When configuring multiple ip_address blocks for the same subnet without explicit IP addresses, AWS should auto-assign unique IP addresses within each subnet, and Terraform should track all configured IP addresses correctly.

According to AWS documentation, this is a supported configuration.

  # aws_route53_resolver_endpoint.test will be created
  + resource "aws_route53_resolver_endpoint" "test" {
      + arn                    = (known after apply)
      + direction              = "OUTBOUND"
      + host_vpc_id            = (known after apply)
      + id                     = (known after apply)
      + protocols              = (known after apply)
      + region                 = "us-east-1"
      + resolver_endpoint_type = (known after apply)
      + security_group_ids     = (known after apply)
      + tags                   = {
          + "Name" = "test-multiple-ips-per-subnet"
        }

      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + ipv6      = (known after apply)
          + subnet_id = "subnet-12345"
        }
      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + ipv6      = (known after apply)
          + subnet_id = "subnet-12345"
        }
      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + ipv6      = (known after apply)
          + subnet_id = "subnet-12345"
        }
      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + ipv6      = (known after apply)
          + subnet_id = "subnet-67890"
        }
      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + ipv6      = (known after apply)
          + subnet_id = "subnet-67890"
        }
      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + ipv6      = (known after apply)
          + subnet_id = "subnet-67890"
        }
    }

Actual Behavior

No error is thrown, but terraform plan shows fewer IP addresses than configured:

  # aws_route53_resolver_endpoint.test will be created
  + resource "aws_route53_resolver_endpoint" "test" {
      + arn                    = (known after apply)
      + direction              = "OUTBOUND"
      + host_vpc_id            = (known after apply)
      + id                     = (known after apply)
      + protocols              = (known after apply)
      + region                 = "us-east-1"
      + resolver_endpoint_type = (known after apply)
      + security_group_ids     = (known after apply)
      + tags                   = {
          + "Name" = "test-multiple-ips-per-subnet"
        }

      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + ipv6      = (known after apply)
          + subnet_id = "subnet-12345"
        }
      + ip_address {
          + ip        = (known after apply)
          + ip_id     = (known after apply)
          + ipv6      = (known after apply)
          + subnet_id = "subnet-67890"
        }
    }

Relevant Error/Panic Output

Sample Terraform Configuration

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

locals {
  subnet_ids = [
    "subnet-12345",
    "subnet-67890",
  ]
}

resource "aws_security_group" "test" {
  name   = "test-resolver-endpoint"
  vpc_id = "vpc-12345"

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "test-resolver-endpoint"
  }
}

# This demonstrates the hash collision bug
# Expected: 6 IP addresses (3 per subnet)
# Actual: 2 IP addresses (1 per subnet due to hash collisions)
resource "aws_route53_resolver_endpoint" "test" {
  direction = "OUTBOUND"

  security_group_ids = [aws_security_group.test.id]

  # Multiple auto-assigned IPs per subnet - this is the problematic case
  ip_address {
    subnet_id = local.subnet_ids[0] # First IP in subnet A
  }
  ip_address {
    subnet_id = local.subnet_ids[0] # Second IP in subnet A - gets deduplicated!
  }
  ip_address {
    subnet_id = local.subnet_ids[0] # Third IP in subnet A - gets deduplicated!
  }
  ip_address {
    subnet_id = local.subnet_ids[1] # First IP in subnet B
  }
  ip_address {
    subnet_id = local.subnet_ids[1] # Second IP in subnet B - gets deduplicated!
  }
  ip_address {
    subnet_id = local.subnet_ids[1] # Third IP in subnet B - gets deduplicated!
  }

  tags = {
    Name = "test-multiple-ips-per-subnet"
  }
}

output "ip_address_count" {
  value       = length(aws_route53_resolver_endpoint.test.ip_address)
  description = "Should be 6 but will show 2 due to hash collision bug"
}

output "ip_addresses" {
  value = aws_route53_resolver_endpoint.test.ip_address
}

Steps to Reproduce

1. Apply the provided configuration with multiple ip_address blocks for the same subnet without explicit ip values
2. Run terraform plan and observe that only 2 IP addresses are planned instead of 6
3. Run terraform apply and verify that only 1 IP address per subnet is created
4. Check the AWS Console to confirm only 2 IP addresses exist on the resolver endpoint
5. Check Terraform state: terraform state show aws_route53_resolver_endpoint.test will show only 2 IP addresses

Alternative reproduction with existing resources:

1. Create a resolver endpoint through AWS Console with multiple IPs per subnet (this works fine)
2. Import it into Terraform using terraform import aws_route53_resolver_endpoint.test rslvr-out-12345
3. Run terraform plan - it will show a diff trying to remove the "extra" IP addresses due to the hash collision

Debug Logging

2025-08-05T11:53:38.760-0700 [DEBUG] ReferenceTransformer: "aws_route53_resolver_endpoint.test" references: [aws_security_group.test local.subnet_ids (expand) local.subnet_ids (expand) local.subnet_ids (expand) local.subnet_ids (expand) local.subnet_ids (expand) local.subnet_ids (expand)]
2025-08-05T11:53:38.760-0700 [DEBUG] ReferenceTransformer: "local.subnet_ids (expand)" references: []

GenAI / LLM Assisted Development

n/a

Important Facts and References

AWS Service Capability vs Provider Limitation:

• AWS Route53 Resolver supports up to 6 IP addresses per endpoint (quotas can be increased)
• AWS Route53 Resolver supports multiple IP addresses per subnet for higher query limits
• AWS Route53 Resolver supports auto-assignment of IP addresses within subnets
• AWS documentation encourages letting AWS manage IP allocation for operational simplicity

Business Impact:

• Prevents users from implementing multiple IP addresses per subnet for DNS resolver configurations for higher query limits
• Forces manual IP management instead of leveraging AWS auto-assignment capabilities
• Creates a gap between AWS service capabilities and Terraform provider support
• May lead to configuration drift when users work around the limitation using AWS Console or CLI

Technical Analysis:

• Issue is specifically in the endpointHashIPAddress function in /internal/service/route53resolver/endpoint.go
• Current hash format: fmt.Fprintf(&buf, "%s-%s-", m[names.AttrSubnetID].(string), m["ip"].(string))
• When ip is empty (auto-assigned), identical subnets produce identical hashes: "subnet-12345-"
• This affects both resource creation and state management (imports, refreshes, etc.)

Similar Patterns in Other Resources:

• aws_instance successfully handles multiple network interfaces per subnet using unique identifiers
• aws_lb_target_group_attachment handles multiple targets per target group without hash collisions
• Other AWS resources with Set-based configurations have solved similar problems

Environment Details:

• Affects all AWS regions and partitions
• No workarounds exist other than manual IP specification
• Issue present since the resource was first implemented

Would you like to implement a fix?

No