Hello everyone. I think this issue might be bigger than the AWS category but because it dwells on AWS behaviour I thought of asking the question here first.
The problem happens when using a data source to source the network interfaces created by a change to an already existing vpc endpoint. Instead of waiting for the creation of the new network interfaces after the apply step for the vpc endpoint has been concluded, the id of the network interfaces is calculated before the plan takes place. This results in using the data source for network interface ids which no longer exist. Hopefully showing the code and the output will make it easier to understand.
1st execution main.tf
data "aws_vpc_endpoint_service" "execute_api" {
service = "execute-api"
}
resource "aws_vpc_endpoint" "vpc-endpoint" {
vpc_id = local.aws_vpc_id
service_name = data.aws_vpc_endpoint_service.execute_api.service_name
vpc_endpoint_type = "Interface"
private_dns_enabled = false
subnet_ids = ["subnet-0acaeb18de056e049", "subnet-0cfd104085418b5ee"] //this is a
}
data "aws_network_interface" "vpc-network-interface" {
# convert a list of strings to a map with static keys and the strings as values
for_each = {
for index, _ in local.aws_private_subnet_ids_list :
index => tolist(aws_vpc_endpoint.vpc-endpoint.network_interface_ids)[index]
}
id = each.value
}
output "network_interface" {
value = data.aws_network_interface.vpc-network-interface[0].id #Just some dummy use of the network interface
}
1st execution plan
# data.aws_network_interface.vpc-network-interface["0"] will be read during apply
# (config refers to values not yet known)
<= data "aws_network_interface" "vpc-network-interface" {
+ arn = (known after apply)
+ association = (known after apply)
+ attachment = (known after apply)
+ availability_zone = (known after apply)
+ description = (known after apply)
+ id = (known after apply)
+ interface_type = (known after apply)
+ ipv6_addresses = (known after apply)
+ mac_address = (known after apply)
+ outpost_arn = (known after apply)
+ owner_id = (known after apply)
+ private_dns_name = (known after apply)
+ private_ip = (known after apply)
+ private_ips = (known after apply)
+ requester_id = (known after apply)
+ security_groups = (known after apply)
+ subnet_id = (known after apply)
+ tags = (known after apply)
+ vpc_id = (known after apply)
}
# data.aws_network_interface.vpc-network-interface["1"] will be read during apply
# (config refers to values not yet known)
<= data "aws_network_interface" "vpc-network-interface" {
+ arn = (known after apply)
+ association = (known after apply)
+ attachment = (known after apply)
+ availability_zone = (known after apply)
+ description = (known after apply)
+ id = (known after apply)
+ interface_type = (known after apply)
+ ipv6_addresses = (known after apply)
+ mac_address = (known after apply)
+ outpost_arn = (known after apply)
+ owner_id = (known after apply)
2nd execution main.tf (changing subnets)
data "aws_vpc_endpoint_service" "execute_api" {
service = "execute-api"
}
resource "aws_vpc_endpoint" "vpc-endpoint" {
vpc_id = local.aws_vpc_id
service_name = data.aws_vpc_endpoint_service.execute_api.service_name
vpc_endpoint_type = "Interface"
private_dns_enabled = false
subnet_ids = ["subnet-0f60a6451935fc47e", "subnet-00d87e4c8a945c9d2"]
}
data "aws_network_interface" "vpc-network-interface" {
# convert a list of strings to a map with static keys and the strings as values
for_each = {
for index, _ in local.aws_private_subnet_ids_list :
index => tolist(aws_vpc_endpoint.vpc-endpoint.network_interface_ids)[index]
}
id = each.value
}
output "network_interface" {
value = data.aws_network_interface.vpc-network-interface[0].id #Just some dummy use of the network interface
}
2nd execution plan
# data.aws_network_interface.vpc-network-interface["0"] will be read during apply
# (depends on a resource or a module with changes pending)
<= data "aws_network_interface" "vpc-network-interface" {
+ arn = (known after apply)
+ association = (known after apply)
+ attachment = (known after apply)
+ availability_zone = (known after apply)
+ description = (known after apply)
+ id = "eni-03eba5a22d9e813f8"
+ interface_type = (known after apply)
+ ipv6_addresses = (known after apply)
+ mac_address = (known after apply)
+ outpost_arn = (known after apply)
+ owner_id = (known after apply)
+ private_dns_name = (known after apply)
+ private_ip = (known after apply)
+ private_ips = (known after apply)
+ requester_id = (known after apply)
+ security_groups = (known after apply)
+ subnet_id = (known after apply)
+ tags = (known after apply)
+ vpc_id = (known after apply)
}
# data.aws_network_interface.vpc-network-interface["1"] will be read during apply
# (depends on a resource or a module with changes pending)
<= data "aws_network_interface" "vpc-network-interface" {
+ arn = (known after apply)
+ association = (known after apply)
+ attachment = (known after apply)
+ availability_zone = (known after apply)
+ description = (known after apply)
+ id = "eni-0c5d02224ccb4f89c"
+ interface_type = (known after apply)
+ ipv6_addresses = (known after apply)
+ mac_address = (known after apply)
+ outpost_arn = (known after apply)
+ owner_id = (known after apply)
+ private_dns_name = (known after apply)
+ private_ip = (known after apply)
+ private_ips = (known after apply)
+ requester_id = (known after apply)
+ security_groups = (known after apply)
+ subnet_id = (known after apply)
+ tags = (known after apply)
+ vpc_id = (known after apply)
}
# aws_vpc_endpoint.vpc-endpoint will be updated in-place
~ resource "aws_vpc_endpoint" "vpc-endpoint" {
id = "vpce-047bf5b3693274528"
~ subnet_ids = [
- "subnet-0acaeb18de056e049",
- "subnet-0cfd104085418b5ee",
+ "subnet-00d87e4c8a945c9d2",
+ "subnet-0f60a6451935fc47e",
]
tags = {
"test" = "test-2"
}
# (19 unchanged attributes hidden)
# (3 unchanged blocks hidden)
}
Plan: 0 to add, 1 to change, 0 to destroy.
The problem is visible on the 2nd plan. Notice that the id of the network interfaces is already defined when the plan occurs even though the ids can only be known after the changes to the resource aws_vpc_endpoint.vpc-endpoint take place, which will result in the creation of new network interfaces. Instead, it uses the previously created ENIs, which will result in an error when the first execution of terraform apply occurs.
│ Error: no matching EC2 Network Interface found
│
│ with data.aws_network_interface.vpc-network-interface["0"],
│ on main.tf line 60, in data "aws_network_interface" "vpc-network-interface":
│ 60: data "aws_network_interface" "vpc-network-interface" {
│
╵
╷
│ Error: no matching EC2 Network Interface found
│
│ with data.aws_network_interface.vpc-network-interface["1"],
│ on main.tf line 60, in data "aws_network_interface" "vpc-network-interface":
│ 60: data "aws_network_interface" "vpc-network-interface" {
After this error if terraform plan is ran again, the value of the ids of the network interfaces has already been updated, resulting in a successful run.