# Module resources recreated on every run - dependency/ordering issue - HELP! :)

**URL:** <https://discuss.hashicorp.com/t/module-resources-recreated-on-every-run-dependency-ordering-issue-help/57465>\
**Category:** Terraform\
**Created:** [August 21, 2023, 9:19am UTC](https://discuss.hashicorp.com/t/module-resources-recreated-on-every-run-dependency-ordering-issue-help/57465 "2023-08-21T09:19:22Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![lambbuster](https://sea2.discourse-cdn.com/hashicorp/user_avatar/discuss.hashicorp.com/lambbuster/32/37502_2.png) [@lambbuster](https://discuss.hashicorp.com/u/lambbuster)\
**Post date:** [August 21, 2023, 9:19am UTC](https://discuss.hashicorp.com/t/module-resources-recreated-on-every-run-dependency-ordering-issue-help/57465/1 "2023-08-21T09:19:22Z")

</div>

Hello there!

I have an issue that’s driving me insane! It’s a clear dependency/ordering issue but so far i haven’t found a way round it…i’ve attempted to force the ordering in the way i need without any luck.

High level:

I have 2 modules, one for creating a VPC (and associated resources such as subnets, NACLs etc.) and the other for creating a VPC endpoint (and associated resources such as policy, security group etc.).

I require the ability to create 1…n VPCs with 1…n sets of subnets in one module, then 1…n VPC Endpoints in the other module that of course needs the VPC config (VPC & Subnet IDs) to exist before creation. When i create a single VPC, everything works fine and nothing changes on reapply. However, when i add a second VPC, the VPC endpoint resources from the first VPC want to recreate stating their VPC ID is not known before apply.

I’ve stripped the 2 modules right back so it creates a barebones VPC with a single subnet, and a single VPC endpoint defined and i’m still hitting the same issue.

I know i could just move the VPC endpoint resources to the VPC module, however i’m looking at this as a last resort as we use the VPC endpoint module in numerous other places and version control it for any updates etc.

Few things to note:

1. We use data sources (aws\_vpc & aws\_subnets) with tag filters in the VPC endpoint module to select the VPC and Subnets
2. If i remove the `depends_on` from the `vpc_endpoint` module then i get an ` Error: no matching EC2 VPC found` error
3. I have stripped back the resources for the sake of troubleshooting, however we would have multiple other VPC modules (e.g. `vpc_edmz`) that would be concatenated and included in the VPC endpoint local map variable for input to the VPC endpoint module.
4. There are many more resources in the VPC module, each having their own `for_each` logic based off the subnets input. I’m guessing these could potentially be affecting the ordering.
5. On first apply i can successfully create more than 1 VPC without issue, it only seems to be a problem if you apply once, add an additional VPC definition, then attempt to apply a second time.

Here are the resources:

**VPC Module** :

```auto
module "vpc_idmz" {
  for_each = { for vpc in var.vpc_idmz : vpc["name"] => vpc }
  source = "git::https://xxx"
  kms_key_arn = local.kms_key_arn
  vpc = merge(each.value,
    {
      network_zone = "idmz"
      create_internet_gateway = false
    }
  )
}

```

**VPC Endpoint Module**

```auto
module "vpc_endpoint" {
  for_each = { for vpc_endpoint in local.vpc_endpoint_map : "${vpc_endpoint["service_name"]}-${vpc_endpoint["vpc_name"]}" => vpc_endpoint }
  source = "git::https://xxx"
  vpc_tag_filters = { Name = module.vpc_idmz[each.value["vpc_name"]].name }
  #vpc_tag_filters = { network_zone = each.value["vpc_network_zone"], Name = each.value["vpc_name"] }
  hosted_zone_id = null
  vpc_endpoint = {
    name = each.value["service_name"]
    vpc_endpoint_type = each.value["vpc_endpoint_type"]
    private_dns_enabled = each.value["private_dns_enabled"]
    service_name = "com.amazonaws.${data.aws_region.current.name}.${each.value["service_name"]}"
    policy = local.vpc_endpoint_policies_map[each.value["service_name"]]["policy"]
    dns_name = null
    subnet_tag_filters = each.value["subnet_tag_filters"]
    security_group_ids = []
    route_table_ids = each.value["route_table_ids"]
    ingress_cidr_blocks = each.value["ingress_cidr_blocks"]
  }
  depends_on = [module.vpc_idmz]
}

```

**VPC Endpoint Module Data Sources**

```auto
data "aws_vpc" "selected" {
  tags = var.vpc_tag_filters
}
data "aws_subnets" "selected" {
  filter {
    name = "vpc-id"
    values = [data.aws_vpc.selected.id]
  }
  tags = var.vpc_endpoint["subnet_tag_filters"]
}

```

**VPC Input Variable**

```auto
variable "vpc_idmz" {
  type = list(object({
    name = optional(string)
    network_zone = optional(string, "idmz")
    network_function = optional(string)
    cidr_block = optional(string)
    additional_cidr_blocks = optional(list(string), [])
    enable_dns_support = optional(bool, true)
    enable_dns_hostnames = optional(bool, true)
    tags = optional(map(string), {})
    subnets = list(object({
      name = string
      type = string
      cidr_blocks = list(string)
      create_nat_gateway = optional(bool, false)
      create_route_to_internet_gateway = optional(bool, false)
      endpoints = optional(list(object({
        vpc_endpoint_type = optional(string, "Interface")
        private_dns_enabled = optional(bool, true)
        service_name = optional(string)
        dns_name = optional(string)
        ingress_cidr_blocks = optional(list(string), [])
        resources = optional(list(string), [])
      })), [])
      nacl = object({
        ingress_rules = optional(list(object({
          protocol = string
          rule_number = number
          rule_action = string
          cidr_block = string
          from_port = number
          to_port = number
          })), [{
          rule_number = 100
          protocol = -1
          rule_action = "allow"
          cidr_block = "0.0.0.0/0"
          from_port = 0
          to_port = 0
        }])
        egress_rules = optional(list(object({
          protocol = string
          rule_number = number
          rule_action = string
          cidr_block = string
          from_port = number
          to_port = number
          })), [{
          rule_number = 100
          protocol = -1
          rule_action = "allow"
          cidr_block = "0.0.0.0/0"
          from_port = 0
          to_port = 0
        }])
      })
      tags = optional(map(string), {})
    }))
    security_group = optional(object({
      ingress_rules = optional(list(object({
        description = string
        from_port = number
        to_port = number
        protocol = string
        cidr_block = string
        security_group_id = string
      })), [])
      egress_rules = optional(list(object({
        description = string
        from_port = number
        to_port = number
        protocol = string
        cidr_block = string
        security_group_id = string
      })), [])
    }), {})
  }))
}

```

**vpc\_endpoint\_map (flattened)**

```auto
vpc_endpoint_map = flatten([
    for vpc in local.vpcs : [
      for subnet in vpc["subnets"] : [
        for vpc_endpoint in subnet["endpoints"] : {
          vpc_endpoint_type = vpc_endpoint["vpc_endpoint_type"]
          private_dns_enabled = vpc_endpoint["private_dns_enabled"]
          service_name = vpc_endpoint["service_name"]
          resources = vpc_endpoint["resources"]
          dns_name = vpc_endpoint["dns_name"]
          subnet_tag_filters = { type = subnet["type"], identifier = subnet["name"] }
          ingress_cidr_blocks = vpc_endpoint["ingress_cidr_blocks"]
          vpc_id = module.vpc_idmz[vpc["name"]].id
          vpc_name = vpc["name"]
          vpc_network_zone = vpc["network_zone"]
          vpc_network_function = vpc["network_function"]
          subnet_name = subnet["name"]
          route_table_ids = module.vpc_idmz[vpc["name"]].route_table_ids
        }
      ]
    ]
  ])

```

Now, the first run creates fine with the following inputs:

```auto
vpc_idmz = [
  {
    name = "vpc-1"
    cidr_block = "10.32.0.0/16"
    subnets = [
      {
        name = "subnets-1"
        type = "private"
        cidr_blocks = ["10.32.4.0/22"]
        create_nat_gateway = false
        create_route_to_internet_gateway = false
        endpoints = [
          {
            service_name = "ec2"
          }
        ]
        nacl = {
          ingress_rules = []
          egress_rules = []
        }
      }
    ]
  }
]

```

…however, when i run the second time with an additional VPC object defined in the `vpc_idmz` input, all the VPC endpoint resources (endpoint, policy, security group) for the endpoint in the first VPC want to recreate.

**New VPC input with second VPC defined:**

```auto
vpc_idmz = [
  {
    name = "vpc-1"
    cidr_block = "10.32.0.0/16"
    subnets = [
      {
        name = "vpc-subnet-1"
        type = "private"
        cidr_blocks = ["10.32.4.0/22"]
        create_nat_gateway = false
        create_route_to_internet_gateway = false
        endpoints = [
          {
            service_name = "ec2"
          }
        ]
        nacl = {
          ingress_rules = []
          egress_rules = []
        }
      }
    ]
  },
  {
    name = "vpc-2"
    cidr_block = "10.38.0.0/16"
    subnets = [
      {
        name = "vpc-subnet-2"
        type = "private"
        cidr_blocks = ["10.38.4.0/22"]
        create_nat_gateway = false
        create_route_to_internet_gateway = false
        endpoints = [
          {
            service_name = "ec2"
          }
        ]
        nacl = {
          ingress_rules = []
          egress_rules = []
        }
      }
    ]
  }

```

**Resources showing recreation:**

```auto
  # module.vpc_endpoint["ec2-vpc-1"].data.aws_vpc.selected will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_vpc" "selected" {
      + arn = (known after apply)
      + cidr_block = (known after apply)
      + cidr_block_associations = (known after apply)
      + default = (known after apply)
      + dhcp_options_id = (known after apply)
      + enable_dns_hostnames = (known after apply)
      + enable_dns_support = (known after apply)
      + enable_network_address_usage_metrics = (known after apply)
      + id = (known after apply)
      + instance_tenancy = (known after apply)
      + ipv6_association_id = (known after apply)
      + ipv6_cidr_block = (known after apply)
      + main_route_table_id = (known after apply)
      + owner_id = (known after apply)
      + state = (known after apply)
      + tags = {
          + "Name" = "vpc-1"
        }

      + timeouts {
          + read = (known after apply)
        }
    }

  # module.vpc_endpoint["ec2-vpc-1"].aws_security_group.endpoint_security_group[0] must be replaced
-/+ resource "aws_security_group" "endpoint_security_group" {
      ~ arn = "arn:aws:ec2:eu-west-1:658084889611:security-group/sg-074d6ba2af5c6b996" -> (known after apply)
      ~ egress = [] -> (known after apply)
      ~ id = "sg-074d6ba2af5c6b996" -> (known after apply)
      ~ ingress = [] -> (known after apply)
        name = "ec2-vpce"
      + name_prefix = (known after apply)
      ~ owner_id = "658084889611" -> (known after apply)
        tags = {
            "Name" = "ec2-access"
        }
      ~ vpc_id = "vpc-0e9ab532cee74b900" -> (known after apply) # forces replacement
        # (3 unchanged attributes hidden)
    }

```

So it looks like the `aws_vpc` data source in the VPC Endpoint module is not being evaluated at the correct time in order for the above to work. My question is…how can i correct this?

I’ve tried everything from using different `depends_on`, adding a data source to the level this is being applied from and trying to force a dependency loop, outputting the inputs from the VPC modules and using those in the local variable to force the correct ordering…no luck with anything!

Apologies for the long post!

---

<div class="post-metadata">

**Author:** ![jbardin](https://sea2.discourse-cdn.com/hashicorp/user_avatar/discuss.hashicorp.com/jbardin/32/3325_2.png) [@jbardin](https://discuss.hashicorp.com/u/jbardin)\
**Post date:** [August 21, 2023, 2:13pm UTC](https://discuss.hashicorp.com/t/module-resources-recreated-on-every-run-dependency-ordering-issue-help/57465/2 "2023-08-21T14:13:45Z")

</div>

Hi @lambbuster,

The key part of the output here is the reason for the data source being deferred: `depends on a resource or a module with changes pending`. This means you have told Terraform that the data source cannot be used if there are any changes pending in its dependencies, which according to the configuration, includes everything within `module.vpc_idmz`.

Adding `depends_on` should be done judiciously only when it’s required, and removing it from the `vpc_endpoint` module call will probably help here.

---

<div class="post-metadata">

**Author:** ![lambbuster](https://sea2.discourse-cdn.com/hashicorp/user_avatar/discuss.hashicorp.com/lambbuster/32/37502_2.png) [@lambbuster](https://discuss.hashicorp.com/u/lambbuster)\
**Post date:** [August 21, 2023, 2:51pm UTC](https://discuss.hashicorp.com/t/module-resources-recreated-on-every-run-dependency-ordering-issue-help/57465/3 "2023-08-21T14:51:20Z")

</div>

@jbardin - thank you for your reply. I totally agree that this is the issue, and i’ve normally been able to resolve similar situations.

However, if i remove the `depends_on` from the `vpc_endpoint` module, i get an error from the `aws_vpc` data source in question stating: `Error: no matching EC2 VPC found`…meaning without the dependency the data source is attempting to be evaluated before the VPC is created.

Is there a different way i can ensure the ordering without the `depends_on` ?

Thanks,  
Phil

---

<div class="post-metadata">

**Author:** ![jbardin](https://sea2.discourse-cdn.com/hashicorp/user_avatar/discuss.hashicorp.com/jbardin/32/3325_2.png) [@jbardin](https://discuss.hashicorp.com/u/jbardin)\
**Post date:** [August 21, 2023, 3:08pm UTC](https://discuss.hashicorp.com/t/module-resources-recreated-on-every-run-dependency-ordering-issue-help/57465/4 "2023-08-21T15:08:06Z")

</div>

So it sounds like the vpc names are not derived from computed values of actual vpc resources, which means that Terraform cannot ensure the correct ordering without the `depends_on` given the current configuration.

To avoid this type of error, you should not have a managed resource and a data source representing the same logical resource within the same configuration. Because the `vpc` is being created within the same configuration, the data from that vpc should be fed directly into the module, rather than relying on the data source to lookup the vpc again.

If for some reason the overall structure of the modules cannot be fixed, the workaround would be to ensure that a _computed_ value from the vpc managed resource is fed into the `vpc_endpoint` module, and have the `aws_vpc` data source explicitly depend only on that value.

---

<div class="post-metadata">

**Author:** ![lambbuster](https://sea2.discourse-cdn.com/hashicorp/user_avatar/discuss.hashicorp.com/lambbuster/32/37502_2.png) [@lambbuster](https://discuss.hashicorp.com/u/lambbuster)\
**Post date:** [August 29, 2023, 3:24pm UTC](https://discuss.hashicorp.com/t/module-resources-recreated-on-every-run-dependency-ordering-issue-help/57465/5 "2023-08-29T15:24:32Z")

</div>

@jbardin - thank you, this solved the issue! I was so close with troubleshooting but missed one small part of the module data source. Closing this gap and removing the depends\_on at module level fixed the ordering. Cheers!