Terraform Test - Mock/Override a data source created with a for_each loop

Terraform
1

Hello,

I am trying to leverage the terraform test command to add tests to a module and I want to use the 1.7.0 mocks to avoid connecting to an AWS account to run the tests.

I am trying to mock data sources but I am facing issues to mock a data source instanced with a for_each loop:

The data source in my module looks like:

data "aws_subnet" "private_zone_1" {
  for_each = toset(data.aws_subnets.private_by_zone[0].ids)
  id       = each.value
}

And I am alos defining a local variable based on this data source:

locals {
  # Find the first subnets in each zone with enough space to fit an NLB.
  selected_public_subnet_zone_1 = [for s in data.aws_subnet.public_zone_1 : s if s.available_ip_address_count >= 8][0]
}

I tried to mock the aws_subnet resource with mock_data block, but this does not work and fails with:

 Error: Invalid index
│ 
│   on main.tf line 4, in locals:
│    4:   selected_public_subnet_zone_1 = [for s in data.aws_subnet.public_zone_1 : s if s.available_ip_address_count >= 8][0]
│     ├────────────────
│     │ data.aws_subnet.public_zone_1 is object with 2 attributes
│ 
│ The given key does not identify an element in this collection value: the collection has no elements.

I understand that it is a data source with multiple instances and I am wondering if mocking or overriding (with an override_data) this kind of resource is already supported.

The docs do not mention anything like that: Tests - Provider Mocking | Terraform | HashiCorp Developer

2

Hi @henri.lefevre,

This error suggests to me that there isn’t any instance of this data resource that meets the filtering condition s.available_ip_address_count >= 8. Therefore the for expression produces a zero-length tuple and so the [0] index fails (there is no element zero in an empty tuple).

Does your mock arrange for at least one of the results to meet that condition?

3

@apparentlymart I created a mock resource satisfying the condition, but as the data source is created in a for_each loop, the terraform test commands sees 0 instances.

I assume I should use an override instead of a mock but I have no idea about how to write that.

I checked Terraform sources/tests/docs and I did not find anything similar

Would you know how I could write the mock to remove that error?

My goal for now is to test variable validation so this part of the module just needs to be silenced with a mock

4

Hi @henri.lefevre,

The concept of for_each lives in the Terraform language runtime rather than in the provider itself, and so it should behave the same way for mocked providers as it does for real providers.

All a provider knows is that it’s being asked to plan a set of resources of a particular type; the provider can’t actually tell whether that’s caused by there being multiple resource blocks, a single resource block with for_each, a single resource block with count, multiple instances of the same module, etc etc… those are all concerns of the main Terraform language that providers don’t participate in.

The error message that you shared includes the following hint:

data.aws_subnet.public_zone_1 is object with 2 attributes

A resource (either resource or data blocks) with for_each appears in expressions as an object with one attribute per element of the for_each collection, and so this hint suggests that the for_each part worked as expected – assuming your data.aws_subnets.private_by_zone[0].ids value is a two-element list or set – but then the for expression filtered out both of those elements, producing an empty tuple.

I’m afraid I’m not sure what to suggest concretely with the information given, but it doesn’t seem to me that the for_each usage is the root of the problem here.

5

Thanks for the support, I finally solved this by overriding data.aws_subnets.private_by_zone[0] or data.aws_subnets.public_by_zone[0] (sorry for the mix in the initial message between public and private resources even though those behave the same way) instead of data.aws_subnets.private_by_zone or data.aws_subnets.public_by_zone

The missing bit was the possibility to override an item using its index, my unit tests are now passing, thanks for adding this great feature in 1.7 :slight_smile: