Override_data in tests not able to override Block attributes?

Hi,

I’ve been battling with a data resource that I’ve been trying to override in a unit test with override_data without success. However other data resources I could override without any issue.

When looking at the implementation, the only difference I could notice is that the said data resource was implemented with a Block attribute. Could that be the issue here? Or do you have any other explanation as to why this cannot be overridden?

The data resource that I could not override: datadog_teams - terraform-provider-datadog/datadog/fwprovider/data_source_datadog_teams.go at master · DataDog/terraform-provider-datadog · GitHub
Other data resource that I could override in the same test:

Using terraform 1.13.4

Thanks!

Hi @IppX, thanks for posting! Your intuition about the block definition causing problems is correct. This is mentioned in passing here, but with a focus on resources rather than data sources.

Essentially, the issue is that blocks (unlike attributes) are not directly marked as computed, and so the test framework will not create them directly. There’s definitely a focus on resources rather than data sources here, because I think what you’re describing sounds like a valid use case. I’d encourage you to file an enhancement request within the issues list in the Terraform repository and describe your use-case there.

One interesting workaround I’ve noticed while investigating this is you can force the data source to return exactly one kind of block. Terraform will populate blocks that are present in the configuration during test, and during plan and apply those empty blocks with only computed values are kind of ignored. So, you can do something like this:

# main.tf

terraform {
  required_providers {
    datadog = {
      source = "datadog/datadog"
    }
  }
}

data "datadog_teams" "teams" {
  # empty teams block is ignored during regular processing but will be 
  # processed by terraform test.
  teams {}
}

output "teams" {
  value = [
    for team in data.datadog_teams.teams.teams : team.id
  ]
}

Then in your test:


mock_provider "datadog" {}

override_data {
  target = data.datadog_teams.teams
  values = {
    teams = {
        id = "my_single_id"
    }
  }
}

run "test" {
    assert {
        condition = output.teams == ["my_single_id"]
        error_message = "bad values"
    }
}

In this case, the block that exists will be populated by the values that are within the override_data block - as the block is written in the config. You can even specify multiple blocks, and each block you specify will be populated by the same values from the override data.

The thing to bear in mind is that, for blocks, you’re not specifying the exact number of blocks or values for each block but a general set of values for Terraform to use for each block that is there. Blocks are complicated internally in Terraform, which is why we have this very simplified approach for mocking them - with a focus on mocking the values for resources rather than data sources as the underlying logic is currently shared.

I’m not saying you should use this approach, as leaving empty blocks in data sources as hints for the testing framework is definitely not the Terraform approved strategy. But, it may be a workaround in the short term since there is definitely an absence of an actual approved strategy for this use case.

As mentioned above, it would be great if you file this as an enhancement and then it can added into our prioritisation and might get worked on one day. I definitely see a path forward where we can separate the logic for mocking data sources and resources so data sources can act more as data sources should.

Thanks!

1 Like

Thanks a lot for the detailed explanation and the little hack. That solved my issue.
I submitted a feature request here: [Feature request][test]: Allow overriding Block in `override_data` · Issue #37862 · hashicorp/terraform · GitHub

Interestingly, the addition of teams {} would mean it’s technically not valid for the provider to return more data in that block, because the configuration already declared that there is exactly one object in the list. This gets missed in validation probably due to the exceptions needed for legacy providers with various misbehaving blocks.