Dependent module and data source

Terraform
1

Hello,

we have two modules, moduleA and moduleB. ModuleB has an implicit dependency on moduleA with input of all moduleA outputs. ModuleB also has a data source inside.
Our problem is that if there is a change in moduleA, the data source in moduleB is read during apply and causes constant diff in plan. Is there a way to get rid of that? The data source is this one (we use it instead of literal json policy on the resource

module moduleA {}
module moduleB {
  moduleA = module.moduleA
}
2

Hi @Crusad,

Without seeing the specific details I can only make a general statement, which is that generally the answer is no: if your data resource configuration depends on something that won’t know known until apply time then the data resource must always be deferred until apply time.

With that said, it’s often possible to avoid data resources depending on managed resources by refactoring so that your modules receive the data they need as inputs, rather than fetching that data themselves. Under that design pattern, if another module in your configuration is the one managing a particular object then you can pass a subset of the resulting managed object into the other module, rather than just passing the identifier and forcing the module to do the lookup itself. This is described in the Terraform guide Module Composition, particularly in the opening section and in the “Dependency Inversion” sub-section.

If you can show a little more of what you’ve tried then I might be able to give a more concrete example of what an dependency injection style would look like for your case, if indeed dependency injection is applicable to your situation.

3

Hi @apparentlymart,

thank you for the response.

This is the data source we have inside module operational (from my first post moduleB)

    data aws_iam_policy_document cert-manager {
      statement {
        actions = [
          "sts:AssumeRole"
        ]
        resources = [
          aws_iam_role.cert-manager[0].arn
        ]
      }
      statement {
        actions = [
          "route53:GetChange"
        ]
        resources = [
          "arn:aws:route53:::change/*"
        ]
      }
      statement {
        actions = [
          "route53:ChangeResourceRecordSets"
        ]
        resources = formatlist("arn:aws:route53:::hostedzone/%s", distinct(concat([ for domain, v in var.domains : v.id ], [ var.operational_dns_id ])))
      }
      statement {
        actions = [
          "route53:ListHostedZones",
          "route53:ListResourceRecordSets"
        ]
        resources = [
          "*"
        ]
      }
    }

This resource doesn’t depend on anything in module cluster, the whole module operational uses module cluster as an input and then inside the module we do select some variables but not in the resource that shows diffs (configuration reduced to only important parts be more readable):

module operational {
for_each                            = local.clusters_indexed
cluster                             = module.moduleA[each.key]
domains                         = local.domains
operational_dns_id       = local.operational_dns_id
}

Module cluster does output quite a lot of values and when only a description in one of the resources that have their AWS ARN exported change, the data source shows diff and it’s read during apply.
For example I was changing description on this resource in module cluster which triggers reread of the data source in module operational:

output logging_role_arn {
  value = aws_iam_role.logging_operational[*].arn
}
4

Hi @Crusad,

The situation you’ve described here is one where the general advice in my previous comment would not apply, because aws_iam_policy_document is a funny sort of data source that doesn’t actually fetch anything from the remote system at all, and instead just encapsulates some local work to produce a JSON IAM policy.

My new question seeing this is to wonder which of the inputs into that data resource are ending up changing. You should be able to see that based on which values show as the (known after apply) placeholder in the plan output. If you’re not sure and are able to share the full plan output I can probably help map that back.

If the value that’s changing really is something that can only be known at apply time then there likely won’t be anything practical you can do about it… it’s just a consequence of how you’ve designed your system. However, if it’s something that ought to be known at plan time then there might be some way to make it be known, or it might suggest a feature request to the AWS provider to produce a more complete plan that includes the value that ought to be known at plan time.