Data Sources not being read during Plan

[tschmidt]

The issue I have is similar to the one here but poster did not follow up.

Data sources forcing replacement even though nothing has changed

I’ve experienced issues with using data sources. In particular, within Azure. Some resources need, for instance vnet IDs to be passed to them. However, because of this, every time I run an apply, the resource “must be replaced”, because the vnet ID is “known after apply”. So, even though nothing has changed, things must be replaced. I wish that when an apply is run, any data sources would be read, and anything that uses those values, would be updated, and checked against state, because, resource…

So I am creating a lot of aws sso account assignments, and I am using data sources for pretty much all of it in order to avoid hard coding arns, etc. I am trying to get some understanding of how to avoid the recreation of resources when the data sources are not being read during the plan.

This is a pretty far off adaptation of some cloud posse code; here is my fork with my branch: GitHub - tschmidty69/terraform-aws-sso at ts-changes

The biggest issue is that it recreates all the account assignments. One solution of course is to go back to hardcoding sso instances, principal arns and what not. Ignore_changes is not a solution of course.

But I just want to understand if there is anything I can do to get these values known before apply.

My plan will say things like this:

# module.sso_account_assignments.data.aws_identitystore_group.this["SOME_GROUP"] will be read during apply
  # (config refers to values not yet known)
 <= data "aws_identitystore_group" "this" {
      + description       = (known after apply)
      + display_name      = (known after apply)
      + external_ids      = (known after apply)
      + group_id          = (known after apply)
      + id                = (known after apply)
      + identity_store_id = (known after apply)
      + alternate_identifier {
          + unique_attribute {
              + attribute_path  = "DisplayName"
              + attribute_value = "SOME_GROUP"
            }
        }
    }
  # module.sso_account_assignments.data.aws_ssoadmin_instances.this will be read during apply
  # (depends on a resource or a module with changes pending)
 <= data "aws_ssoadmin_instances" "this" {
      + arns               = (known after apply)
      + id                 = (known after apply)
      + identity_store_ids = (known after apply)
    }

[tschmidt]

I found that this behavior is only when the permission sets are assigned. I am guessing because of the depends_on clause.

[apparentlymart]

Hi @tschmidt,

In the code for this module I see that the second data resource has a totally empty configuration:

data "aws_ssoadmin_instances" "this" {}

A totally-empty data block can’t create any dependencies on other resources itself, and so indeed the only explanation I can think of here is that you’ve written a module block calling this module which includes a depends_on argument, and therefore you’re telling Terraform that it must complete changes to some other thing in your broader configuration before it can take actions for anything inside this module, including this data resource.

Using depends_on in a module block should be considered as a last resort, because it creates a very conservative dependency graph. Instead, I suggest describing your dependencies more precisely using normal reference expressions.

If you’re not sure what I mean by that, please share the configuration of your own module that’s calling this one (the one containing the module block with depends_on) and then I can try to make some more specific recommendations.

[tschmidt]

@apparentlymart

In the code for this module I see that the second data resource has a totally empty configuration:

data "aws_ssoadmin_instances" "this" {}

A totally-empty data block can’t create any dependencies on other resources itself

That makes sense (but stack overflow said to do it that way /s).

So an 11pm guess is that grabbing that in the main.tf and passing the sso instance to the module(s) would help there?

@apparentlymart

Using depends_on in a module block should be considered as a last resort, because it creates a very conservative dependency graph.

So this was to try to work around the fact that it was trying to do account assignments before creating the permission set (if I recall depends_on didn’t actually help). I have been getting around that by doing perms then assignments, but it seems like maybe this is a “bug” in the terraform dependency graph (ie, accounts assignments are not dependent on perm sets)? I have not dug into the details of the dependency graphs at all, but would that be the expectation? (ie. acc ass should depend on perm sets?)

@apparentlymart I really appreciate you taking the time to answer these questions and you do a really good job explaning the intricacies which is exactly what I was looking for here. Your answers on this forum have helped me quite a bit so thank you sincerely.