Produce list of strings from list of objects with experimental optional attributes

Terraform
1

Hi,

I have been trying to → Produce list of strings from list of objects with experimental optional attributes.

Consider the variable below :

variable host_rule_set {
  type = list(object({
    h = string
    a_h = optional(list(string))
    lb  = string
    iap    = string
    int_lb = string
    gke    = optional(string)
    cert = optional(string)
  }))
}

Now we have a tfvar file which defines the host_rule_set var like below :

host_rule_set = [
  {
    h = "abc1.xyz.com"
    lb = "false"
    iap = "false"
    int_lb = "true"
    gke = "bln"
  },
  {
    h = "abc2.xyz.com"
    a_h = ["abc2.xyz.dev","abc2.xyz.qa"]
    lb = "false"
    iap = "false"
    int_lb = "true"
    gke = "nlb"
  },
...
]

i wish to collate ONLY the list of strings of attribute “a_h” in each of the above items into a single list but have been largely unsuccessful.

Please could some one help me understand if its atleast possible to traverse optional attributes in a list of objects and retrieve their values ?

I tried using the merge + … expansion operator as mentioned in these discussions :

Also tried with nested for loops like mentioned here :

Have even tried the lookup() & matchKeys() function by passing the variable as a map and trying to lookup the [“a_h”] as the search set.

2

terraform version 0.14.11

3

Hi @shrikant.rajappan,

I think the main new situation in your case compared to some other examples discussed here before is that some of the values for this attribute will be null because the attribute is optional. Therefore you’ll need to filter out those null values before trying to combine the remaining values.

[for r in var.host_rule_set : r.a_h if r.a_h != null]

The result of that should be a list of lists ignoring any of the items that had a null list. You could then use the flatten function to flatten that into a single list of strings.

4

Thank you so much for your reply @apparentlymart ! you are a rockstar !!

I am able to retrieve the values now!

I had initially tried to filter out the nulls but the terraform plan errors out saying
" var.host_rule_set is list of object with 323 elements
This value does not have any attributes."

As a workaround we can tweak this to reflect unique hosts instead of that list of strings as well but thats too much duplication of code too as against the DRY principle.

But thats because of the paradox - the optional attributes are only supported within the context of the objects. However our definition does not comply to the =.

As informed we are tweaking the definition to have a more uniform & unique key and values as a work around.

Also could you advise from what terraform version have optional attributes part of alpha or stable ?

Thanks,
Shrikant R

5

Hi @shrikant.rajappan,

The current optional attributes experiment will not become stable in its current form. We are currently working on a new design which will be incompatible with the current experiment but should be easier to use for common situations where we’ve heard feedback from those who tried the current experiment.

6

Got it @apparentlymart we will await this feature support.

Thanks and regards,
Shrikant R

7

Hi @apparentlymart ,

I had a follow up query if its ok OR i can post in a new topic.

Now since we know from the above code that the host_rule_set is a list of objects , it was earlier defined as simply a list or list(any).

In my terraform plan i can see that the host rules for the GLB resource are being completely rebuilt OR repopulated even when the A records themselves are not being destroyed and created again because the values of urls have not changed.

I wanted to understand the annals of how this change is propagated ?

Does the change in definition of a variable (like host_rule_set which is further used downstream by lot of resources like the GLB definition for dynamic host rule mapping etc.,) entail in first deletion and then repopulating of the whole list of host rules of GLB ?

I understand that terraform works on a dependancy graph of resources - is the behaviour related to such a DAG ? Referred to below issue clarification you provided :

Again the only change is in the definition of the topmost terraform variable which holds the set of hosts. The rest of the code (modifications have been made in how to process this variable) is the same but we are a bit nervous before APPLY of changes since it seems like lot of url’s are deleted and then added again in the dynamic host_rules .

Also there is no change in the DNS A record by the providers’ resoource google_dns_record_set because I assume the hostnames are ultimately url strings and unless the name of the host is modified OR if there are additions/deletions only then the a record will be updated - please correct my understanding if otherwise.

Thanks and regards,
Shrikant R

8

Hi @shrikant.rajappan,

Could you share an example output from terraform plan showing the proposed changes you are concerned about? There are a few different things that could be happening here but I can’t guess which one applies to your situation without seeing what exactly Terraform is proposing to replace.

Thanks!

9

Thank you for looking into this @apparentlymart

Here is a snippet of the terraform apply logs :

# google_compute_url_map.default will be updated in-place
  ~ resource "google_compute_url_map" "default" {
.........
- host_rule {
          - hosts        = [
              - "*.mno-dev.abc.digital",
              - "*.pcca-staging.abc.com",
              - "portal-internal.abc.com",
              - "audit-exp-dev.abc.com",
              - "audit-exp.abc.com",
              - "axep-lower.abc.com",
              - "capacity-mgmt-prod.abc.com",
              - "capacity-mgmt--dev.abc.com",

.........

+ host_rule {
          + hosts        = [
              + "*.mno-dev.abc.digital",
              + "*.pcca-staging.abc.com",
              + "portal-internal.abc.com",
              + "audit-exp-dev.abc.com",
              + "audit-exp.abc.com",
              + "axep-lower.abc.com",
              + "capacity-mgmt-prod.abc.com",
              + "capacity-mgmt--dev.abc.com",

.........

Seems like there is an in-place update for each hostname eventhough there aren’t any new additions.

10

Hi @shrikant.rajappan,

From looking in the provider code itself I can see that the provider’s schema declares this host_rule block as representing a set of objects. In a set the elements are identified directly by their content, and so there is no sense of an “in-place update” of a set element – instead, Terraform understands any change to a host_rule block as being the removal of one block and the addition of another.

Although I can see from the snippet you shared that hosts itself isn’t changing, this would happen for any change to arguments inside the host_rule block, which also includes the path_matcher and description arguments. Is Terraform reporting any changes to those? My first guess here would be that either the provider itself or the remote API is subtly changing something about these values as a normalization rule, but the provider is lacking a rule to classify that change as normalization and so it’s instead treating it like a real change and proposing to restore it “back” to the non-normalized value you wrote in the configuration.

Unfortunately at this point this is more of a provider-specific behavior and so I’m less familiar with this behavior than I am with Terraform Core. If what I’ve described above doesn’t explain the behavior then it might be better to start a new topic in the provider’s own community forum, where the participants are generally more familiar with the specific details of this provider and underlying API.

11

Thank you so much for the detailed reply @apparentlymart !

You have rightly inferred that there are changes reported only in the host_rule block.

I will also post on the provider specific community for delving more into the dynamic object management works.