Terraform hides static default_tags values from provider when var.xxxx are used

Terraform
1

Hi TF forum

I am trying to create some policies on openPolicyAgent related to Terraform so I need to convert the plan into a plan json which is created with the terraform show -json command.

If I have this on the providers.tf file:

provider "aws" {
  default_tags {
    tags = {
      BusinessUnit      = "Risk"
      Tribe             = "Cash Management"
      Project           = "risk-api"
      CostCenter       = "123456"
    }
  }
}

I get the below json

"configuration": {
    "provider_config": {
      "aws": {
        "name": "aws",
        "full_name": "registry.terraform.io/hashicorp/aws",
        "version_constraint": "~> 5.70.0",
        "expressions": {
          "default_tags": [
            {
              "tags": {
                "constant_value": {
                  "BusinessUnit": "Risk",
                  "CostCenter": "123456",
                  "Project": "risk-api",
                  "Tribe": "Cash Management"
                }
              }
            }
          ]
        }
      }
    },

But the monent I put in the defaults_tags a variable sustitution (var.xxxxx)

provider "aws" {
  default_tags {
    tags = {
      BusinessUnit      = "Risk"
      Tribe             = "Cash Management"
      Project           = "risk-api"
      CostCenter       = "123456"
      Dop              = var.dop_tag
    }
  }
}

I just get below json conversion … as you see now the static values are hidden and only the var.xxxxxx is shown.

    "provider_config": {
      "aws": {
        "name": "aws",
        "full_name": "registry.terraform.io/hashicorp/aws",
        "version_constraint": "~> 5.70.0",
        "expressions": {
          "default_tags": [
            {
              "tags": {
                "references": [
                  "var.dop_tag"
                ]
              }
            }
          ]
        }
      }
    },

The static values will ONLY be seen on the resources you create on the tags_all section (together with the dynamic value), so there is only one place the static values can be check.

I would love to get some advice …. What I am looking for is to enforce users to add specific tags (CostCenter) to their default_tags (so they are sync to ALL the resouces they create - and we can charge them on those resource created). If static values are hidden (we have some other dynamic values added to the default_tags) I can never know if the CostCenter was added to the default_tags unless I check specific resources which they might be or might not be deploy …. so it is a bit awkards just the sudden hidden of the static values when var.xxxx is use in the default_tags.

Sentinel is out of the question since open policy agent was the tchnology that was choosen.

Many Thanks

Jo

2

Hi @joaquin386,

The “configuration” stored in the plan file is a snapshot of the actual literal configuration, it does not contain any of the values as they were later evaluated.

A provider’s configuration value is ephemeral in nature, and not stored in the plan file or state at all.

3

The requirement is to analyze the plan file (no state) and what is going to change, before the actual change.

I am looking for alternative but above I did show how depending on what you put on the providers default_tags (aws) configuration it changes the behaviour of the default_tags object. I had it working but then I introduce the var.xxxx and the object change completely not showing anymore the static values.

Even if ephemeral it shows its configuration in the plan file.

OPA can not check plain text files but it can JSON/YAML files so there is a need to have what is going to be applied (the plan) and analize its behaviour and if not good (because it is breaking policy) then reject it.

But I think I can iterate through the planned objects and check if tag in tags_all then it should have been set via default_tags.

4

Yes, it only shows the literal configuration. It may be in a json format (albeit in a somewhat limited format, so it’s not really possible to evaluate), but it can’t give you any more information than looking at the HCL file text for that block. Provider configurations are not stored, so cannot be inspected from the plan file.

1 Like
5

I think probably the most robust approach here would be to apply your policy rules to the resource instance configurations themselves, instead of to the provider configuration that’s currently being used as part of the resource instance configurations.

That would mean you could use the “Values Representation” instead of the configuration representation, and therefore you can inspect the dynamic results of expression evaluation instead of inspecting just the static configuration (which is what that "provider_config" property belongs to).

For example, you could visit each of the resources from the plan JSON in turn, skipping over any that aren’t associated with an instance of the hashicorp/aws provider, and then implement a rule like: if there’s a tags_all attribute then it must include at least this set of required keys and values.

This would achieve essentially the same effect – making sure that any objects that are created will meet your organization’s tag naming convention – but describe the rule in terms of the planned new state (after expressions have been evaluated and the provider’s planning logic has run) instead of the static configuration.

One trick with the above is that it depends on the hashicorp/aws provider correctly populating the tags_all argument during the planning phase, which I’ve not confirmed that it does.

It’s possible that the provider just leaves tags_all marked as “known after apply” during the planning phase during initial creation, in which case unfortunately there isn’t really any good way to check the values based only on the plan because those values would not actually appear as part of the plan that the provider returned.

Hopefully the provider knows how to populate tags_all more precisely during planning though, since it ought to have all of the necessary information available to do it. It’s just a matter of whether the planning logic in the provider actually makes use of that information.

6

Thanks for the reply. I went as you mention for the resource tags_all which I think is more consistant than looking at the provider json data.

If this would help anyone here is the OPA Rule enforcing tags with the terraform plan json as input

Data:

{"tags_enforcement": {"CostCenter": ["123456", "654321"], "Dummy": ["DummyValue1", "DummyValue2"]}}

Rego File.

package terraform.default_tags

import future.keywords.in
import input.planned_values.root_module.resources as resources

default allow = true

required_tags := object.keys(data.tags_enforcement.tags_enforcement)

information_url := object.get(data.tags_enforcement, ["information", "url"], "https://my-documentation.for.help")

# Check provider default_tags tag
deny contains msg if {
    some required_tag in required_tags
    tag_resource := object.get(resources[_].values.tags_all, required_tag, "undefined")
    tag_resource == "undefined"

    msg := {
        "code": "AWS_DEFAULT_TAGS_MISSING_REQUIRED_TAG",
        "message": sprintf("AWS provider default_tags is missing required tag: %v", [concat(", ", [required_tag])]),
        "information": sprintf("Check following url: %v", [information_url]),
    }
}

# Check Required tag values
deny contains msg if {
    some required_tag in required_tags

    tag_resource := object.get(resources[_].values.tags_all, required_tag, "undefined")
    not tag_resource in data.tags_enforcement.tags_enforcement[required_tag]

    msg := {
        "code": "AWS_DEFAULT_TAGS_INVALID_TAG_VALUE",
        "message": sprintf("Invalid tag found: '%v=%v'. Valid values are: %v", [
        required_tag,
        tag_resource,
        concat(", ", sort(data.tags_enforcement.tags_enforcement[required_tag]))]),
        "information": sprintf("Check following url: %v", [information_url]),
    }
}