Conditional object with concat result expressions must have consistent types

[jrobison-sb]

My goal is to define a container definition which has a “main” container like nginx or apache, and it optionally has multiple sidecar containers. The sidecar containers are re-used by a bunch of different services, so I’m defining those as a local variable which I can reuse as many times as needed. Also, the sidecar containers need to be conditionally enabled, such that we can use sidecars in prod but not use them in dev, or vice versa, etc.

Here is a super minimal reproducable example using apache as a “main” container and conditionally concatting the sidecars into the definition:

variable "use_generic_sidecars" {
  default = true
}

locals {
  # These generic sidecars are conditionally reused by a bunch
  # of different services, so we define them here as their own
  # variable and we re-use that variable in various other
  # container definitions.
  generic_sidecars = [
    {
      name      = "sidecar1"
      image     = "sidecar:latest"
      essential = false
    },
    {
      name      = "sidecar2"
      image     = "sidecar:latest"
      essential = false
    }
  ]
  apache = concat([
    {
      name      = "apache"
      image     = "httpd:latest"
      essential = true
      portMappings = [
        {
          containerPort = 80
          hostPort      = 80
        }
      ]
    }
    ],
    var.use_generic_sidecars ? local.generic_sidecars : [],
  )
}

resource "aws_ecs_task_definition" "generic_service" {
  family                = "service"
  container_definitions = jsonencode(local.apache)
}

I can use terraform plan and this plans as expected. It will build a task definition which has 3 total containers. 1 apache container plus two sidecars which are brought it by way of concat(). This suggests to me that the general method I’m using for composing this resource is valid.

  # aws_ecs_task_definition.generic_service will be created
  + resource "aws_ecs_task_definition" "generic_service" {
      + arn                   = (known after apply)
      + arn_without_revision  = (known after apply)
      + container_definitions = jsonencode(
            [
              + {
                  + essential    = true
                  + image        = "httpd:latest"
                  + name         = "apache"
                  + portMappings = [
                      + {
                          + containerPort = 80
                          + hostPort      = 80
                        },
                    ]
                },
              + {
                  + essential = false
                  + image     = "sidecar:latest"
                  + name      = "sidecar1"
                },
              + {
                  + essential = false
                  + image     = "sidecar:latest"
                  + name      = "sidecar2"
                },
            ]
        )
      + family                = "service"
      + id                    = (known after apply)
      + network_mode          = (known after apply)
      + revision              = (known after apply)
      + skip_destroy          = false
      + tags_all              = (known after apply)
      + track_latest          = false
    }

Plan: 1 to add, 0 to change, 0 to destroy.

But when I try the same method with a more real-world example like using datadog sidecar containers, it doesn’t work. Here is a reproducable example using nginx as the “main” container, plus two datadog containers with the same conditional concat method:

variable "use_datadog" {
  default = true
}

locals {
  # These datadog sidecars are conditionally reused by a bunch
  # of different services, so we define them here as their own
  # variable and we re-use that variable in various other
  # container definitions.
  datadog_containers = [
    {
      # https://docs.datadoghq.com/integrations/ecs_fargate/?tab=cloudformation#aws-cloudformation-task-definition
      name      = "datadog-agent"
      image     = "public.ecr.aws/datadog/agent:latest"
      essential = false
      cpu       = null
      memory    = null
      environment = [
        {
          name  = "ECS_FARGATE"
          value = "true"
        },
        # https://docs.datadoghq.com/security/cloud_security_management/setup/csm_pro/agent/ecs_ec2/
        {
          name  = "DD_CONTAINER_IMAGE_ENABLED"
          value = "true"
        },
        {
          name  = "DD_SBOM_ENABLED"
          value = "true"
        },
        {
          name  = "DD_SBOM_CONTAINER_IMAGE_ENABLED"
          value = "true"
        },
      ]
    },
    {
      name      = "cws-instrumentation-init"
      image     = "datadog/cws-instrumentation:latest"
      essential = false
      user      = "0"
      command = [
        "/cws-instrumentation",
        "setup",
        "--cws-volume-mount",
        "/cws-instrumentation-volume"
      ]
      mountPoints = [
        {
          sourceVolume  = "cws-instrumentation-volume"
          containerPath = "/cws-instrumentation-volume"
          readOnly      = false
        }
      ]
    },
  ]
  nginx = concat([
    {
      name      = "nginx"
      image     = "nginx:latest"
      essential = true
      portMappings = [
        {
          containerPort = 80
          hostPort      = 80
        }
      ]
    }
    ],
    var.use_datadog ? local.datadog_containers : [],
  )
}

resource "aws_ecs_task_definition" "service_with_datadog" {
  family                = "service"
  container_definitions = jsonencode(local.nginx)
}

Error message from terraform plan:

│ Error: Inconsistent conditional result types
│ 
│   on main.tf line 71, in locals:
│   71:     var.use_datadog ? local.datadog_containers : [],
│     ├────────────────
│     │ local.datadog_containers is tuple with 2 elements
│ 
│ The true and false result expressions must have consistent types. The 'true' tuple has length 2, but the 'false' tuple
│ has length 0.

In the nginx/datadog example, if I comment out the cws-instrumentation-init container, then I can get a valid plan with only one sidecar. But I do need that other sidecar.

Anybody know how I can unblock this inconsistent usage of conditional concatting?

And in case it is helpful, both of the above HCL examples are reproducable by pasting the HCL seen above into a main.tf file in an empty directory, run an init and then run a plan. The apache example will plan cleanly and the nginx example will break with the same error seen above.

$ terraform version
Terraform v1.8.0
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v5.46.0

Thanks.

[apparentlymart]

Hi @jrobison-sb,

The Terraform language has two broad kinds of compound data type: collection types which have a dynamically-decided number of elements all of the same type, or structural types which have a statically-decided number of elements/attributes that can then be of different types.

The “tuple” type kind is the structural type kind for sequences, where the number of elements and their types is fixed as part of the tuple type can can’t vary dynamically.

The conditional operator needs to be able to predict its own result type and so it tries to find a single type that both the true and false expressions can convert to. This allows the result to still be type checked even if the condition result isn’t known during the planning phase.

With all of that said, it seems like your data structure here “wants to be” a list of objects rather than a tuple type, because you want to decide dynamically how many elements to return and list is the collection type kind for sequences.

To use a list you will need to make sure all of the elements have the same object type. You can achieve that by adding portMappings = [] to the objects that don’t have any port mappings, so that they can all have the same object type and vary only in how many items are in the port mappings list.

Another option, if you don’t want to use a list of objects type, would be to use a for expression to dynamically create a new tuple of a dynamically-chosen tuple type:

[
  for sidecar in local.generic_sidecars : sidecar
  if var.use_generic_sidecars
]

This will filter out all of the elements of local.generic_sidecars unless the variable is set, producing a different type in each case. Because for expressions define a new value with a new type rather than just choosing between two values with their own types, this expression can produce a different tuple type depending on the input.

A caveat of this is that if the variable is unknown during planning then Terraform will not be able to type-check the for expression result at all, because its type will be unknown. However, since you are just passing the result to jsonencode directly anyway, and that function accepts values of any type, there would be no type checking to be done in this case anyway.

[jrobison-sb]

@apparentlymart thanks for your reply. That for loop unblocked me and got my task definitions working as desired.

Though I still don’t understand why adding portMappings = [] would be necessary for the other option, as in my example above with Apache, the two sidecars didn’t have any portMappings key and the Apache example was able to plan cleanly. I don’t doubt you that this key is causing the breakage, I just don’t understand why it didn’t also cause breakage in the Apache example.

Anyway, thanks for your help with this one. You’ve helped me a number of times over the years and are a huge asset to the Terraform community. Thanks for all your efforts.