Newbie: Dynamic variables and resource generation - for_each?

Terraform
1

Hi all,

I’m a rather newbie with Terraform and while looking at what I built I realised it could look way better…
In this regard I started reading about for, for_each, count but got lost somewhere along the way.
Basically what I am trying to achieve is:

  • still use Terraform Cloud

  • have a variable let’s say “clouds = (aws, azure, etc)”

  • Use Case 1

    • based on that one spawn resources like this:
      module "aws_client_spoke_gateway" { 
   source = "..."
   cidr = var.clouds[0]_ipcidr -> cidr = var.aws_ipcidr 
   instance_size = var.cloud[0].isize
    • then repeat the same automagically for cloud[1]

  • Use Case 1a
    Also somehow dynamically generate those variable definitions (aws.ipcidr, azure.ipcidr; did not find something clear stating if I can do something along the lines of:

    • for var in (aws, azure)
    • define block of variables ( aws_ipcidr = string, aws_something1 = … , aws_something2 = )
      then exactly same block for azure_…, azure_…

  • Use Case 2: - “nesting” count & for_each
    I thought at some point to do something with count as I have for example a resource “linux_vm” and there I would need to spawn let’s say 6 in aws, 8 in azure (the 6, resp. 8 given in a variable like aws_count, azure_count I imagined).
    Was thinking (pseudocode) along the lines of: for i in variable cloud, create “count” (= var.cloud[i]~aws|azure|etc_number_of_instances) linux VMs.

    name          = "client${count.index}"
region        = var.aws_client_gw_src_region --> clouds[0] instead
                of aws, then repeat the same for clouds[1] with
                "count" interrations
vpc_id        = module.aws_client_spoke.vpc.vpc_id --> clouds[0] 
                instead of aws, then clouds[1], etc

Is something like this achievable and without major headaches?
I’m not sure if I’m approaching the issue logically here and how I’m new to Terraform it all starts to get a bit fuzz…

Hope someone can help me out a bit:)

Thanks.

2

Hi @mihaime,

Dynamically deciding which vendor(s) to use is a bit outside of typical Terraform usage patterns: usually a particular module is written for a particular provider and if we want to use a different vendor then we find or write a similar module tailored to that provider, and then either switch to the other module or call both of the modules.

It sounds like you might be trying to pack a wide variety of different functionality into a single module, which tends to get pretty complicated and hard to follow due to the number of conditionals and other dynamic behaviors. If possible I’d recommend taking a different approach where you declare more directly the result you want, though I’m not sure I understand your underlying problem well enough to make concrete suggestions for alternatives.

It might help if you could share what you have already and say a little about what you’d like to improve about it. If you can talk about some specific challenges you’ve run into with what you did so far then I can hopefully suggest some different approaches that make different tradeoffs.

3

@apparentlymart

Hmm sorry for the confusion I generated, I realised I also was not really very good at explaining what I’m trying to do or to study (better said) what is possible to optimise to have a cleaner code.
I put here my test repo:

I was thinking if there would be a way to avoid repetitive lines somehow.
For example:

  • in variables.tf I have aws_client_gw_src_size / azure_client_gw_src_size
    If in the future I expand this to other clouds I wanted a way to dynamically generate the variables definition blocks (gcp_, alibaba) for them as well to avoid huge blocks of vars.

    I found by chance the locals block in Terraform but that’s a bit the reverse of what I was wondering if it’s possible to do.

  • then in main.tf I have (just one example…most sections in the file are repeatable)
    for AWS:

    module "aws_client_spoke" {
source        = "terraform-aviatrix-modules/aws-spoke/aviatrix"
version       = "4.0.1"
name          = "aws-client"
cidr          = var.aws_iperf_client_cidr
region        = var.aws_client_gw_src_region
account       = var.aws_aviatrix_account_name
transit_gw    = module.aws_trans.transit_gateway.gw_name
instance_size = var.aws_client_gw_src_size
ha_gw         = var.aws_client_spoke_ha
insane_mode   = var.aws_insane_mode
}

    and for Azure

    module "azure_client_spoke" {
source        = "terraform-aviatrix-modules/azure-spoke/aviatrix"
version       = "4.0.1"
name          = "azure-client"
cidr          = var.azure_iperf_client_cidr
region        = var.azure_client_gw_src_region
account       = var.azure_aviatrix_account_name
transit_gw    = module.azure_trans.transit_gateway.gw_name
instance_size = var.azure_client_gw_src_size
ha_gw         = var.azure_client_spoke_ha
insane_mode   = var.azure_insane_mode
}

Such blocks are also kind of universal but they require to use different variables (aws_ , azure_).
I also wanted to see if there’s a way to eliminate all of these redundant declarations and avoid having a very long TF file.

Not sure if I make sense, I’m also not so experienced with TF.
Please let me know.

4

Thanks for sharing those examples, @mihaime! That helps a lot to understand what you’re intending to achieve and what challenges you’ve run into.

Since this configuration is built from a number of third-party modules the options here are somewhat constrained by some design decisions of those modules. I won’t dwell on this a lot because changing those third-party modules is not really in scope for what you are working on anyway, but it’s true that if you are using modules that require a lot of different inputs then you can’t really avoid assigning values to all of them.

One thing I’m noticing reading over all of these module blocks is that although there are several repeated blocks with similar structure, there seems to be very little overlap in actual content between them. For example: module "aws_trans", module "aws_client_spoke", and module "aws_server_spoke" all have an argument called region, but they all take different values for that argument because the purpose of this design is (presumably) to create connectivity between regions.

With that said then, it doesn’t seem like there’s a lot of content redundancy across these blocks. The differences in settings between these need to be represented somewhere, and so what you have is the most direct and therefore least redundant way to represent those: directly inside the module blocks. It’s important to recognize the difference between something being long because there are really lots of different things to declare and long because the declarations are done in an inefficient way.

The main opportunity I can see for simplification here is that you seem to have a general pattern in how the different modules all contribute DNS records to a route53 zone: each module instance contributes one A record to a route53 zone where all of the other settings are always the same. If we can gather that information all together into a simpler data structure then that could all potentially be reduced to a single resource "aws_route53_record" block with one instance for each recordset.

To achieve that we need to think about how to transform the various source module data into the shape needed to represent the DNS recordsets. That ultimately means a collection that has one element for each recordset, where the elements each describe the name, zone_id, and the IP address to write into records.

The zone IDs seem to vary only by the target platform (AWS vs Azure) so I think my first step would be to build two separate collections of records per platform so that we don’t need to worry yet about the zone IDs and fully-qualified domain names:

locals {
  aws_dns_records = merge(
    { for i, m in module.aws_client : "aws-client${i}" => m.public_ip },
    { for i, m in module.aws_client : "aws-client${i}-priv" => m.private_ip },
    { for i, m in module.aws_server : "aws-server${i}" => m.public_ip },
    { for i, m in module.aws_server : "aws-server${i}-priv" => m.private_ip },
  )
  azure_dns_records = merge(
    { for i, m in module.azureclient : "azure-client${i}" => m.public_ip.ip_address },
    { for i, m in module.azureclient : "azure-client${i}-priv" => m.nic.private_ip_address },
    { for i, m in module.azureserver : "azure-server${i}" => m.public_ip.ip_address },
    { for i, m in module.azureserver : "azure-server${i}-priv" => m.nic.private_ip_address },
  )
}

The general goal here is to discard the parts of these module instances that are not relevant to the DNS records and focus only normalizing away the little differences in how they are structured between the modules. The end result of both of these should be a flat map from hostname to IP address, because it’s constructing one mapping for each module (using for expressions) and then merge-ing them all together into a single map. The hostname structures are different for each so there should be no conflicts during the merge.

With both of those now in a regular structure, we can follow a similar pattern to merge those together to produce a single local value with all of the DNS records in it:

locals {
  dns_records = tomap(merge(
    {
      for n, ip in local.aws_dns_records :
      "${n}.${data.aws_route53_zone.awslinuxdns.name}" => {
        zone_id    = data.aws_route53_zone.awslinuxdns.zone_id
        ip_address = ip
      }
    },
    {
      for n, ip in local.azure_dns_records :
      "${n}.${data.aws_route53_zone.azurelinuxdns.name}" => {
        zone_id    = data.aws_route53_zone.azurelinuxdns.zone_id
        ip_address = ip
      }
    },
  ))
}

Again here the goal is to isolate the ways in which these two cases differ – the domain name and the zone ID – and normalize that away so that all of them have the same structure and meaning. The result of this is a map from fully qualified domain names to objects that contain both the target zone ID and the IP address.

This final data structure now fully represents all of the systematic differences between your DNS records, and does so in a way that’s compatible with resource for_each, and so it should be possible to declare them all together with a single resource block:

resource "aws_route53_record" "servers" {
  for_each = local.dns_records

  zone_id = each.value.zone_id
  name    = each.key
  type    = "A"
  ttl     = "1"
  records = [each.value.ip_address]
}

Due to the design of resource for_each, Terraform will identify each of the instances of this resource by the keys in the map, which are the fully-qualified domain names. If we assume that your AWS DNS zone is called example.com and your Azure one is called example.net then you’d end up with resource instances with addresses like this:

  • aws_route53_record.servers["aws-client0.example.com"]
  • aws_route53_record.servers["aws-client1.example.com"]
  • aws_route53_record.servers["aws-client0-priv.example.com"]
  • aws_route53_record.servers["aws-client1-priv.example.com"]
  • aws_route53_record.servers["azure-client0.example.net"]
  • aws_route53_record.servers["azure-client1.example.net"]
  • aws_route53_record.servers["azure-client0-priv.example.net"]
  • aws_route53_record.servers["azure-client1-priv.example.net"]

I hope that this has at least given some ideas about how you can use transformation expressions to pull out the common elements of various separately-declared objects so that you can use them systematically. I don’t see any other opportunities for designs like this in the module you’ve shared here, but this approach of starting with a bunch of things that all are all separate due to essential differences and then using expressions to normalize them into a simpler, common structure to use elsewhere is a common technique for reducing redundancy and repetition in Terraform modules.

1 Like
5

@apparentlymart really nice explanation.
For some reason I did not get the notification for this post but really wow.
Thank you very much!!
I will do the DNS part simplification in this case. Love how you did it and looks so tidy.