Looping through multiple json files to create azure dns records

Terraform
1

Hello there,

I’m planning to manage all our Azure DNS records in one json-file per DNS zone.

One json file would look like this

{
  "a": [
    {
      "name": "@",
      "ip": "111.222.333.444"
    },
    {
      "name": "www",
      "ip": "111.222.333.444"
    }
  ],
  "aaaa": [],
  "cname": [
    {
      "name": "autodiscover",
      "alias": "autodiscover.outlook.com"
    },
    {
      "name": "lyncdiscover",
      "alias": "webdir.online.lync.com"
    },
    {
      "name": "sip",
      "alias": "sipdir.online.lync.com"
    }
  ],
  "mx": [
    {
      "preference": 10,
      "host": "mail01.domain.com"
    },
    {
      "preference": 10,
      "host": "mail02.domain.com"
    }
  ]
}

I’m using local variable block to read the json files and use jsondecode to get the content

locals {
  #   # get dns data from json file
  dns_records_files = fileset(path.module, "/files/dns_zones/*.json")
  dns_records_data  = [for file in local.dns_records_files : jsondecode(file("${path.module}/${file}"))]
}

I now want to loop through each file and get all A, CNAME, MX, … records and create the appropriate DNS resource with the AzureRM provider.

I’ve tried this approach (alhtough I’m not sure if this is even possible)

resource "azurerm_dns_a_record" "dns_a_records" {
  for_each            = { for record in local.dns_records_data.a : record.name => record }
  name                = each.value.name
  zone_name           = azurerm_dns_zone.dns_zone[trimsuffix(basename(each.key), ".json")].name
  resource_group_name = azurerm_resource_group.resource_group_core_dns.name
  ttl                 = 3600
  records             = each.value.ip
}

Terraform validate outputs the following error

│ Error: Unsupported attribute
│
│   on dns_records.tf line 31, in resource "azurerm_dns_a_record" "dns_a_records":
│   31:   for_each            = { for record in local.dns_records_data.a : record.name => record }
│     ├────────────────
│     │ local.dns_records_data is tuple with 2 elements
│
│ This value does not have any attributes.

Question:

  • Is this even possible to loop through multiple json files, filter on A records and use only one resource block?
  • What’s the correct approach to filter/select the A records?

Thank you so much for any hints and a happy new year in advance!

Denny

2

I’m uncertain this is the way you should go, because:

Unless you resort to telling Terraform to not re-verify that its current state is still accurate (-refresh=false), Terraform is probably going to be doing at least (total number of records) GET operations on every plan attempt.

At the scale that needs such a generic definition framework for DNS records, you’re going to run up against that 500 limit pretty quickly.

If the scale is sufficiently small that it’s not going to be a problem, it may well be better to keep your Terraform simpler by not layering your own fairly complex JSON abstraction on top of it, and writing individual Terraform blocks with simpler for_each expressions for specific use-cases.

But, if you really want to give it a try, you could attempt something like this (where I’ve used output blocks to demonstrate and display expressions that you could use for for_each in your configuration):

locals {
  records = flatten(
    [for file_name in fileset(path.module, "*.json") : 
      [for record_type, records in jsondecode(file("${path.module}/${file_name}")) :
        [for record in records :
          merge(record, {
            zone        = trimsuffix(basename(file_name), ".json")
            record_type = record_type
          })
        ]
      ]
    ]
  )
}

output "a_records" {
  value = { 
    for record in local.records :
    "${record.zone} ${record.name}" => record
    if record.record_type == "a"
  }
}

output "cname_records" {
  value = { 
    for record in local.records : 
    "${record.zone} ${record.name}" => record 
    if record.record_type == "cname" 
  }
}

output "mx_records" {
  value = { 
    for record in local.records : 
    "${record.zone} ${record.host}" => record
    if record.record_type == "mx"
  }
}

(The idea being that with these expressions, each.value.whatever would have all the data you needed.)

Except, I’ve never actually worked with Azure DNS before, and now I skim the documentation for the relevant Terraform resources, I see they don’t actually correspond to DNS records, but DNS record sets, so although this is a good start, a little bit more tweaking would be required…

3

Perhaps something like this, then? Getting pretty fearsomely unreadable, though.

locals {
  records = flatten(
    [for file_name in fileset(path.module, "*.json") :
      [for record_type, records in jsondecode(file("${path.module}/${file_name}")) :
        [for record in records :
          merge(record, {
            zone        = trimsuffix(basename(file_name), ".json")
            record_type = record_type
          })
        ]
      ]
    ]
  )

  a_records = {
    for record in local.records :
    "${record.zone} ${record.name}" => record...
    if record.record_type == "a"
  }

  cname_records = {
    for record in local.records :
    "${record.zone} ${record.name}" => record
    if record.record_type == "cname"
  }

  mx_records = {
    for record in local.records :
    "${record.zone}" => record...
    if record.record_type == "mx"
  }
}

resource "azurerm_dns_a_record" "record" {
  for_each            = local.a_records
  name                = each.value[0].name
  zone_name           = each.value[0].zone_name
  records             = each.value[*].ip
  ttl                 = 0
  resource_group_name = "blah"
}

resource "azurerm_dns_cname_record" "record" {
  for_each            = local.cname_records
  name                = each.value.name
  zone_name           = each.value.zone_name
  record              = each.value.alias
  ttl                 = 0
  resource_group_name = "blah"
}

resource "azurerm_dns_mx_record" "record" {
  for_each  = local.mx_records
  name      = each.value[0].name
  zone_name = each.value[0].zone_name
  dynamic "record" {
    for_each = each.value
    content {
      preference = record.preference
      exchange   = record.host
    }
  }
  ttl                 = 0
  resource_group_name = "blah"
}
4

@maxb thank you for your reply and hints. I wasn’t aware there’s a 500 item “limit” and it’s really getting hard to read/understand. Nevertheless I’ll give it a try in the next couple of days.

At least (I think) I understand looping a bit better :slight_smile: