Problem with the use of JSON config syntax for a built-in function in a resource block

Terraform
1

I use terraform 0.12.26 and aws provider version 2.67.0. I work locally on a linux fedora 32 computer, and run servers in AWS cloud.

I hope that someone more experienced can help me with this problem.

I want to create a subnet with IPv4 and IPv6 CIDR blocks, and custom subnets in a custom VPC. AWS allocates an IPv6 CIDR-block during the deployment of the new VPC. In the module covering the subnet(s), I believe I can write in native syntax:

resource "aws_subnet" "mainVPC" {
  vpc_id     = "aws_vpc.mainVPC.id
  cidr_block = "var.subnet_cidrs_X1A"
  ipv6_cidr_block = "cidrsubnet(aws_vpc.mainVPC.ipv6_cidr_block,8,16)"

  tags = {
    Name = "Main VPC"
  }
}

NOW MY PROBLEM

I want to write (all) my config files in JSON, not in native syntax / HCL. [I have reasons why; I know it’s not recommended; but Hashicorp has also stated that they want JSON to be a first-class alternative language; and this is not the place for that discussion] I have not managed to find a way of doing it.

I have run three scenarios.

Scenario 1

My most-likely-to-succeed version was the following:

subnet.tf.json

{
  "resource": [
    {
      "aws_subnet": {
        "X1A-subnet": [
          {
            "vpc_id": "${aws_vpc.mainVPC.id}"
            "cidr_block": "${var.public_cidrs_X1A}",
            "ipv6_cidr_block": "cidrsubnet(${aws_vpc.mainVPC.ipv6_cidr_block},8,16)",
            "tags": {
              "Name": "X1A Subnet"
            },
          }
        ]
      }
    }
  ]
}

This passes terraform init and terraform validate. I can run terraform plan without an error. But the terraform engine refuses to deploy a subnet; running terraform apply produces the error message:

Error: Error creating subnet: InvalidParameterValue: invalid value for parameter ipv6-cidr-block: cidrsubnet(2a05:d014:aac:7700::/56,8,16)
status code: 400, request id: aed1>>>

Scenario 2

The variant:

"ipv6_cidr_block": "cidrsubnet(aws_vpc.THEVPC77.ipv6_cidr_block,8,16)"

also passes init and validate, but is rejected when I run terraform plan as an error:

Error: "ingress.4.ipv6_cidr_blocks.0" must contain a valid CIDR, got error parsing: invalid CIDR address: cidrsubnet(aws_vpc.mainVPC.ipv6_cidr_block,8,32)

  on NAT_sec_group.tf.json line 223, in resource[0].aws_security_group.NAT_instance[0]:
 223:           }

This error is different, and I understand it even less. It comes from a security group file, and does not seem directly related to the issue I’m trying to solve here.

In a very small-scale separate pure test-case that I ran, the error was located differently. It got through without errors all the way to terraform plan. But when I ran terraform apply, it could not create the subnet, and the error message was:

Error: Error creating subnet: InvalidParameterValue: invalid value for parameter ipv6-cidr-block: cidrsubnet(aws_vpc.mainVPC.ipv6_cidr_block,8,16)
status code: 400, request id: e1d162>>>

  on subnet.tf.json line 15, in resource[0].aws_subnet.X1A-subnet[0]:
  15:           }

Scenario 3

The following variant passes terraform init and terraform validate :

"ipv6_cidr_block": "cidrsubnet("${aws_vpc.mainVPC.ipv6_cidr_block}",8,16)"

This causes a terraform crash when running terraform plan, with the initial crash message reading:

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/hashicorp/hcl/v2/json.(*peeker).Read(...)
	/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/hashicorp/hcl/v2@v2.3.0/json/peeker.go:20
github.com/hashicorp/hcl/v2/json.parseObject.func1(0x0, 0xc000458bb6, 0x1, 0x8ae, 0xc000049240, 0x1a, 0xa, 0x2d, 0x136, 0xa, ...)

In my very small-scale test run with only a few files, and the most basic of set-ups, just running terraform init crashed. First few lines:

panic: runtime error: index out of range

goroutine 1 [running]:
github.com/hashicorp/hcl/v2/json.(*peeker).Read(...)
	/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/hashicorp/hcl/v2@v2.3.0/json/peeker.go:20
github.com/hashicorp/hcl/v2/json.parseObject.func1(0x0, 0xc0003c612a, 0x1, 0x2d6, 0xc000044600, 0xe, 0xa, 0x2d, 0x12a, 0xa, ...)

MY COMMENTS / UNDERSTANDING

I am pretty baffled about what’s going on. I understand and can work with the different “interpolation” (old name) syntax between HCL and JSON files. But for this particular expression I don’t know what to do. I understand that Scenario 3 could not work, though I find it a bit disappointing that the parser can do nothing better than crashing.

But I don’t know how to make this work, and it is essential for my main use-case.

There are probably quite involved work-arounds to deal with this - including separating a run for creating the VPC, subnets, security groups, etc from a run for deploying the instances. But I thought that the whole idea of Terraform was that you could do it all in one go. And overall my application is tiny compared with the industrial-scale usage of Terraform. At the moment 4 instances, maybe over the years growing to between 10 and 20. All small; nothing really sophisticated.

I would be very grateful if anyone could throw light on this, tell me how to solve this specifically, and ideally, also with some indication of where in the documentation I can find more background to this.

2

Hi @henrystrick,

It looks like your original native syntax example actually had a similar problem as your JSON scenario 2, and would’ve produced a similar error. If you intend to call the cidrsubnet function then you need to write it out directly, rather than in quotes. If you put it in quotes then you’re telling Terraform to use a string that literally contains the characters cidrsubnet( etc, which isn’t a valid IPv6 CIDR block and so causes a validation error.

Here, then, is how we’d write your first example in correct native syntax:

resource "aws_subnet" "mainVPC" {
  vpc_id          = aws_vpc.mainVPC.id
  cidr_block      = var.subnet_cidrs_X1A
  ipv6_cidr_block = cidrsubnet(aws_vpc.mainVPC.ipv6_cidr_block,8,16)

  tags = {
    Name = "Main VPC"
  }
}

To translate these expressions to JSON, the general rule is that any expression that isn’t a literal value and that isn’t already a string template must be wrapped in "${ }" delimiters to turn it into a string template. If we do that mechanically to each of the arguments above that isn’t already a quoted string, we get this:

{
  "resource": [
    {
      "aws_subnet": {
        "X1A-subnet": [
          {
            "vpc_id": "${aws_vpc.mainVPC.id}",
            "cidr_block": "${var.public_cidrs_X1A}",
            "ipv6_cidr_block": "${cidrsubnet(aws_vpc.mainVPC.ipv6_cidr_block,8,16)}",
            "tags": {
              "Name": "X1A Subnet"
            },
          }
        ]
      }
    }
  ]
}

In a Terraform template, a ${ … } interpolation directive causes Terraform to evaluate the expression inside and then replace the directive with the expression’s result. In the case of ipv6_cidr_block, the expression to be evaluated is cidrsubnet(aws_vpc.mainVPC.ipv6_cidr_block,8,16), and its result is then used as the entire value for that argument because there are no other characters in the template string.

3

Marvellous. That makes complete sense. Surprising that I could not find it for myself. I understand the principle of the interpolation directive, and just did not apply it in the only sensible way, and got confused. This works indeed seamlessly. Thank you so much for the complete and ultra-rapid solution!