Increment subnet ID's when building out new environments

Hi,

I have some current code which builds out an environment within an aws vpc with multiple different subnets for public/private etc. and this all works fine. However if I build a new environment it also assigns the same public/private subnet ranges, which is fine internally however I will be linking up these environments to my office via VPNs for each environment so I want to have separate subnet ranges for each environment terraform builds & manages.

Something like:
workspace: TEST1

resource “aws_subnet” “ApplicationSubnet1” {
cidr_block = “10.0.0.0/24”
vpc_id = aws_vpc.application.id
availability_zone = “${var.aws-region}a”
}

workspace: TEST2

resource “aws_subnet” “ApplicationSubnet1” {
cidr_block = “10.0.1.0/24”
vpc_id = aws_vpc.application.id
availability_zone = “${var.aws-region}a”
}

I’m not sure the best way to do this? Do I store a config entry in S3 that I read and then increment subnet based on that? Is there any examples of doing this? Is it even supported?

Thanks,
Terry

It is totally up to you where you want to store the IP address information. If you have an existing IPAM system you could try to get the information from there (either HTTP calls or extracted by a wrapper script).

We don’t, so we store the information in .tfvar files which we then choose using -var-file depending on the environment.

Ahh interesting, that might be the way for me too. I’ve never used external .tfvar files before so I guess I will need to read up on it.

As @stuart-c noted, there are lots of ways to get this done depending on how you’d like your system to be structured.

To add another idea to the pile, at a previous job we had a shared module called ip-ranges whose entire purpose was to represent our global network addressing scheme, and then we called that module from each configuration that would declare network objects and retrieved the appropriate CIDR prefixes out of that data structure1.

In that module we had some lookup tables to systematically assign different numbers to different discriminators that we were allocating the addresses to. For example, we first had a lookup table giving a number per cloud vendor, and then one for a number per region for each vendor, and finally one for availability zones. We then combined all that together with cidrsubnet to turn the individual numbers into full address prefixes.

I no longer have access to the source code for that, and it was written for Terraform 0.6 anyway so I expect I wrote it pretty differently to how I’d write it today, but here’s a little snippet showing the general idea in case you’d like to extrapolate it into a full solution. For simplicity I’m going to leave out the “vendor” table and just focus on the two-level heirarchy of region and AZ, because I think that’ll be enough to show what I mean:

locals {
  # If we allocate two addressing bits to regions and
  # three addressing bits to zones then we can have
  # up to four regions and up to eight zones, with
  # each zone then requiring five subnet addressing
  # bits in total.
  regions = {
    us-west-2 = 0
    us-east-1 = 1
    eu-west-1 = 2
    # number 3 available for future expansion
  }
  zones = {
    a = 0
    b = 1
    c = 2
    d = 3
    e = 4
    f = 5
    g = 6
    h = 7
  }

  base_cidr_block = "10.0.0.0/12"
  region_blocks = {
    for name, num in local.regions : name => {
      cidr_block = cidrsubnet(local.base_cidr_block, 2, num)
    }
  }
  zone_blocks = {
    for name, region_num in local.regions : name => {
      for letter, num in local.zones : "${name}${letter}" => {
        cidr_block = cidrsubnet(local.region_blocks[name].cidr_block, 3, num)
      }
    }
  }
}

output "networks" {
  value = {
    aws = {
      cidr_block = local.base_cidr_block
      regions = local.region_blocks
      zones   = local.zone_blocks
    }
  }
}

The idea here is that the output of this module doesn’t define which networks to create but rather provides the answer to a question like "If I’m creating a network for us-west-2a, what should its cidr_block be?

module "ip_ranges" {
  source = "../modules/ip-ranges"
}

resource “aws_subnet” “application_subnet_1” {
  cidr_block        = module.ip_ranges.aws.zones["${var.aws-region}a"]
  vpc_id            = aws_vpc.application.id
  availability_zone = "${var.aws-region}a"
}

Notice how it just ignores the rest of the data structure, because those address assignments are for some other resource to use, possibly in an entirely different configuration. Other configurations might end up referring to the cidr_block of this subnet too, such as if they’re configuring cross-VPC routing tables, and so having the full address tree available to all callers can make that sort of thing easier to represent. IP addressing scheme is a cross-cutting concern, after all.

Region and availability zone might not be the only discriminators that make sense in your particular addressing scheme – you might want to assign another addressing bit to distinguish private vs. public subnets, for example – but the general idea here is to assign some addressing bits to each of the items that will contribute to your addressing hierarchy and then centralize that addressing scheme in a shared module so that everything in your infrastructure will agree on what the addresses for each network ought to be.

This is certainly not the only way to do it. It might be overkill for simple systems, and equally might not be flexible enough for more complex systems that have many components that all have their own networks, but I’m just sharing it in case it’s useful inspiration.


1 (An interesting historical note is that the module I’m describing is what prompted me to contribute the cidrsubnet and cidrhost to Terraform, before I was a HashiCorp employee working on Terraform full-time.)