[Solved - Workaround] How to use iterator from ouptut list from a module?

Terraform CDK for Terraform
1

Hello,
I want to try to use an iterator to cycle through the element of a list output of a resource within the same stack but coming from a module.

The use case is this: I want to create an AWS VPC with 3 private subnets and for each of them I want to create some EC2 instances. I want to keep these resources within the same stack because I want to use the instances as cheap NAT instances, so conceptually they are still part of the CDK stack.

What am I doing wrong?

from constructs import Construct
from cdktf import TerraformStack, S3Backend, TerraformIterator, Token, Fn
from imports.aws import AwsProvider
from imports.aws.ec2 import Instance
from imports.terraform_aws_modules.aws import Vpc
from time import strftime,localtime

# VPC() VPC is the custom name of this stack.
class VPCCust(TerraformStack):
    def __init__(self, scope: Construct, ns: str):
        super().__init__(scope, ns)
        # define resources here
        #Set params as per https://www.terraform.io/language/settings/backends/s3
        backend = S3Backend(self, 
            bucket="infrastructure-status-01",
            key="terraform/state",
            region="eu-west-1",
            dynamodb_table="terraform-lock",
            profile="blah"
        )

        AwsProvider(self, 'Aws', region='eu-west-1', profile="blah-dev")
        appVPC = Vpc(self, 'CustomVpcFoo',
            tags = {
                'owner': 'me',
                'created': strftime("%d-%m-%Y.%H:%M:%ST%Z", localtime())
            },
            name='custom-vpc-blah',
            cidr='10.0.0.0/16',
            azs=['eu-west-1a', 'eu-west-1b', 'eu-west-1c'],
            private_subnets=['10.0.1.0/24', '10.0.2.0/24', '10.0.3.0/24'],
            private_subnet_tags={
                'scope': 'private'
            },
            public_subnets=['10.0.4.0/24', '10.0.5.0/24', '10.0.6.0/24'],
            public_subnet_tags={
                'scope': 'public'
            },
            enable_nat_gateway = False,
            single_nat_gateway = False,
            one_nat_gateway_per_az = False
            )
        print(appVPC.private_subnets_output)
        private_subnets_iterator = TerraformIterator.from_list(Fn.tolist(Token().as_string(appVPC.private_subnets_output)))
        print(private_subnets_iterator)
        inst = Instance(self, 
            'natInstance',
            for_each=private_subnets_iterator,
            instance_type='t3.micro', 
            ami='ami-01333d6a593e92414',
            subnet_id=Token().as_string(private_subnets_iterator.value))
2

Hi @fpb,

could you try private_subnets_iterator = TerraformIterator.from_list(Token().as_list(appVPC.private_subnets_output))? But that’s just a shot in the dark, could you share the error you are getting?

– Ansgar

3

Thanks @ansgarm .

Sorry I should have posted it before! The error I have happens after the synthesys part, when the actual terraform plan takes over. I have the same error also with your suggestion:

             │ Error: Invalid for_each argument
             │ 
             │   on cdk.tf.json line 88, in resource.aws_instance.natInstance (natInstance):
             │   88:         "for_each": "${toset(module.CustomVpcFoo.private_subnets)}",
             │     ├────────────────
             │     │ module.CustomVpcFoo.private_subnets is tuple with 3 elements
             │ 
             │ The "for_each" set includes values derived from resource attributes that
             │ cannot be determined until apply, and so Terraform cannot determine the
             │ full set of keys that will identify the instances of this resource.
             │ 
             │ When working with unknown values in for_each, it's better to use a map
             │ value where the keys are defined statically in your configuration and where
             │ only the values contain apply-time results.
             │ 
             │ Alternatively, you could use the -target planning option to first apply
             │ only the resources that the for_each value depends on, and then apply a
             │ second time to fully converge.
4

Ah I think I found the issue and also a workaround.

The issue is that it seems to be an open bug of cdktf: Incorrectly generated iterator for ComplexList · Issue #2001 · hashicorp/terraform-cdk · GitHub

The same bug report, however, also reports a workaround to implement a for_each in an escape hatch. So implementing the workaround I managed to obtain what I wanted to achieve (see subnet_id=“${each.value}” and inst.add_override) :

from constructs import Construct
from cdktf import TerraformStack, S3Backend, TerraformIterator, Token
from imports.aws import AwsProvider
from imports.aws.ec2 import Instance
from imports.aws.vpc import DataAwsSubnetIds
from imports.terraform_aws_modules.aws import Vpc
from time import strftime,localtime

# VPC() VPC is the custom name of this stack.
class VPCCust(TerraformStack):
    def __init__(self, scope: Construct, ns: str):
        super().__init__(scope, ns)
        # define resources here
        #Set params as per https://www.terraform.io/language/settings/backends/s3
        backend = S3Backend(self, 
            bucket="infrastructure-status-01",
            key="terraform/state",
            region="eu-west-1",
            dynamodb_table="terraform-lock",
            profile="blah-build"
        )

        AwsProvider(self, 'Aws', region='eu-west-1', profile="blah-dev")
        appVPC = Vpc(self, 'CustomVpcFoo',
            tags = {
                'owner': 'blah',
                'created': strftime("%d-%m-%Y.%H:%M:%ST%Z", localtime())
            },
            name='custom-vpc-blah',
            cidr='10.0.0.0/16',
            azs=['eu-west-1a', 'eu-west-1b', 'eu-west-1c'],
            private_subnets=['10.0.1.0/24', '10.0.2.0/24', '10.0.3.0/24'],
            private_subnet_tags={
                'scope': 'private'
            },
            public_subnets=['10.0.4.0/24', '10.0.5.0/24', '10.0.6.0/24'],
            public_subnet_tags={
                'scope': 'public'
            },
            enable_nat_gateway = False,
            single_nat_gateway = False,
            one_nat_gateway_per_az = True,
            putin_khuylo = True
            )

        inst = Instance(self, 
            'natInstance',
            instance_type='t3.micro', 
            ami='ami-01333d6a593e92414',
            subnet_id="${each.value}",
            depends_on=[appVPC])

        inst.add_override('for_each',
            '${{for idx, nid in module.CustomVpcFoo.public_subnets: idx => nid }}')