How can I pass values from Terraform modules to user data inputs in Terraform CDK?

I am trying to use the ARN (it is just a simple example not the real use case) from a VPC module as the user data input but I’m having trouble as the value is always a token. I have tried using the local provider but it doesn’t seem to work, and the local file is not being created. What is the best practice for this scenario?"

Example: main.ts

import { App, TerraformStack } from 'cdktf';
import { Construct } from 'constructs';
import { LocalProvider } from '@cdktf/provider-local/lib/provider';
import { File } from '@cdktf/provider-local/lib/file';
import { VpcTestStack } from './lib/vpc-test';

// Initialize the CDK app
const app = new App();

// Define a Terraform stack for converting to CDK
export class TerraformToCdkStack extends TerraformStack {
  // Constructor for the Terraform to CDK stack
  constructor(scope: Construct, id: string) {
    // Call the super constructor
    super(scope, id);

    // Initialize the local provider
    new LocalProvider(this, 'local');

    // Create an instance of the VpcTestStack
    const testValue = new VpcTestStack(app, 'vpc');

    // Define the user data
    const userData = `#!/usr/bin/bash
    echo "${testValue.arn}" > /temp/test.txt
`;

    // Create a local file with the specified content and filename
    new File(this, 'al2UserDataFile', {
      content: userData,
      filename: 'testdata.txt',
    });
  }
}

// Create an instance of the Terraform to CDK stack
new TerraformToCdkStack(app, 'thisFile');

// Synthesize the Terraform plan
app.synth();

here the VpcTestStack:

import { AwsProvider } from '@cdktf/provider-aws/lib/provider';
import { Vpc } from '@cdktf/provider-aws/lib/vpc';
import { TerraformStack } from 'cdktf';
import { Construct } from 'constructs';

// Define a Terraform stack for the VPC
export class VpcTestStack extends TerraformStack {
  // Variables to store the ZPA provisioning key, the provisioning key data, the temporary key, and the test
  zpaProvisioningKey: any;
  provisioningKeyData: any;
  tempKey: any;
  test: any;

  // Variable to store the AWS region
  region: string;

  // Variable to store the VPC
  vpc: any;

  // Constructor for the VPC stack
  constructor(scope: Construct, id: string) {
    // Call the super constructor
    super(scope, id);

    // Set the AWS region
    this.region = 'eu-south-1';

    // Initialize the AWS provider with the specified region
    new AwsProvider(this, 'AWS', { region: this.region });

    // Create a VPC with the specified CIDR block and tags
    this.vpc = new Vpc(this, 'vpc_1', {
      cidrBlock: '10.0.0.0/16',
      tags: { Name: 'vpc_1' },
    });
  }

  // Getter method to return the ARN of the VPC
  get arn(): string {
    return this.vpc.arn;
  }

  // Setter method to set the ARN of the VPC
  set arn(arn) {
    this.vpc.arn = arn;
  }
}

I am still grappling with this issue. If I use the File from LocalProvider, the token will be resolved with the appropriate value. However, I am also considering using the Locals option in Terraform HCL, but I am unable to locate the appropriate function in Terraform CDK. I am seeking a way to convert the following HCL solution to CDK:

################################################################################
# B. Create the user_data file with necessary bootstrap variables for App
#     Connector registration. Used if variable use_zscaler_ami is set to false.
################################################################################
locals {
  al2userdata = <<AL2USERDATA
#!/usr/bin/bash
sudo touch /etc/yum.repos.d/zscaler.repo
sudo cat > /etc/yum.repos.d/zscaler.repo <<-EOT
[zscaler]
name=Zscaler Private Access Repository
baseurl=https://yum.private.zscaler.com/yum/el7
enabled=1
gpgcheck=1
gpgkey=https://yum.private.zscaler.com/gpg
EOT
sleep 60
#Install App Connector packages
yum install zpa-connector -y
#Stop the App Connector service which was auto-started at boot time
systemctl stop zpa-connector
#Create a file from the App Connector provisioning key created in the ZPA Admin Portal
#Make sure that the provisioning key is between double quotes
echo "${module.zpa_provisioning_key.provisioning_key}" > /opt/zscaler/var/provision_key
chmod 644 /opt/zscaler/var/provision_key
#Run a yum update to apply the latest patches
yum update -y
#Start the App Connector service to enroll it in the ZPA cloud
systemctl start zpa-connector
#Wait for the App Connector to download latest build
sleep 60
#Stop and then start the App Connector for the latest build
systemctl stop zpa-connector
systemctl start zpa-connector
AL2USERDATA
}

# Write the file to local filesystem for storage/reference
resource "local_file" "al2_user_data_file" {
  count    = var.use_zscaler_ami == true ? 0 : 1
  content  = local.al2userdata
  filename = "../user_data"
}

# Create the specified AC VMs via Launch Template and Autoscaling Group
module "ac_asg" {
  source                      = "../../modules/terraform-zsac-asg-aws"
  name_prefix                 = var.name_prefix
  resource_tag                = random_string.suffix.result
  global_tags                 = local.global_tags
  ac_subnet_ids               = module.network.ac_subnet_ids
  instance_key                = aws_key_pair.deployer.key_name
  user_data                   = var.use_zscaler_ami == true ? local.appuserdata : local.al2userdata
  acvm_instance_type          = var.acvm_instance_type
  iam_instance_profile        = module.ac_iam.iam_instance_profile_id
  security_group_id           = module.ac_sg.ac_security_group_id
  associate_public_ip_address = var.associate_public_ip_address

  max_size                  = var.max_size
  min_size                  = var.min_size
  target_cpu_util_value     = var.target_cpu_util_value
  health_check_grace_period = var.health_check_grace_period
  launch_template_version   = var.launch_template_version
  target_tracking_metric    = var.target_tracking_metric

  warm_pool_enabled = var.warm_pool_enabled
  ### only utilzed if warm_pool_enabled set to true ###
  warm_pool_state                       = var.warm_pool_state
  warm_pool_min_size                    = var.warm_pool_min_size
  warm_pool_max_group_prepared_capacity = var.warm_pool_max_group_prepared_capacity
  reuse_on_scale_in                     = var.reuse_on_scale_in
  ### only utilzed if warm_pool_enabled set to true ###

Hi @fdervisi :wave:

You might be looking for TerraformLocal?

Thanks @ansgarm,
yes this should be the solution however it don’t understand how it works. According to this guide, I need to use it like this:

        const testLocal = new TerraformLocal(
          this,
          `user-data-asg-${config.asgIndex}`,
          {
            test: `echo "${this.provisioningKey.provisioningKey}"> /opt/zscaler/var/provision_key`,
          }
        );

        this.renderedTemplate = testLocal.expression;

and this is the Debug:

testLocal._expression
test: "echo \"${TfToken[TOKEN.39]}\"> /opt/zscaler/var/provision_key"

I need to access the test key ( this.renderedTemplate = testLocal.expression.test) but it does not work, I get this error:

⠸  Synthesizing
[2023-02-17T09:05:10.131] [ERROR] default - /home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tfExpression.ts:43
      .replace(/\n/g, "\\n") // escape newlines
       ^

[2023-02-17T09:05:10.134] [ERROR] default - TypeError: Resolution error: Resolution error: Cannot read properties of undefined (reading 'replace').
Object creation stack:
  at new Intrinsic (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tokens/private/intrinsic.ts:34:26)
  at new TFExpression (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tfExpression.ts:11:1)
  at new RawString (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tfExpression.ts:113:5)
  at Object.rawString (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tfExpression.ts:128:10)
  at Function.rawString (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/terraform-functions.ts:1268:21)
  at new AutoScalingGroupStack (/home/ubuntu/CloudServiceTree/lib/asg-stack.ts:122:12)
  at /home/ubuntu/CloudServiceTree/lib/subnet-stack.ts:98:9
  at Array.forEach (<anonymous>)
  at new SubnetStack (/home/ubuntu/CloudServiceTree/lib/subnet-stack.ts:97:38)
  at /home/ubuntu/CloudServiceTree/lib/vpc-stack.ts:95:29
  at Array.forEach (<anonymous>)
  at new VpcStack (/home/ubuntu/CloudServiceTree/lib/vpc-stack.ts:94:26)
  at /home/ubuntu/CloudServiceTree/lib/region-stack.ts:25:9
  at Array.forEach (<anonymous>)
  at new RegionStack (/home/ubuntu/CloudServiceTree/lib/region-stack.ts:23:26)
  at Object.<anonymous> (/home/ubuntu/CloudServiceTree/main.ts:19:3)
  at Module._compile (node:internal/modules/cjs/loader:1120:14)
  at Module.m._compile (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/index.ts:1618:23)
  at Module._extensions..js (node:internal/modules/cjs/loader:1174:10)
  at Object.require.extensions.<computed> [as .ts] (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/index.ts:1621:12)
  at Module.load (node:internal/modules/cjs/loader:998:32)
  at Function.Module._load (node:internal/modules/cjs/loader:839:12)
  at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
  at phase4 (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/bin.ts:649:14)
  at bootstrap (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/bin.ts:95:10)
  at main (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/bin.ts:55:10)
  at Object.<anonymous> (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/bin.ts:800:3)
  at Module._compile (node:internal/modules/cjs/loader:1120:14)
  at Object.Module._extensions..js (node:internal/modules/cjs/loader:1174:10)
  at Module.load (node:internal/modules/cjs/loader:998:32)
  at Function.Module._load (node:internal/modules/cjs/loader:839:12)
  at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
  at node:internal/main/run_main_module:17:47.
Object creation stack:
  at new Intrinsic (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tokens/private/intrinsic.ts:34:26)
  at new TFExpression (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tfExpression.ts:11:1)
  at new FunctionCall (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tfExpression.ts:318:5)
  at Object.call (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tfExpression.ts:344:10)
  at /home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/terraform-functions.ts:159:14
  at Function.base64encode (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/terraform-functions.ts:691:69)
  at new AutoScalingGroupStack (/home/ubuntu/CloudServiceTree/lib/asg-stack.ts:121:45)
  at /home/ubuntu/CloudServiceTree/lib/subnet-stack.ts:98:9
  at Array.forEach (<anonymous>)
  at new SubnetStack (/home/ubuntu/CloudServiceTree/lib/subnet-stack.ts:97:38)
  at /home/ubuntu/CloudServiceTree/lib/vpc-stack.ts:95:29
  at Array.forEach (<anonymous>)
  at new VpcStack (/home/ubuntu/CloudServiceTree/lib/vpc-stack.ts:94:26)
  at /home/ubuntu/CloudServiceTree/lib/region-stack.ts:25:9
  at Array.forEach (<anonymous>)
  at new RegionStack (/home/ubuntu/CloudServiceTree/lib/region-stack.ts:23:26)
  at Object.<anonymous> (/home/ubuntu/CloudServiceTree/main.ts:19:3)
  at Module._compile (node:internal/modules/cjs/loader:1120:14)
  at Module.m._compile (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/index.ts:1618:23)
  at Module._extensions..js (node:internal/modules/cjs/loader:1174:10)
  at Object.require.extensions.<computed> [as .ts] (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/index.ts:1621:12)
  at Module.load (node:internal/modules/cjs/loader:998:32)
  at Function.Module._load (node:internal/modules/cjs/loader:839:12)
  at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
  at phase4 (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/bin.ts:649:14)
  at bootstrap (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/bin.ts:95:10)
  at main (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/bin.ts:55:10)
  at Object.<anonymous> (/home/ubuntu/CloudServiceTree/node_modules/ts-node/src/bin.ts:800:3)
  at Module._compile (node:internal/modules/cjs/loader:1120:14)
  at Object.Module._extensions..js (node:internal/modules/cjs/loader:1174:10)
  at Module.load (node:internal/modules/cjs/loader:998:32)
  at Function.Module._load (node:internal/modules/cjs/loader:839:12)
  at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
  at node:internal/main/run_main_module:17:47
    at RawString.escapeString (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tfExpression.ts:43:8)
    at RawString.resolve (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tfExpression.ts:118:26)
    at DefaultTokenResolver.resolveToken (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tokens/resolvable.ts:187:24)
    at resolve (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tokens/private/resolve.ts:226:29)
    at Object.resolve (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tokens/private/resolve.ts:75:16)
    at TokenizedStringFragments.mapTokens (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tokens/string-fragments.ts:173:34)
    at DefaultTokenResolver.resolveString (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tokens/resolvable.ts:213:22)
    at resolve (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tokens/private/resolve.ts:156:30)
    at Object.resolve (/home/ubuntu/CloudServiceTree/node_modules/cdktf/lib/tokens/private/resolve.ts:75:16)
ERROR: cdktf encountered an error while synthesizing

what do I miss?

Hi @fdervisi,

could you share a reproducable example on Github and share a link to that, so I can give it a better look?

Hi @ansgarm, I shared with you my private repo.

My goal is to fetch the resolved provisioningKeyString into an other stack:

https://github.com/fdervisi/CloudServiceTree/blob/master/lib/zpa-provisionin-key-stack.ts

import { Construct } from 'constructs';
import {
  appConnectorGroup,
  provisioningKey,
  Region,
  Vpc as IVpc,
} from './CloudServiceTreeInterface';
import { AppConnectorGroup } from '../.gen/providers/zpa/app-connector-group';
import { ProvisioningKey } from '../.gen/providers/zpa/provisioning-key';
import { DataZpaEnrollmentCert } from '../.gen/providers/zpa/data-zpa-enrollment-cert';
import { DataZpaProvisioningKey } from '../.gen/providers/zpa/data-zpa-provisioning-key';
import { ZpaProvider } from '../.gen/providers/zpa/provider';
import { TerraformLocal, TerraformOutput, TerraformStack } from 'cdktf';

interface ZscalerProvisioningKeyStackConfig {
  vpc: IVpc;
  region: Region;
  appConnectorGroup: appConnectorGroup;
  provisioningKey: provisioningKey;
}

export class ZscalerProvisioningKeyStack extends TerraformStack {
  zpaProvisioningKey: any;
  provisioningKeyData: any;
  provisioningKeyOutput: any;
  provisioningKeyString: any;
  output: any;
  tempKey: any;
  constructor(
    scope: Construct,
    id: string,
    config: ZscalerProvisioningKeyStackConfig
  ) {
    super(scope, id);

    new ZpaProvider(this, 'Zscaler');

    const zpaEnrollmentCert = new DataZpaEnrollmentCert(
      this,
      `enfrollment-cert-vpc-${config.vpc.name}-${config.region.name}`,
      {
        name: 'Connector',
      }
    );

    const appConnectorGroup = new AppConnectorGroup(
      this,
      `app-connector-group-vpc-${config.vpc.name}-${config.region.name}`,
      {
        latitude: config.appConnectorGroup.latitude,
        location: config.appConnectorGroup.location,
        name: config.appConnectorGroup.name,
        longitude: config.appConnectorGroup.longitude,
        versionProfileName: config.appConnectorGroup.versionProfileName,
        cityCountry: config.appConnectorGroup.cityCountry || undefined,
        countryCode: config.appConnectorGroup.countryCode || undefined,
        dnsQueryType: config.appConnectorGroup.dnsQueryType || undefined,
        overrideVersionProfile:
          config.appConnectorGroup.overrideVersionProfile || undefined,
      }
    );

    this.zpaProvisioningKey = new ProvisioningKey(
      this,
      `zpa-provisioning-key-vpc-${config.vpc.name}-${config.region.name}`,
      {
        maxUsage: config.provisioningKey.maxUsage,
        name: config.provisioningKey.name,
        associationType: config.provisioningKey.associationType,
        enrollmentCertId: zpaEnrollmentCert.id,
        zcomponentId: appConnectorGroup.id,
        dependsOn: [appConnectorGroup],
      }
    );
    this.provisioningKeyData = new DataZpaProvisioningKey(
      this,
      `data-provisioning-key-vpc-${config.vpc.name}-${config.region.name}`,
      {
        name: this.zpaProvisioningKey.name,
        associationType: config.provisioningKey.associationType,
      }
    );
    this.provisioningKeyString = new TerraformLocal(
      this,
      `provisioning-key-string-vpc-${config.vpc.name}-${config.region.name}`,
      this.provisioningKeyData.provisioningKey
    );

    this.output = new TerraformOutput(this, `output`, {
      value: this.provisioningKeyString,
    });
  }
  get provisioningKey(): string {
    return this.provisioningKeyString;
  }
  set provisioningKey(provisioningKey) {
    this.provisioningKeyString = provisioningKey;
  }
}

This is the output, when I try to access the provisioningKeyString

aws-eu-south-1/vpc0/subnet1/asg1_0/provisioning-key-stack-vpc-fdervisi-vpc1-eu-south-1/provisioning-key-string-vpc-fdervisi-vpc1-eu-south-1

but it should be this value, as I can see on TerraformOutput:

  provisioning-key-stack-vpc-fdervisi-vpc1-eu-south-1
                              output = 4|api.private.zscaler.com|bDVegD3f9iEu4xdiuHSRz4Q/QQtTnmyhQiaJ4PVOZ3LGi+75dgKCUo78hhUwo0TtAwuAOHPn9R <snip>

Hi @fdervisi :wave:

Unfortunately I won’t be able to debug your code base. I was looking for a minimal reproduction case instead that can be shared publicly.

Hi @ansgarm , first of all thank you very much for your help! My goal was to give you more context, however most of the informations are already here.

I try to call the stack and use the provisioningKeyString in a template which then will be used as user_data for a EC2 instance.

        this.provisioningKey = new ZscalerProvisioningKeyStack(
          this,
          `provisioning-key-stack-vpc-${config.vpc.name}-${config.region.name}`,
          {
            appConnectorGroup: config.vpc.appConnector.appConnectorGroup,
            vpc: config.vpc,
            provisioningKey: config.vpc.appConnector.provisioningKey,
            region: config.region,
          }
        );
        const testTemplate = `echo "${this.provisioningKey.provisioningKeyString}"> /opt/zscaler/var/provision_key`;

however the parameter will not be resolved as you can see here:

echo aws-eu-south-1/vpc0/subnet1/asg1_0/provisioning-key-stack-vpc-fdervisi-vpc1-eu-south-1/provisioning-key-string-vpc-fdervisi-vpc1-eu-south-1

and here is the Stack again:

import { Construct } from 'constructs';
import {
  appConnectorGroup,
  provisioningKey,
  Region,
  Vpc as IVpc,
} from './CloudServiceTreeInterface';
import { AppConnectorGroup } from '../.gen/providers/zpa/app-connector-group';
import { ProvisioningKey } from '../.gen/providers/zpa/provisioning-key';
import { DataZpaEnrollmentCert } from '../.gen/providers/zpa/data-zpa-enrollment-cert';
import { DataZpaProvisioningKey } from '../.gen/providers/zpa/data-zpa-provisioning-key';
import { ZpaProvider } from '../.gen/providers/zpa/provider';
import { TerraformLocal, TerraformOutput, TerraformStack } from 'cdktf';

interface ZscalerProvisioningKeyStackConfig {
  vpc: IVpc;
  region: Region;
  appConnectorGroup: appConnectorGroup;
  provisioningKey: provisioningKey;
}

export class ZscalerProvisioningKeyStack extends TerraformStack {
  zpaProvisioningKey: any;
  provisioningKeyData: any;
  provisioningKeyOutput: any;
  provisioningKeyString: any;
  output: any;
  tempKey: any;
  constructor(
    scope: Construct,
    id: string,
    config: ZscalerProvisioningKeyStackConfig
  ) {
    super(scope, id);

    new ZpaProvider(this, 'Zscaler');

    const zpaEnrollmentCert = new DataZpaEnrollmentCert(
      this,
      `enfrollment-cert-vpc-${config.vpc.name}-${config.region.name}`,
      {
        name: 'Connector',
      }
    );

    const appConnectorGroup = new AppConnectorGroup(
      this,
      `app-connector-group-vpc-${config.vpc.name}-${config.region.name}`,
      {
        latitude: config.appConnectorGroup.latitude,
        location: config.appConnectorGroup.location,
        name: config.appConnectorGroup.name,
        longitude: config.appConnectorGroup.longitude,
        versionProfileName: config.appConnectorGroup.versionProfileName,
        cityCountry: config.appConnectorGroup.cityCountry || undefined,
        countryCode: config.appConnectorGroup.countryCode || undefined,
        dnsQueryType: config.appConnectorGroup.dnsQueryType || undefined,
        overrideVersionProfile:
          config.appConnectorGroup.overrideVersionProfile || undefined,
      }
    );

    this.zpaProvisioningKey = new ProvisioningKey(
      this,
      `zpa-provisioning-key-vpc-${config.vpc.name}-${config.region.name}`,
      {
        maxUsage: config.provisioningKey.maxUsage,
        name: config.provisioningKey.name,
        associationType: config.provisioningKey.associationType,
        enrollmentCertId: zpaEnrollmentCert.id,
        zcomponentId: appConnectorGroup.id,
        dependsOn: [appConnectorGroup],
      }
    );
    this.provisioningKeyData = new DataZpaProvisioningKey(
      this,
      `data-provisioning-key-vpc-${config.vpc.name}-${config.region.name}`,
      {
        name: this.zpaProvisioningKey.name,
        associationType: config.provisioningKey.associationType,
      }
    );
    this.provisioningKeyString = new TerraformLocal(
      this,
      `provisioning-key-string-vpc-${config.vpc.name}-${config.region.name}`,
      this.provisioningKeyData.provisioningKey
    );

    this.output = new TerraformOutput(this, `output`, {
      value: this.provisioningKeyString,
    });
  }
  get provisioningKey(): string {
    return this.provisioningKeyString;
  }
  set provisioningKey(provisioningKey) {
    this.provisioningKeyString = provisioningKey;
  }
}

However, I Terraform Output resolve the stack correctly:

provisioning-key-stack-vpc-fdervisi-vpc1-eu-south-1
                              output = 4|api.private.zscaler.com|bDVegD3f9iEu4xdiuHSRz4Q/QQtTnmyhQiaJ4PVOZ3LGi+75dgKCUo78hhUwo0TtAwuAOHPn9R <snip>

I don’t understand why it works in TerraformOutput but not if I call the parameter of the stack?

Hi @fdervisi

Thank you for being patient with me!
I just spent some more time with your code and it appears that there is a bug in there that is pretty hard to figure out:

This function states that it returns a string whereas it really is returning a TerraformLocal

get provisioningKey(): string {
    return this.provisioningKeyString;
}

When using that variable here:

const testTemplate = `echo "${this.provisioningKey.provisioningKeyString}"> /opt/zscaler/var/provision_key`;

It will actually receive a TerraformLocal and TypeScript will turn this into a string by using its toString() method as it is used in a template string which will ultimately return the address of that construct node (which is aws-eu-south-1/vpc0/subnet1/asg1_0/provisioning-key-stack-vpc-fdervisi-vpc1-eu-south-1/provisioning-key-string-vpc-fdervisi-vpc1-eu-south-1).

Whereas your TerraformOutput directly gets passed the TerraformLocal which won’t cause toString() to be called and it will resolve properly because a TerraformLocal can be resolved.

One thing that made this harder to discover, is the any used here:

provisioningKeyString: any;

Without that line, TypeScript could’ve detected that provisioningKey is not actually a string which might’ve helped find this quirk.

My proposed mitigation would be to actually return a string in that getter by changing it to:

get provisioningKey(): string {
    return this.provisioningKeyString.asString();
}

The any could also be replaced with:

provisioningKeyString: TerraformLocal;

That said, there maybe might a way for us to catch this class of errors and print a warning to help when referencing locals in template strings.

@ansgarm thanks for the conformation that there is something wrong, I check everything :slight_smile:

I adjusted the code with

provisioningKeyString: TerraformLocal;

however I get this error:

This expression is not callable because it is a 'get' accessor. Did you mean to use it without '()'?
  Type 'String' has no call signatures.

and this:

Type 'string' is not assignable to type 'TerraformLocal'.

BTW: Normally it should also work if I just directly access without TerraformLocal

this.provisioningKeyData.provisioningKey

at leaset this works when I do it for the AWS provider. Could it be, that there is something wrong with the provider?

import { Construct } from 'constructs';
import {
  appConnectorGroup,
  provisioningKey,
  Region,
  Vpc as IVpc,
} from './CloudServiceTreeInterface';
import { AppConnectorGroup } from '../.gen/providers/zpa/app-connector-group';
import { ProvisioningKey } from '../.gen/providers/zpa/provisioning-key';
import { DataZpaEnrollmentCert } from '../.gen/providers/zpa/data-zpa-enrollment-cert';
import { DataZpaProvisioningKey } from '../.gen/providers/zpa/data-zpa-provisioning-key';
import { ZpaProvider } from '../.gen/providers/zpa/provider';
import { TerraformLocal, TerraformOutput, TerraformStack } from 'cdktf';

interface ZscalerProvisioningKeyStackConfig {
  vpc: IVpc;
  region: Region;
  appConnectorGroup: appConnectorGroup;
  provisioningKey: provisioningKey;
}

export class ZscalerProvisioningKeyStack extends TerraformStack {
  zpaProvisioningKey: any;
  provisioningKeyData: any;
  provisioningKeyOutput: any;
  provisioningKeyString: TerraformLocal;
  output: any;
  tempKey: any;
  constructor(
    scope: Construct,
    id: string,
    config: ZscalerProvisioningKeyStackConfig
  ) {
    super(scope, id);

    new ZpaProvider(this, 'Zscaler');

    const zpaEnrollmentCert = new DataZpaEnrollmentCert(
      this,
      `enfrollment-cert-vpc-${config.vpc.name}-${config.region.name}`,
      {
        name: 'Connector',
      }
    );

    const appConnectorGroup = new AppConnectorGroup(
      this,
      `app-connector-group-vpc-${config.vpc.name}-${config.region.name}`,
      {
        latitude: config.appConnectorGroup.latitude,
        location: config.appConnectorGroup.location,
        name: config.appConnectorGroup.name,
        longitude: config.appConnectorGroup.longitude,
        versionProfileName: config.appConnectorGroup.versionProfileName,
        cityCountry: config.appConnectorGroup.cityCountry || undefined,
        countryCode: config.appConnectorGroup.countryCode || undefined,
        dnsQueryType: config.appConnectorGroup.dnsQueryType || undefined,
        overrideVersionProfile:
          config.appConnectorGroup.overrideVersionProfile || undefined,
      }
    );

    this.zpaProvisioningKey = new ProvisioningKey(
      this,
      `zpa-provisioning-key-vpc-${config.vpc.name}-${config.region.name}`,
      {
        maxUsage: config.provisioningKey.maxUsage,
        name: config.provisioningKey.name,
        associationType: config.provisioningKey.associationType,
        enrollmentCertId: zpaEnrollmentCert.id,
        zcomponentId: appConnectorGroup.id,
        dependsOn: [appConnectorGroup],
      }
    );
    this.provisioningKeyData = new DataZpaProvisioningKey(
      this,
      `data-provisioning-key-vpc-${config.vpc.name}-${config.region.name}`,
      {
        name: this.zpaProvisioningKey.name,
        associationType: config.provisioningKey.associationType,
      }
    );
    this.provisioningKeyString = new TerraformLocal(
      this,
      `provisioning-key-string-vpc-${config.vpc.name}-${config.region.name}`,
      this.provisioningKeyData.provisioningKey
    );

    this.output = new TerraformOutput(this, `output`, {
      value: this.provisioningKeyString,
    });
  }
  get provisioningKey(): string {
    return this.provisioningKeyString.asString();
  }
  set provisioningKey(provisioningKey) {
    this.provisioningKeyString = provisioningKey;
  }
}

Hi @fdervisi :wave:

Oh, my bad. It needs to be:

get provisioningKey(): string {
    return this.provisioningKeyString.asString;
}

without the ().

Hi @ansgarm ,

Thank you for your assistance. I have implemented your suggestion, but I am still encountering an issue:

echo "$local.vpc0_subnet1_asg1_0_provisioning-key-stack-vpc-fdervisi-vpc1-eu-south-1_provisioning-key-string-vpc-fdervisi-vpc1-eu-south-1_11242F40"> /opt/zscaler/var/provision_key

I believe the issue may be related to how I am calling the stack. When I use Construct,"nothing is resolved, but when I try to use TerraformStack I receive an error. I suspect that I need to add a dependency somewhere, but I am unsure of where to add it.

Error: Usage Error: The following dependencies are not included in the stacks to run: aws-eu-south-1/vpc0/subnet1/asg1_0/provisioning-key-stack-vpc-fdervisi-vpc1-eu-south-1. Either add them or add the --ignore-missing-stack-dependencies flag.

Below is the relevant code. Perhaps you can help me identify the issue.

import { Construct } from 'constructs';
import {
  appConnectorGroup,
  provisioningKey,
  Region,
  Vpc as IVpc,
} from './CloudServiceTreeInterface';
import { AppConnectorGroup } from '../.gen/providers/zpa/app-connector-group';
import { ProvisioningKey } from '../.gen/providers/zpa/provisioning-key';
import { DataZpaEnrollmentCert } from '../.gen/providers/zpa/data-zpa-enrollment-cert';
import { DataZpaProvisioningKey } from '../.gen/providers/zpa/data-zpa-provisioning-key';
import { ZpaProvider } from '../.gen/providers/zpa/provider';
import { TerraformLocal, TerraformOutput } from 'cdktf';

interface ZscalerProvisioningKeyStackConfig {
  vpc: IVpc;
  region: Region;
  appConnectorGroup: appConnectorGroup;
  provisioningKey: provisioningKey;
}

export class ZscalerProvisioningKeyStack extends Construct {
  zpaProvisioningKey: any;
  provisioningKeyData: any;
  provisioningKeyOutput: any;
  provisioningKeyString: TerraformLocal;
  output: any;
  tempKey: any;
  constructor(
    scope: Construct,
    id: string,
    config: ZscalerProvisioningKeyStackConfig
  ) {
    super(scope, id);

    new ZpaProvider(this, 'Zscaler');

    const zpaEnrollmentCert = new DataZpaEnrollmentCert(
      this,
      `enfrollment-cert-vpc-${config.vpc.name}-${config.region.name}`,
      {
        name: 'Connector',
      }
    );

    const appConnectorGroup = new AppConnectorGroup(
      this,
      `app-connector-group-vpc-${config.vpc.name}-${config.region.name}`,
      {
        latitude: config.appConnectorGroup.latitude,
        location: config.appConnectorGroup.location,
        name: config.appConnectorGroup.name,
        longitude: config.appConnectorGroup.longitude,
        versionProfileName: config.appConnectorGroup.versionProfileName,
        cityCountry: config.appConnectorGroup.cityCountry || undefined,
        countryCode: config.appConnectorGroup.countryCode || undefined,
        dnsQueryType: config.appConnectorGroup.dnsQueryType || undefined,
        overrideVersionProfile:
          config.appConnectorGroup.overrideVersionProfile || undefined,
      }
    );

    this.zpaProvisioningKey = new ProvisioningKey(
      this,
      `zpa-provisioning-key-vpc-${config.vpc.name}-${config.region.name}`,
      {
        maxUsage: config.provisioningKey.maxUsage,
        name: config.provisioningKey.name,
        associationType: config.provisioningKey.associationType,
        enrollmentCertId: zpaEnrollmentCert.id,
        zcomponentId: appConnectorGroup.id,
        dependsOn: [appConnectorGroup],
      }
    );
    this.provisioningKeyData = new DataZpaProvisioningKey(
      this,
      `data-provisioning-key-vpc-${config.vpc.name}-${config.region.name}`,
      {
        name: this.zpaProvisioningKey.name,
        associationType: config.provisioningKey.associationType,
      }
    );
    this.provisioningKeyString = new TerraformLocal(
      this,
      `provisioning-key-string-vpc-${config.vpc.name}-${config.region.name}`,
      this.provisioningKeyData.provisioningKey
    );

    this.output = new TerraformOutput(this, `output`, {
      value: this.provisioningKeyString,
    });
  }
  get provisioningKey(): string {
    return this.provisioningKeyString.asString;
  }
  // set provisioningKey(provisioningKey) {
  //   this.provisioningKeyString.asString = provisioningKey;
  // }
}

and this is the parent class where the stack is being called:

import { AutoscalingGroup } from '@cdktf/provider-aws/lib/autoscaling-group';
import { AutoscalingPolicy } from '@cdktf/provider-aws/lib/autoscaling-policy';
import { DataAwsAmi } from '@cdktf/provider-aws/lib/data-aws-ami';
import { LaunchTemplate } from '@cdktf/provider-aws/lib/launch-template';
import { Fn } from 'cdktf';
import { Construct } from 'constructs';
import * as fs from 'fs';
import { renderTemplate } from './render-template';
import { File } from '@cdktf/provider-local/lib/file';
import { LocalProvider } from '@cdktf/provider-local/lib/provider';
import {
  IAsg,
  //   INics,
  //   Instance as IInstance, // interface for a Instance object
  Region,
  Vpc,
} from './CloudServiceTreeInterface';
import { SecurityGroup } from '@cdktf/provider-aws/lib/security-group';
import { ZscalerProvisioningKeyStack } from './zpa-provisionin-key-stack';

interface AwsAutoScalingGroupStackConfig {
  autoScalingGroup: IAsg;
  subnetIndex: number;
  region: Region;
  subnetId: string;
  asgIndex: number;
  vpcId: string;
  vpc: Vpc;
}

interface DataAwsAmiFilterProps {
  name: string;
  values: string[];
}

export class AwsAutoScalingGroupStack extends Construct {
  instance: any;
  temp: any;
  securityGroup: any;
  amiFilterProps: DataAwsAmiFilterProps[] = [];
  // launchTemplate: LaunchTemplate={};
  nicList: any[] = [];
  provisioningKey: any;
  renderedTemplate: any;
  zpaProvisioningKey: any;
  zpaProvisioningKeyObject: { provisioning_key: string } = {
    provisioning_key: '',
  };

  constructor(
    scope: Construct,
    id: string,
    config: AwsAutoScalingGroupStackConfig
  ) {
    super(scope, id);

    if (config.autoScalingGroup.zpa) {
      if (config.vpc.appConnector) {
        this.provisioningKey = new ZscalerProvisioningKeyStack(
          this,
          `provisioning-key-stack-vpc-${config.vpc.name}-${config.region.name}`,
          {
            appConnectorGroup: config.vpc.appConnector.appConnectorGroup,
            vpc: config.vpc,
            provisioningKey: config.vpc.appConnector.provisioningKey,
            region: config.region,
          }
        );


        // const testTemplate = fs.readFileSync(
        //   `user_data/${config.autoScalingGroup.userDataTemplate}`,
        //   'utf-8'
        // );

        // this.provisioningKey.output.value
        const testTemplate = `echo "${this.provisioningKey.provisioningKey}"> /opt/zscaler/var/provision_key`;
        // Initialize the local provider

        new LocalProvider(this, 'local');

        new File(this, 'UserDataFile', {
          content: testTemplate,
          filename: '../../../user_data.txt',
        });
        //this.renderedTemplate = fs.readFileSync('user_data.txt', 'utf8');

        // const testLocal = new TerraformLocal(
        //   this,
        //   `user-data-asg-${config.asgIndex}`,
        //   `echo "${this.provisioningKey.provisioningKeyString}"> /opt/zscaler/var/provision_key`
        // );
        //console.log(testLocal);
        this.renderedTemplate = `echo "${this.provisioningKey.provisioningKey}"> /opt/zscaler/var/provision_key`;
        console.log('********************* renderTemplate ******************');
        console.log(this.renderedTemplate);
        console.log('********************* renderTemplate ******************');
        //new TerraformOutput(this, 'test', { value: testLocal });
      }
    }

    // search for AMI with Filter
    const amiFilter = new DataAwsAmi(
      this,
      `ami${config.subnetIndex}${config.asgIndex}`,
      {
        filter: this.amiFilterProps,
        mostRecent: config.autoScalingGroup.amiFilter?.mostRecent,
        owners: config.autoScalingGroup.amiFilter?.owners || undefined,
      }
    );

    if (config.autoScalingGroup.userDataTemplate) {
      // Set default value for userDataVars if it is not provided
      const userDataVars = config.autoScalingGroup.userDataVars || {};
      // Read the contents of the specified user data template file
      const templateString = fs.readFileSync(
        `user_data/${config.autoScalingGroup.userDataTemplate}`,
        'utf-8'
      );

      // Render the template with the given variables
      // this.renderedTemplate = 'TESET';
      if (!config.autoScalingGroup.zpa) {
        this.renderedTemplate = renderTemplate(templateString, userDataVars);
      }

      // Set the rendered template as the user data for the instance
      config.autoScalingGroup.userData = Fn.base64encode(
        //Fn.rawString(this.renderedTemplate)
        Fn.rawString(this.renderedTemplate)
      );
    }

    
    <-- snip -->

  }
}

Hi @fdervisi,

thank you for sharing the code and for being patient with me. Unfortunately, I won’t be able to continue debugging it. However, if you are able to isolate the specific issue you are encountering into a condensed example that only contains the relevant parts that make it fail, please file an issue report on our main Github repository.

– Ansgar

@ansgarm thanks for all your help! I will open a issue…