Mixing legacy HCL and AwsTerraformAdapter

Hi! I’m having trouble using existing Terraform modules in a step function using the new AwsTerraformAdapter in CDK. We have some existing Lambdas (imported from HCL modules) and would like to pass in their ARNs to the state machine defined in Terraform CDK, but the AWS library complains that the Terraform CDK token doesn’t look like an ARN.

To give some additional context, our process is to run cdktf synth and then include the JSON output within our parent Terraform module, e.g.

# main.tf
module "cdksfn" {
  source = "./cdk.out"
  lambda-arn = module.my_lambda.lambda.arn
# cdk/main.ts

class MyStack extends TerraformStack {

  myLambdaArn = new TerraformVariable(this, 'lambda-arn', { type: "string" });

  constructor(scope: Construct, name: string) {
    super(scope, name);

    new AwsProvider(this, "aws", { region: 'us-west-2' });
    const awsAdapter = new AwsTerraformAdapter(this, "adapter");

    new stepfunctions.StateMachine(awsAdapter, "sfn", {
      definition: new aws_stepfunctions_tasks.LambdaInvoke(awsAdapter, "my-lambda-step", {
        lambdaFunction: lambda.Function.fromFunctionArn(
          // Apparently the Terraform variable below can't be used as the ARN 
          // because it represents a TF token
      }).next(new stepfunctions.Succeed(awsAdapter, "success"))

The above stack fails to synthesize with

ARNs must start with "arn:" and have at least 6 components: ${TfToken[TOKEN.1]}

Is it possible to use a Terraform variable within a step function definition defined in CDK (using the AwsTerraformAdapter)?

Could use create an AWS CDK Token that wraps the value? Something like cdk.Token.asString(this.myLambdaArn.stringValue).

Certainly not the most ideal, but could be a workaround.

Thanks @jsteinich! Gave that a try but still got the same error as above. I then tried using

    const lambdaArn = this.myLambdaArn.stringValue;
    awscdk.Lazy.string({ produce() { return lambdaArn; } })

and this allowed me to synth successfully, but the tf plan subsequently failed. It looks like doing it this way results in a state machine definition that isn’t properly escaped:

"aws_cloudcontrolapi_resource": {
      "adapter_sfn5CB133D2_8A34CE5C": {
        "//": {
          "metadata": {
            "path": "cdk/adapter/sfn5CB133D2",
            "uniqueId": "adapter_sfn5CB133D2_8A34CE5C"
        "desired_state": "${jsonencode({RoleArn = aws_iam_role.adapter_sfnRoleF032151D_E74BDE95.arn, DefinitionString = join(\"\", [\"{\\\"StartAt\\\":\\\"my-lambda-step\\\",\\\"States\\\":{\\\"my-lambda-step\\\":{\\\"Next\\\":\\\"success\\\",\\\"Retry\\\":[{\\\"ErrorEquals\\\":[\\\"Lambda.ServiceException\\\",\\\"Lambda.AWSLambdaException\\\",\\\"Lambda.SdkClientException\\\"],\\\"IntervalSeconds\\\":2,\\\"MaxAttempts\\\":6,\\\"BackoffRate\\\":2}],\\\"Type\\\":\\\"Task\\\",\\\"Resource\\\":\\\"arn:\", data.aws_partition.adapter_aws-partition_5B16AD9D.partition, \"\":states:::lambda:invoke\",\"Parameters\":{\"FunctionName\":\"${var.lambda-arn}\",\"Payload.$\":\"$\"}},\"success\":{\"Type\":\"Succeed\"}}}\"\"])})}",
        "type_name": "AWS::StepFunctions::StateMachine"

i.e. note that it almost looks correct in that the FunctionName is set to "${var.lambda-arn}", but a) I don’t think TF will know to correctly interpolate that since it’s a nested interpolation and b) everything after the ${var.lambda-arn} in the state machine definition is not double-escaped, i.e. \" should be \\\" in most places but it’s not

Hopefully I’ll have a chance to dig into this a little more and see if I can uncover a bug somewhere in the cdktf code. But would love to hear if anyone else has suggestions for potential workaround.

@ansgarm any ideas here? I don’t believe full token interop is supported, but seems like there should be a workaround.

One extreme workaround would be to use a stack level escape hatch to set the desired_state of the state machine.

No immediate idea yet, but I was already lurking here and will have a look at this issue early next week :slight_smile:

1 Like

Hi :wave:

I did quite a bit of debugging today and probably found the underlying issue.
This part of the adapter is responsible for escaping strings that contain quotes ("). It was supposed to ignore strings that are just "${TfToken[112]}" but the check ignores all strings containing tokens. This is the case here and of course it needs to be properly detected and wrapped to support passing CDKTF Tokens to AWS CDK constructs.

I started to build a fix which will work similar to these internals in TFExpression. I’ll report back here as soon as I got a working fix and a PR / new release up.

tl;dr: The adapter needs to support “mapping” CDKTF Tokens in strings.

PR with a fix is up: fix string token interop by ansgarm · Pull Request #69 · hashicorp/cdktf-aws-cdk · GitHub
I was able to deploy the included example successfully to AWS, so this should resolve your issue.
As soon as the PR is merged, a new release of the adapter will be triggered automatically.

edit: Merged and released: Release v0.3.4 · hashicorp/cdktf-aws-cdk · GitHub
(I had to edit instead of posting a new message, because Discuss does not allow three consecutive messages from the same person without replies in between)