Deperecated Resource Warning on data.null_data_source

Terraform
1

Hi All,

I’ve started seeing a warning on my terraform stacks which leverage data.null_data_source

Warning: Deprecated Resource

The null_data_source was historically used to construct intermediate values to
re-use elsewhere in configuration, the same can now be achieved using locals

I’m not sure how I’m supposed to replace the functionality of this data object with locals as I use for_each on the null_data_source object.

I primarily use the data object with aws lambda running the Golang runtime. I am using terrraform to produce a binary with null_resource and provisioner "local-exec" {}, and then the archive provider to create a zip archive which gets deployed to AWS.

If terraform state sees that the null_resource produced a binary AND the binary doesn’t exist in the output directory (e.g. deleted, or CI is doing the deploy), terraform throws an error and cannot proceed until the binary is manually built and in the correct directory.

My way around this is using the null_data_source. Recall that data objects runs before all resources, null_resource included. As a result, null_data_source checks for the binary, and produces a result depending on if the binary is present or not.

If the binary isn’t there, null_resource runs again.

How do I replicate this with locals? Especially considering that I’m using for_each everywhere. I really really hope that the solution does not involves hard coding a new object in a locals block each time I add a new lambda to be deployed in local.lambdas. That would defeat the whole purpose of using for_each as heavily as I do.

Here’s an example stack. I haven’t included the IAM resources for brevity.

locals {
  lambdas = {
    events_debug_logger = {
      description = "Logs eventbridge events for ${var.service_name}."
      timeout     = 10
    }
  }
}


data "null_data_source" "wait_for_lambda_build" {
  for_each = local.lambdas

  inputs = {
    lambda_build_id = null_resource.lambda_build[each.key].id
    source          = "${path.module}/lambdas/bin/${each.key}"
  }
}

data "archive_file" "this" {
  for_each = local.lambdas

  type        = "zip"
  source_file = data.null_data_source.wait_for_lambda_build[each.key].outputs["source"]
  output_path = "${path.module}/archive/${each.key}.zip"
}

resource "null_resource" "lambda_build" {
  for_each = local.lambdas

  triggers = {
    binary_exists = fileexists("${path.module}/lambdas/bin/${each.key}")

    main = join("", [
      for file in fileset("${path.module}/lambdas/cmd/${each.key}", "*.go") : filebase64("${path.module}/lambdas/cmd/${each.key}/${file}")
    ])
  }

  provisioner "local-exec" {
    command = "export GO111MODULE=on"
  }

  provisioner "local-exec" {
    command = "GOOS=linux go build -ldflags '-s -w' -o ${path.module}/lambdas/bin/${each.key} ${path.module}/lambdas/cmd/${each.key}/."
  }
}

resource "aws_lambda_function" "this" {
  depends_on = [null_resource.lambda_build]
  for_each   = local.lambdas

  filename         = "${path.module}/archive/${each.key}.zip"
  function_name    = "${each.key}_${local.service_name}"
  description      = each.value.description
  role             = aws_iam_role.this.arn
  handler          = each.key
  publish          = false
  source_code_hash = data.archive_file.this[each.key].output_base64sha256
  runtime          = "go1.x"
  timeout          = "10"
  tags             = var.tags
}
2

Figured it out with a level of dynamism that I’m happy with.

The key was making data.archive_file depend on null_resource.lambda_build, and adding a binary_exists trigger to null_resource.lambda_build which is based on local.null.lambda_binary_exists

locals {
  lambdas = {
    events_debug_logger = {
      description = "Logs eventbridge events for ${var.service_name}."
      timeout     = 10
    }
  }

  null = {
    lambda_binary_exists = { for key, _ in local.lambdas : key => fileexists("${path.module}/lambdas/bin/${key}") }
  }
}

data "archive_file" "this" {
  depends_on = [null_resource.lambda_build]
  for_each   = local.lambdas

  type        = "zip"
  source_file = "${path.module}/lambdas/bin/${each.key}"
  output_path = "${path.module}/archive/${each.key}.zip"
}

resource "null_resource" "lambda_build" {
  for_each = local.lambdas

  triggers = {
    binary_exists = local.null.lambda_binary_exists[each.key]

    main = join("", [
      for file in fileset("${path.module}/lambdas/cmd/${each.key}", "*.go") : filebase64("${path.module}/lambdas/cmd/${each.key}/${file}")
    ])
  }

  provisioner "local-exec" {
    command = "export GO111MODULE=on"
  }

  provisioner "local-exec" {
    command = "GOOS=linux go build -ldflags '-s -w' -o ${path.module}/lambdas/bin/${each.key} ${path.module}/lambdas/cmd/${each.key}/."
  }
}

resource "aws_lambda_function" "this" {
  depends_on = [null_resource.lambda_build]
  for_each   = local.lambdas

  filename         = "${path.module}/archive/${each.key}.zip"
  function_name    = "${each.key}_${local.service_name}"
  description      = each.value.description
  role             = aws_iam_role.this.arn
  handler          = each.key
  publish          = false
  source_code_hash = data.archive_file.this[each.key].output_base64sha256
  runtime          = "go1.x"
  timeout          = "10"
  tags             = var.tags
}