Error referencing data resource

Terraform
1

Hi,
I have a crypto-keys module, through which I create key rings and keys for different regions.
I can see in google console that the rings and keys got created fine.

Now in a different module I reference them as data resources, like this,

data "google_kms_key_ring" "disk" {
  for_each = toset(local.list_regions)

  name     = "${var.cluster_name}-${each.value}-disk-${data.terraform_remote_state.crypto-keys.outputs.crypto_keys_module_outputs.random.hex}"
  location = each.value
}

data "google_kms_crypto_key" "disk" {
  for_each = toset(local.list_regions)

  key_ring = data.google_kms_key_ring.disk[each.key].id
  name = "${var.cluster_name}-${each.value}-disk-${data.terraform_remote_state.crypto-keys.outputs.crypto_keys_module_outputs.random.hex}"
}

And I reference them like this,

resource "google_kms_crypto_key_iam_binding" "disk" {
  for_each      = toset(local.list_regions)

  crypto_key_id = data.google_kms_crypto_key.disk[each.key].id
  role          = "roles/cloudkms.cryptoKeyEncrypterDecrypter"

  members = [
    "serviceAccount:${google_project_service_identity.apigee.email}",
  ]
}

Now terraform plan is throwing an error saying,

Error: Missing required argument

  on ../../../modules/apigee-x/server/main.tf line 70, in resource "google_kms_crypto_key_iam_binding" "disk":
  70:   crypto_key_id = data.google_kms_crypto_key.disk[each.key].id

The argument "crypto_key_id" is required, but no definition was found.

Is that not how data resources are referenced?

2

Hi @arun-a-nayagam,

I’m not familiar with this particular data source, but it seems like what’s happened here is that it is returning an id attribute set to null, which Terraform is then understanding as you wanting to leave crypto_key_id unset (because that’s what null represents in a resource configuration).

However, that’s a pretty unusual thing for a data source to do, and I can’t see any case where that should be true in the implementation of this data source:

So I think something else is happening here, but I’m not sure what exactly it is. I think to understand better what’s happening it would help to see the exact values that this data resource is returning. One way to achieve that could be to temporarily comment out your resource "google_kms_crypto_key_iam_binding" "disk" block, to prevent the error, and then apply the result just so that the data resource results will be recorded in the current state, and then you can hopefully use terraform console to inspect its values by typing just data.google_kms_crypto_key.disk at the console prompt.

3

Hi @apparentlymart ,

Thank you for the response.
I did a terraform apply and went into the terraform console, this is what I see,

> data.google_kms_crypto_key.disk

> ╷
│ Error: Reference to undeclared resource
│
│   on <console-input> line 1:
│   (source code not available)
│
│ A data resource "google_kms_crypto_key" "disk" has not been declared in the root module.
╵

Perhaps my module composition isn’t right?

For now, I have temporarily got around the issue using a rather lengthy state file reference like this,

data.terraform_remote_state.crypto-keys.outputs.crypto_keys_module_outputs.disk[each.value]

4

Hi @arun-a-nayagam,

I’m sorry I didn’t notice that your resources you showed here belong to a nested module, not to the root module. Unfortunately terraform console can only work with objects in the root module, but perhaps instead you can find the relevant data resource in the terraform show output.

In that command’s output you should see all of the resources that Terraform is currently tracking in the state, which should hopefully include data "google_kms_crypto_key" "disk" with a comment above it saying that it belongs to the module where you saw this error.

5

Hi @apparentlymart , here’s the terraform show output,

# module.crypto-key.google_kms_crypto_key.disk["us-east4"]:
resource "google_kms_crypto_key" "disk" {
    id                            = "projects/tf-automation/locations/us-east4/keyRings/tf-x-us-east4-disk-a4e2/cryptoKeys/tf-x-us-east4-disk-a4e2"
    key_ring                      = "projects/tf-automation/locations/us-east4/keyRings/tf-x-us-east4-disk-a4e2"
    name                          = "tf-x-us-east4-disk-a4e2"
    purpose                       = "ENCRYPT_DECRYPT"
    self_link                     = "projects/tf-automation/locations/us-east4/keyRings/tf-x-us-east4-disk-a4e2/cryptoKeys/tf-x-us-east4-disk-a4e2"
    skip_initial_version_creation = false

    version_template {
        algorithm        = "GOOGLE_SYMMETRIC_ENCRYPTION"
        protection_level = "SOFTWARE"
    }
}
6

Hi @arun-a-nayagam,

This seems to be a resource "google_kms_crypto_key" "disk" rather than a data "google_kms_crypto_key" "disk" block – notice the resource rather than data. Do you see a data block in the terraform show output too?

Note that it’s not expected for a particular Terraform configuration to both manage an object (using a resource block) and read that same object (using a data block), because that can cause ordering problems if Terraform tries to read the object before it’s been created (since those two operations are independent as far as Terraform is concerned). I don’t know if that’s related to what we’re discussing, but if you do have both a resource and data block for the same object in your configuration (even if in two separate modules) then it would be good to mention that since it might help explain the strange behavior here.

7

Hi @apparentlymart , yes, I do create that in a different module and refer to it using a data block in a different module. As I haven’t learnt how to refer other module resources/vars.
And so my approach has been to either refer them through data terraform_remote_state or as data blocks.
There aren’t good examples of how to compose modules and refer resources from one another.

In any case, I run the module that creates the resource first before referring it in a data block in a different module.
You are right, in the data block module it shows up like this, when I do a terraform show

# module.server.data.google_kms_crypto_key.disk["us-east4"]:
data "google_kms_crypto_key" "disk" {}

I have a crypto key defined for one other region, that doesn’t show up at all.

Is there a good tutorial that describes how to access resources across modules. I would ideally like to access them as module.xyz. google_kms_crypto_key.disk.id for example.

8

Same situation here, I create the resource in a separate module, apply that module and then I reference that in different module via data source using a name. Crypto key and ring exist. This works perfectly fine for all other resources so it must be some bug with this particular resource.

This is on v0.12.31 so perhaps it was fixed already though.

9

I think this may be a disconnect between with the Google provider version you are using and the version of the docs you are referencing.

In older versions (3.x) of the Google provider you should use:

data.google_kms_crypto_key.disk[each.key].self_link

For newer versions (4.x) of the Google provider you should use:

data.google_kms_crypto_key.disk[each.key].id