Value from previous resource seems not to be used

Terraform
1

I can’t get Terraform to use values from previous resources. To be specific I get:

│ Error: echo-server failed to create kubernetes rest client for update of resource: Get "http://localhost/api?timeout=32s": dial tcp [::1]:80: connect: connection refused

Ofcourse, it’s not suppose to use localhost. I need it to use google_container_cluster.primary.endpoint like so:

resource "null_resource" "wait_for_cluster" {
  depends_on = [google_container_cluster.primary]
}

provider "kubectl" {
  host                   = google_container_cluster.primary.endpoint
  client_certificate     = base64decode(google_container_cluster.primary.master_auth.0.client_certificate)
  client_key             = base64decode(google_container_cluster.primary.master_auth.0.client_key)
  cluster_ca_certificate = base64decode(google_container_cluster.primary.master_auth.0.cluster_ca_certificate)
}


resource "kubectl_manifest" "namespace" {
  depends_on = [null_resource.wait_for_cluster]

  yaml_body = <<-EOT
  apiVersion: v1
  kind: Namespace
  metadata:
    name: echo-server
  EOT
}

What is happening, I think, is that the google_container_cluster.primary.endpoint somehow is not being used? I’m not sure.

I tried setting

resource "kubectl_manifest" "namespace" {
  ...
  load_config_file       = false
}

And got:

echo-server failed to create kubernetes rest client for update of resource: unknown
│ 
│   with kubectl_manifest.namespace,
│   on main.tf line 82, in resource "kubectl_manifest" "namespace":
│   82: resource "kubectl_manifest" "namespace" ***
│

The even stanger part is that this works on localhost (without load_config_file = false ofcourse).

So how do I fix this?

Thanks kindly for your help and concern.

2

Hi @jenia,

I think the problem is probably that the kubectl provider needs the complete configuration in order to create a plan, but the cluster details can’t be known until after those changes are applied.

You need to create the infrastructure first, usually via a separate configuration, then apply the k8s on top of that infrastructure via a second configuration. It’s possible to also use -target to apply the individual layers in most cases from a single config, but that isn’t as robust a setup.

3

Makes sense. I broke up the dependent resources into their own terraform “layers” and I’m using data blocks to fetch the relevant info in the dependent resources.
I call terraform apply ... in each of the layers. I think that the regular setup?

But then the question arises, what is the depends_on = [...] keyword used for? It seems to be a mistake to chain dependent resources using depends_on, rather you need to use a separate terraform layer (or folder) for each?

4

depends_on is used to create a reference between resources when there is a dependency but no applicable natural attribute to reference and assign in the configuration. It ensures the order of evaluation; kubectl_manifest.namespace will be always be planned after google_container_cluster.primary, and any changes to kubectl_manifest.namespace will be applied after google_container_cluster.primary. What it can’t do is cause kubectl_manifest.namespace to be planned after google_container_cluster.primary is applied, because the plan must happen before apply.

Also note that your null_resource.wait_for_cluster indirection does nothing here, putting depends_on = [google_container_cluster.primary] directly in the kubectl_manifest.namespace would create the same dependency.

5

I see. So depends_on is used when you only need something to exist but you don’t need to know any of its values downsteam? That way the planner can statically plan its work?

6

Basically yes, though a plan is not statically determined, the individual instance changes are planned by the provider.

More fundamentally, terraform determines the order of operations via references between resources, and depends_on is a way to add a reference without using any attributes. If B references A, then A will be planned before B, and any changes to A will be applied before any changes to B.