Inexplicable dependencies shown by terraform graph from data sources to unrelated resources

Terraform
1

Recently i made a change to our GKE kubernets cluster resource in Terraform. This change prompted a lot of data source to be re-read by Terraform that I couldn’t explain, and any resources that depended on those data sources, to be replaced/modified. It was a LOT of resources. I started to explore dependencies with the terraform graph tool, and got stuck with data sources that are very simple, but for some reason depend on other resources that i have no way of explaining.

Does anyone have an idea why data sources in Terraform can depend on resources they aren’t referencing (in)directly? How could I explore the source of this dependency further? Initially i thought it might be a count I had on the cluster and some of the other resources to introduce an if condition on the resources. I since removed that count statement (it really over complicated it anyway) just to see if that changed anything, but it didn’t. The gke cluster resource is pasted at the end for reference.

I am running terraform 1.8.5 with google provider 5.11.0

I am picking one of the data sources as an example.

I have a data.cloudflare_zone datasource like this:

data "cloudflare_zone" "my-app" {
  name = local.tld
}
locals {
  tld                   = "my-app.co.uk"
}

If I explore the dependencies with terraform graph it shows these dependencies:

"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.google_container_node_pool.primary_preemptible_nodes";
"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.google_project_iam_member.artifactregistryReader";
"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.google_project_iam_member.logWriter";
"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.google_project_iam_member.metricWriter";
"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.google_project_iam_member.monitoringViewer";
"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.google_project_iam_member.resourceMetadataWriter";
"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.kubernetes_cluster_role_binding.gcp_admins";
"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.kubernetes_deployment.my-app-debug";
"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.kubernetes_role_binding.developers";

For some reason the datasource depends on the gke cluster resource in another module, and also on a bunch of unrelated IAM membership resources.

If i check the data source with terraform plan this is the json it shows (no references or depends_on entries).

{
                "address": "module.my-app-api.data.cloudflare_zone.my-app",
                "mode": "data",
                "type": "cloudflare_zone",
                "name": "my-app",
                "provider_name": "registry.terraform.io/cloudflare/cloudflare",
                "schema_version": 0,
                "values": {
                  "account_id": ",<redacted>",
                  "id": "<redacted>",
                  "name": "my-app.co.uk",
                  "name_servers": [
                    "danica.ns.cloudflare.com",
                    "khalid.ns.cloudflare.com"
                  ],
                  "paused": false,
                  "plan": "Free Website",
                  "status": "active",
                  "vanity_name_servers": [],
                  "zone_id": "<redacted>"
                },
                "sensitive_values": {
                  "name_servers": [
                    false,
                    false
                  ],
                  "vanity_name_servers": []
                }
              },

If i then further dive into the gke cluster resource and see what else depends on it, it turns out there are a ton of other data sources that depend on that cluster, that are equally hard to explain:

"module.my-app-api.data.cloudflare_zone.my-app" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.data.google_artifact_registry_repository.my-app-docker" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.data.google_iam_policy.cloud_run_service" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.data.google_iam_policy.create_superuser_execute" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.data.google_iam_policy.django_migration_job_execute" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.data.google_iam_policy.sentry_version_job_execute" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.data.google_secret_manager_secret_version_access.common-data-token" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.data.google_secret_manager_secret_version_access.eccolab-secret" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.data.google_secret_manager_secret_version_access.geocodify_api_key" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.google_secret_manager_secret.config" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.google_secret_manager_secret.kube-config" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.google_secret_manager_secret.sentry_auth_token" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.google_service_account.eventarc" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.google_service_account.service-account" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.random_password.db-password" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.random_password.django-secret" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api.sentry_project.my-app-api" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.data.cloudflare_zone.my-app" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.data.google_artifact_registry_repository.my-app-docker" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.data.google_iam_policy.cloud_run_service" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.data.google_iam_policy.create_superuser_execute" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.data.google_iam_policy.django_migration_job_execute" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.data.google_iam_policy.sentry_version_job_execute" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.data.google_secret_manager_secret_version_access.common-data-token" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.data.google_secret_manager_secret_version_access.eccolab-secret" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.data.google_secret_manager_secret_version_access.geocodify_api_key" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.google_secret_manager_secret.config" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.google_secret_manager_secret.kube-config" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.google_secret_manager_secret.sentry_auth_token" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.google_service_account.eventarc" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.google_service_account.service-account" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.random_password.db-password" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.random_password.django-secret" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-api-v2.sentry_project.my-app-api" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners.data.google_iam_policy.dead-letter-subscription" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners.data.google_iam_policy.dead-letter-topic" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners.google_pubsub_topic.dead-letter" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners.google_pubsub_topic.home-my-app-response" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners.google_pubsub_topic.retrofit-planner-response" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners.google_service_account.gcp_service_account" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners.kubernetes_deployment.home-my-app-debug" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners.kubernetes_deployment.retrofit-planner-debug" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners.kubernetes_role.job_runner" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners-v2.data.google_iam_policy.dead-letter-subscription" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners-v2.data.google_iam_policy.dead-letter-topic" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners-v2.google_pubsub_topic.dead-letter" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners-v2.google_pubsub_topic.home-my-app-response" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners-v2.google_pubsub_topic.retrofit-planner-response" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners-v2.google_service_account.gcp_service_account" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners-v2.kubernetes_deployment.home-my-app-debug" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners-v2.kubernetes_deployment.retrofit-planner-debug" -> "module.k8s_cluster.google_container_cluster.cluster";
"module.my-app-runners-v2.kubernetes_role.job_runner" -> "module.k8s_cluster.google_container_cluster.cluster";

The gke cluster resource for reference:

resource "google_container_cluster" "cluster" { 
  name                        = var.cluster_name
  location                    = var.region
  network                     = var.vpc
  networking_mode             = "VPC_NATIVE"
  subnetwork                  = var.vpc_subnet
  datapath_provider           = "ADVANCED_DATAPATH"
  enable_intranode_visibility = true

  node_locations = [
    "europe-north1-a",
    "europe-north1-b",
    "europe-north1-c",
  ]
  maintenance_policy {
    recurring_window {
      # critical security updates are still applied outside this window
      start_time = "2023-07-03T00:00:00Z"
      end_time   = "2023-07-03T06:00:00Z"
      recurrence = "FREQ=DAILY"
    }
  }

  addons_config {

    dns_cache_config {
      enabled = false
    }
    horizontal_pod_autoscaling {
      disabled = false
    }

    http_load_balancing {
      disabled = false
    }
    network_policy_config {
      disabled = true
    }

  }
  authenticator_groups_config {
    security_group = "gke-security-groups@my-app.co.uk"
  }

  binary_authorization {
    evaluation_mode = "DISABLED"
  }

  timeouts {}

  cluster_autoscaling {
    enabled = false
  }
  project = var.project_id
  resource_labels = {
    "env" = "production"
  }

  database_encryption {
    state = "DECRYPTED"
  }
  default_snat_status {
    disabled = false
  }

  dns_config {
    cluster_dns        = "CLOUD_DNS"
    cluster_dns_domain = "cluster.local"
    cluster_dns_scope  = "CLUSTER_SCOPE"
  }


  gateway_api_config {
    channel = "CHANNEL_STANDARD"
  }


  ip_allocation_policy {
    stack_type = "IPV4"

    pod_cidr_overprovision_config {
      disabled = false
    }
  }
  deletion_protection = false
  private_cluster_config {
    enable_private_nodes    = true
    enable_private_endpoint = false

    master_ipv4_cidr_block = "<redacted>"

    master_global_access_config {
      enabled = false
    }
  }
  initial_node_count       = 1
  remove_default_node_pool = true
  logging_config {
    enable_components = [
      "SYSTEM_COMPONENTS",
      "WORKLOADS",
      "SCHEDULER",
    ]
  }

  master_auth {
    client_certificate_config {
      issue_client_certificate = false
    }
  }

  monitoring_config {
    enable_components = [
      "SYSTEM_COMPONENTS",
      "SCHEDULER",
      "POD",
      "DEPLOYMENT",
      "HPA",
      "STATEFULSET",
      "DAEMONSET",
    ]

    advanced_datapath_observability_config {
      enable_metrics = true
    }

    managed_prometheus {
      enabled = true
    }
  }

  network_policy {
    enabled  = false
    provider = "PROVIDER_UNSPECIFIED"
  }

  notification_config {
    pubsub {
      enabled = true
      topic   = "projects/${var.project_id}/topics/kubernetes-alerts"
    }
  }

  release_channel {
    channel = "REGULAR"
  }

  security_posture_config {
    mode               = "BASIC"
    vulnerability_mode = "VULNERABILITY_BASIC"
  }

  service_external_ips_config {
    enabled = false
  }
  workload_identity_config {
    workload_pool = "${var.project_id}.svc.id.goog"
  }
}
2

Hi @dolfandringa,

In order to see what references are in the configuration to determine how the dependencies arise, we would need to see the actual configuration. We can see that data.cloudflare_zone.my-app references local.tld, but not what expressions local.tld is comprised of. Also since these are contained in separate modules, the containing module and associated module calls would also be required to know exactly how these resources are related.

3

By default terraform graph focuses only on the broad dependencies between resources so that it’s at a similar level of detail to description of planned changes returned by terraform plan, but that does mean that it hides some details that would be needed to fully answer this question.

If you use terraform graph -type=plan instead then Terraform will return a description of the graph it would actually use to perform the planning phase, which would include extra nodes for details like module expansions, local values, variables passing values between modules, etc.

The full graph can be hard to interpret because it includes a number of implementation details. If you find something in there which you’re not sure about then I’d be happy to help to interpret it if you share the resulting graph description.

A typical cause of a large number of dependencies between resources in different modules is a module block that includes a depends_on argument referring to another module. Such a declaration effectively means “everything in this module depends on everything in that other module”, which often causes a very conservative dependency graph.

I can’t say whether that’s the case here because your question doesn’t include the module blocks, but that would be the first thing I’d check while trying to explain what you shared.