I have simple workflow. Create a cluster, install ingress with helm and once it’s ready, get load balancer’s IP address to update CF DNS records. It’s all in one go.
After creating nginx-ingress controller, I was trying to get IP address of load balancer.
resource "helm_release" "nginx_ingress" {
name = "ingress-nginx"
...
}
data "kubernetes_service" "this" {
metadata {
name = "ingress-nginx-controller"
namespace = "ingress-nginx"
}
depends_on = [ helm_release.nginx_ingress ]
}
This never worked for me and the reason was that it was trying to connect to localhost.
Then I found this limitation.
WARNING When using interpolation to pass credentials to the Kubernetes
provider from other resources, these resources SHOULD NOT be created
in the same Terraform module where Kubernetes provider resources are
also used. This will lead to intermittent and unpredictable errors
which are hard to debug and diagnose. The root issue lies with the
order in which Terraform itself evaluates the provider blocks vs.
actual resources. Please refer to this section of Terraform docs for
further explanation.
and moved kubernetes provider to another module.
However, I have now a new problem.
module "cluster" {
source = "./modules/cluster"
cluster_node_count = var.cluster_node_count
age_path = var.age_path
}
module "network" {
source = "./modules/network"
host = module.cluster.host
token = module.cluster.token
cluster_ca_certificate = module.cluster.cert
cloudflare_email = var.cloudflare_email
cloudflare_api_key = var.cloudflare_api_key
main_domain = var.main_domain
cluster_subdomains = var.cluster_subdomains
depends_on = [module.cluster]
}
This also doesn’t work and giving
Providers cannot be configured within modules using count, for_each or depends_on.
Without depends_on the network module is called at the same time as cluster, which of course leads to fail, since for the network module cluster should be ready.
So it’s kind of dead-loop which I am not sure how to break.