Hello!
I’m trying to create an Internal regional Application Load Balancer with GKE NEG as a backend using Terraform. I had no issues during the first run, but when I update my application (Jira), which causing removing and adding back endpoints to NEG, there’s a warning message. Meanwhile, all resources were created and works as expected.
Please find my Terraform code below:
data "google_compute_network_endpoint_group" "ilb_network_endpoint_group_zonal" {
count = var.environment == "qa" || var.environment == "test" || var.environment == "dev" ? 1 : 0
name = "name-${var.environment}-jira-neg"
project = local.project_id
zone = "europe-west3-a"
depends_on = [
helm_release.jira
]
}
resource "google_compute_region_health_check" "ilb_health_check_zonal" {
count = var.environment == "qa" || var.environment == "test" || var.environment == "dev" ? 1 : 0
name = "name-${var.environment}-ilb-health-check"
project = local.project_id
region = local.region
timeout_sec = 5
check_interval_sec = 5
healthy_threshold = 2
unhealthy_threshold = 2
http_health_check {
port = "8080"
request_path = "/status"
port_specification = "USE_FIXED_PORT"
}
}
resource "google_compute_region_backend_service" "ilb_backend_service_zonal" {
count = var.environment == "dev" || var.environment == "test" || var.environment == "qa" ? 1 : 0
name = "name-${var.environment}-ilb-backend-service"
project = local.project_id
region = local.region
health_checks = [google_compute_region_health_check.ilb_health_check_zonal[0].id]
protocol = "HTTP"
load_balancing_scheme = "INTERNAL_MANAGED"
enable_cdn = false
session_affinity = "GENERATED_COOKIE"
locality_lb_policy = "RING_HASH"
timeout_sec = 300
backend {
group = data.google_compute_network_endpoint_group.ilb_network_endpoint_group_zonal[0].id
balancing_mode = "RATE"
max_rate_per_endpoint = 1000
capacity_scaler = 1.0
}
consistent_hash {
minimum_ring_size = 1024
}
}
resource "google_compute_region_url_map" "ilb_url_map_zonal" {
count = var.environment == "dev" || var.environment == "test" || var.environment == "qa" ? 1 : 0
name = "name-${var.environment}-ilb-url-map"
project = local.project_id
region = local.region
default_service = google_compute_region_backend_service.ilb_backend_service_zonal[0].id
}
resource "google_compute_region_target_http_proxy" "ilb_target_http_proxy_zonal" {
count = var.environment == "dev" || var.environment == "test" || var.environment == "qa" ? 1 : 0
name = "name-${var.environment}-ilb-https-proxy"
project = local.project_id
region = local.region
url_map = google_compute_region_url_map.ilb_url_map_zonal[0].id
}
resource "google_compute_forwarding_rule" "ilb_global_forwarding_rule_zonal" {
count = var.environment == "dev" || var.environment == "test" || var.environment == "qa" ? 1 : 0
name = "name-${var.environment}-ilb-global-forwarding-rule"
project = local.project_id
region = local.region
network = data.google_compute_network.network.self_link
subnetwork = data.google_compute_subnetwork.subnet.self_link
ip_protocol = "TCP"
load_balancing_scheme = "INTERNAL_MANAGED"
port_range = "80"
target = google_compute_region_target_http_proxy.ilb_target_http_proxy_zonal[0].self_link
ip_address = data.google_compute_address.nginx_ingress_ip.address
}
NEG changes during application update:
jira 83s Normal Detach service/jira Detach 1 network endpoint(s) (NEG "name-test-jira-neg" in zone "europe-west3-a")
jira 69s Normal Attach service/jira Attach 1 network endpoint(s) (NEG "name-test-jira-neg" in zone "europe-west3-a")
Warning message:
Error: Provider produced inconsistent final plan
When expanding the plan for
google_compute_region_backend_service.ilb_backend_service_zonal[0] to
include new values learned so far during apply, provider
"registry.terraform.io/hashicorp/google" produced an invalid new value for
.backend: planned set element
cty.ObjectVal(map[string]cty.Value{"balancing_mode":cty.StringVal("RATE"),
"capacity_scaler":cty.NumberIntVal(1), "description":cty.StringVal(""),
"failover":cty.UnknownVal(cty.Bool), "group":cty.UnknownVal(cty.String),
"max_connections":cty.NullVal(cty.Number),
"max_connections_per_endpoint":cty.NullVal(cty.Number),
"max_connections_per_instance":cty.NullVal(cty.Number),
"max_rate":cty.NullVal(cty.Number),
"max_rate_per_endpoint":cty.NumberIntVal(1000),
"max_rate_per_instance":cty.NullVal(cty.Number),
"max_utilization":cty.NullVal(cty.Number)}) does not correlate with any
element in actual.
This is a bug in the provider, which should be reported in the provider's
own issue tracker.
Steps to Reproduce:
- Create GKE service with NEG using annotations:
annotations:
cloud.google.com/neg: '{"exposed_ports": {"80":{"name": "name-${environment}-jira-neg"}}}'
- Use Terraform code above to deploy Internal regional Application Load Balancer while using NEG as a backend.
- Update app parameters to trigger adding and removing endpoints to NEG and run
terraform apply
Second execution of terraform apply shows no issues.
One more thing, I have no issues with the similar solution, but with External HTTPS Load Balancer instead.
What I tried so far:
- I tried to add a delay before
data.google_compute_network_endpoint_group.ilb_network_endpoint_group_zonalto address some possible internal processes in GCP - no luck with 360s delay; - Reported issue to GCP Support - they found no issues with created resources;
- Reported issue on GitHub - no solution so far.
I wonder if anyone has the same issue? If yes, pls let me know if there’s a way to solve it and how did you do that. If you found no solution, please thumbs up of my issue report.
Thanks in advance!