Use dynamic datasource in local variables and output's

Terraform
1

I have two use-cases

  1. I am trying to create a local variable from Datasource output. During terraform plan the local variable is failing with null data.
data "google_compute_subnetwork" "gke_subnet" {
  for_each = { for x in var.cloudamqp_vpc_peerings : "${x.subnet_name}" => x }
  region = lookup(each.value,"subnet_region",var.cloudamqp_provider_region)
  name   = each.value.subnet_name
}

locals {
  subnet_values = data.google_compute_subnetwork.gke_subnet
  primary_range_firewall_rules = distinct(flatten([
    for x,y in local.subnet_values
    :  { name = x, cidr_range = y.ip_cidr_range, protocols = ["AMQP", "AMQPS"], ports = [15672] }   
  ]))
  secondary_range_firewall_rules = distinct(flatten([
    for x in var.cloudamqp_vpc_peerings 
    : [
      for y in local.subnet_values[x.subnet_name].secondary_ip_range
      : { name = y.range_name, cidr_range = y.ip_cidr_range, protocols = ["AMQP", "AMQPS"], ports = [15672] }
    ]    
  ]))
}

local.subnet_values is object with 2 attributes
    | x.subnet_name is "scanfarm-infra-subnet1"

A null value cannot be used as the collection in a 'for' expression.
  1. I have an output dependent on a datasource. After the first tf apply the output will be null. Once the data source is computed and added to tf statefile, Output will be added on the second run.
output "username" {
  value = data.cloudamqp_credentials.credentials.username
}

output "password" {
  value     = data.cloudamqp_credentials.credentials.password
  sensitive = true
}
2

Hi @gdineshkumar627,

That appears to be a problem with the cloudamqp_credentials data source. It sounds like the provider is not consistently returning values for the fields you are accessing, and I would file and issue with the provider.

3

@jbardin Thanks for the quick response.
I do have an issue with google Datasource.

data "google_compute_subnetwork" "gke_subnet" {
  for_each = { for x in var.cloudamqp_vpc_peerings : "${x.subnet_name}" => x }
  region = lookup(each.value,"subnet_region",var.cloudamqp_provider_region)
  name   = each.value.subnet_name
}
5

OK, you seem to have 2 unrelated problems here. I’m not sure what is incorrect with the google_compute_subnetwork configuration from the little shown here. It would help to have a more complete reproduction, showing the variables and types of values expected, and the complete context of the error to show exactly which expression is failing.

6

I am trying to create a loca variable (type: list of maps) from the google_compute_subnetwork data source output.

datasource block

data "google_compute_subnetwork" "gke_subnet" {
  for_each = { for x in var.cloudamqp_vpc_peerings : "${x.subnet_name}" => x }
  region = lookup(each.value,"subnet_region",var.cloudamqp_provider_region)
  name   = each.value.subnet_name
}

Local variables block

locals {
  subnet_values = data.google_compute_subnetwork.gke_subnet
  primary_range_firewall_rules = distinct(flatten([
    for x,y in local.subnet_values
    :  { name = x, cidr_range = y.ip_cidr_range, protocols = ["AMQP", "AMQPS"], ports = [15672] }   
  ]))

  secondary_range_firewall_rules = distinct(flatten([
    for x in var.cloudamqp_vpc_peerings 
    : [
      for y in local.subnet_values[x.subnet_name].secondary_ip_range
      : { name = y.range_name, cidr_range = y.ip_cidr_range, protocols = ["AMQP", "AMQPS"], ports = [15672] }
    ]    
  ]))
}

Issue:
Datasource is returning null data, the local variables are trying to iterate over that and failing
Failed expression : for y in local.subnet_values[x.subnet_name].secondary_ip_range

local.subnet_values is object with 2 attributes
    | x.subnet_name is "scanfarm-infra-subnet1"

A null value cannot be used as the collection in a 'for' expression.
7

The error indicates that local.subnet_values[x.subnet_name].secondary_ip_range is null, which is not valid in a for expression. If this is something which is expected to never be null, that may be an issue with the provider. If null is a possible value in this case, you will need to make the it conditional at some point. Either using something like

local.subnet_values[x.subnet_name].secondary_ip_range == null ? [] : ...

Or wrapping it in try, and returning an empty collection if it fails.

try([ for y in local.subnet_values[x.subnet_name].secondary_ip_range
   : { name = y.range_name, cidr_range = y.ip_cidr_range, protocols = ["AMQP", "AMQPS"], ports = [15672] }
], [])