Can't import resources with map values

MohnJadden · July 15, 2022, 1:19pm

I’m using map(any) variables in an Azure Terraform environment. I need to import some existing Azure resource groups, and they need to correlate to the mappings. For example:

and so forth.

If I just run terraform import --var-file=varfile.tfvars azurerm_resource_group.rg-list /subscriptions/guid/resourceGroups/Marketing , it imports the resource, but running terraform apply with the mappings doesn’t reflect that the RG already exists and wants to destroy it.

If I run it with escaped quotes, e.g. terraform import -var-file varfile.tfvars 'azurerm_resource_group.rg-list[\"Team2\"]' /subscriptions/guid/resourceGroups/Marketing, it prepares the import but then errors out multiple times that the given key does not identify an element in the collection value.

How do I go about importing these resource groups corresponding to their map keys so that they remain under management?

jbardin · July 15, 2022, 1:41pm

Hi @MohnJadden,

I think the problem is that in your second example you’ve double-escaped the quotes. The single quotes already prevent the shell from interpolating the double quotes in the string, but the unnecessary backslashes are now also going to be used literally. Only use one method for escaping the string:

% terraform import -var-file varfile.tfvars 'azurerm_resource_group.rg-list["Team2"]'

If that doesn’t work, you can use the output from TF_LOG=trace to see exactly how the arguments were passed to terraform from the shell.

MohnJadden · July 15, 2022, 3:14pm

When I run terraform import -var-file varfile.tfvars 'azurerm_resource_group.rg-list["Team2"]' , I get an error: Index brackets must contain either a literal number or a literal string.

I’m not sure how to use TF_LOG=trace. The closest thing I could find was to just run the command $tf_log=“trace”, which I ran. Nothing happened, nor did I get any log files or further details when I re-ran the command as you listed it.

I’m doing this in Windows, if that makes a difference.

jbardin · July 15, 2022, 3:31pm

TF_LOG is an environment variable used by terraform to determine the level of log output. Setting that variable to trace will show the greatest details of debug logs.

Windows definitely makes a difference, as will the shell you are using on windows. Wrapping the string in single quotes is the standard for most shells, but windows does have some unique requirements for parsing commands line arguments. I don’t use windows myself, but I think in this case you can escape the quotes as """:

terraform import -var-file varfile.tfvars "azurerm_resource_group.rg-list["""Team2"""]"

Someone more familiar with windows might be able to confirm, otherwise the trace log output would definitely help figure out what the shell is actually passing to the terraform process.

MohnJadden · July 15, 2022, 4:01pm

Tried with the “”" escaping pattern - same error regarding literals.

How do I actually set this environment variable? Again my google-fu fails me.
(Using Visual Studio Code)

MohnJadden · July 18, 2022, 12:22pm

Checking in one final time on this - is it just that TF is not capable of importing against a map? I suppose I gotta take this one to Fiverr or something :-/

jbardin · July 18, 2022, 1:33pm

@MohnJadden, the problem here is not Terraform, but rather the input to the Windows shell. Terraform only can see what is passed to it from the shell process, and if the syntax is incorrect, then the errors are to be expected. If you are using cmd.exe, you can set environment variables using the set command.

This document seems to have significant details about the Windows parsing options (as you can see, there is a lot more to document about windows behavior), and also mentions that ^ may work as an escape character here.

MohnJadden · July 18, 2022, 3:58pm

I’m using Visual Studio Code.

I’m at the point where I might just throw up my hands and run tf apply, Devil be damned. Let me ask you this: if I run terraform plan and TF wants to create a resource group that already exists, will it delete the RG itself in Azure? Or will it just bring it under TF management?

I don’t know how it came out of TF management in the first place, these RGs have been in place for months with numerous instances of TF apply run against 'em. They hadn’t even come up on resources to change until now. I’ve been trying to import the RGs to be able to just continue but that’s turning into a massive exercise now that I have to figure out how to do whatever programming-language stuff this is.

jbardin · July 18, 2022, 4:21pm

I can’t answer the question definitively, but most likely the Azure provider will return an error during apply that the resource group cannot be created due to a naming conflict. What exactly happens depends on the provider implementation.

It’s also possible that manually replacing a resource like azurerm_resource_group could force the replacement of anything which depends on it, since that sounds like it something which conceptually “contains” other resources, which is something to be aware of.

MohnJadden · July 18, 2022, 4:54pm

Welp, I ran a plan, and TF errored out saying that the resource needs to be imported to be managed in the state. Fiverr it is.

Appreciate the attempt to help!

apparentlymart · July 18, 2022, 11:42pm

On Windows it matters a lot whether you’re using the Command Prompt (cmd.exe) or PowerShell, because each one has its own quoting/escaping rules.

PowerShell in particular has some quirks with running external software, since it’s primarily designed for running its own cmdlets. For the sake of making this work, I suggest using the normal Command Prompt, which has more straightforward handling of escaping.

If you are using Command Prompt then you can ignore the command interpreter’s own escaping here, because it doesn’t do anything special with quotes and only requires special escaping for metacharacters like |, >, %%, etc. (The ^ character is for escaping those command interpreter metacharacters only.)

However, you do still need to escape the quotes in a way that the command line parser in Terraform itself will understand it, which involves using backslashes to escape the quotes that will be taken literally, like this:

terraform import -var-file varfile.tfvars azurerm_resource_group.rg-list[\"Team2\"]

The single quote character ' has no special meaning in either the Windows command interpreter or in the command line escaping rules on Windows, so including those will cause parse errors. The single quotes are useful on platforms other than Windows because then it avoids the need for any special escaping inside, but Windows does not follow that convention.

MohnJadden · July 20, 2022, 5:19pm

I’ve been using Visual Studio Code all this time, working with a checked out branch from my company’s Github repo.

For giggles, I tried running your example in regular old Windows cmd. It runs for a moment, notes that it’s prepared for import, then throws a few errors:

│ Error: Invalid index
│
│   on C:\Repos\Infra_Azure_Terraform_Source-2\main.tf line 62, in module "pim_assignment":
│   62:   resource_group_name  = azurerm_resource_group.rg-list[each.key].name
│     ├────────────────
│     │ azurerm_resource_group.rg-list is object with 1 attribute "Team1"
│     │ each.key is "Team6"
│
│ The given key does not identify an element in this collection value.

It then repeats the error six times - once for each key in the map.

The pim_assignment module is referenced in my main.tf as follows:

module "pim_assignment" {

  source               = "./PIM Assignment - Resource Group"
  for_each             = var.team_name
  resource_group_name  = azurerm_resource_group.rg-list[each.key].name
  principal_id         = lookup(var.SecurityGroup, each.key)
  role_definition_name = "Virtual Machine Administrator Login"
}

The main.tf for the module references the resource group as

variable "resource_group_name" {
  type = string
}

and it’s used in resources as follows:

resource "random_uuid" "eligible_schedule_request_id" {
  keepers = {
    principalId         = var.principal_id
    roleDefinitionId    = data.azurerm_role_definition.role.id
    requestType         = var.request_type
    startDateTime       = "${formatdate("YYYY-MM-DD", time_rotating.eligible_schedule_request_start_date.id)}T${formatdate("HH:mm:ss.0000000+02:00", time_rotating.eligible_schedule_request_start_date.id)}"
    duration            = "P${tostring(var.assignment_days)}D"
    resource_group_name = azurerm_resource_group.rg-list[each.key].name
  }
}

If I comment out the module in my main.tf, I’m able to import the resource groups. However, that leaves all the other resources which I’d then have to import again, so it’s still at the question of how I’m meant to do the original task - importing resources against map values.

MohnJadden · July 22, 2022, 4:09pm

Found the issue. In all the responses people generously provided, everyone forgot to add the Azure resource ID. All this time I was trying to just import nothing

The actual solution:

terraform import -var-file="varfile.tfvars" 'azurerm_resource_group.vdi-rg[\"Team1\"]' /subscriptions/guid/resourceGroups/Marketing

As long as the map key you specify within the ["MapKeyItemGoesHere"] exists within the mapping in tfvars, it works.

Yes, you do need to iterate each resource with its Terraform resource name and Azure resource ID, and it does change depending on the Terraform and Azure resource provider, but it works. I bet the more coder-inclined could write something to just loop through, but I leave that to the devs.

I just concatenated based on TF/Azure provider names in Excel and called it done.

snpreece · November 21, 2023, 11:00am

THANK YOU! - was banging my head off the desk trying to figure out the same problem running an import from powershell

encapsulating the whole module name (in my case it’s a module) in single quotes and the index string in escaped speechmarks i.e. "my_index_string" worked a treat