Issue with escaping variables

I am using vRA provider in my TF code and passing several variables via bash command line, these variables will then be passed to Ansible via vRA during post build life cycle of VM build (but issue is not with Ansible/vRA, issue is just how TF is sourcing variable to vRA).

Server OS version where my TF is running:

[root@san1 vra]# cat /etc/redhat-release

Red Hat Enterprise Linux Server release 7.7 (Maipo)

Scenario 1 (working as expected) :

If I input the variable during prompt by TF, it is working as expected and passing variable as I want exactly for my ansible lvs=[“lv_opt_san”]

[root@san1 vra]# /usr/local/bin/terraform apply

var.lvs

lv name

Enter a value: ["“lv_opt_san”"]

Variable is passed by TF to vRA as expected with “ ”

lvs=[“lv_opt_san”]

Scenario 2 (not working as expected) :

But if I try to pass the variable from command line (which the format that actually works on my shell), it is not being passed to vRA as expected by TF.

[root@san1 vra]# lvs=["“lv_opt_san”"]

[root@san1 vra]# echo $lvs

[“lv_opt_san”]

[root@san1 vra]# /usr/local/bin/terraform apply -auto-approve -var lvs=["“lv_opt_san”"]

Variable is not passed by TF to vRA as expected with “ ”

lvs=[lv_opt_san]

Tried following combinations (for scenario 2 above mentioned) now but none working,

lvs=[’“lv_opt_san”’] --> [lv_opt_san]
lvs=["\“lv_opt_san\”"] --> [“lv_opt_san”]
lvs=["\\“lv_opt_san\\”"] --> [\lv_opt_san\]
lvs=["“lv_opt_san”"] --> [lv_opt_san]
lvs=[’\“lv_opt_san\”’] --> [\“lv_opt_san\”]
lvs=[\“lv_opt_san\”] --> [\lv_opt_san]
lvs=["\“lv_opt_san\”"] --> [/opt/san]
lvs=["\“lv_opt_san\”"] --> [\lv_opt_san]
lvs="[\“lv_opt_san\”]" --> [“lv_opt_san”]
“[\“lv_opt_san\”]” --> [\lv_opt_san]

Hi @sandeep444033,

Unfortunately this seems like it might be a shell escaping problem rather than directly a Terraform problem, but that doesn’t mean we can’t try to figure out the answer together.

Before we talk about shell escaping, I want to note that you can potentially avoid shell escaping issues altogether by using the -var-file option instead, and putting your variable value in a file that uses Terraform language syntax. For example, if your lvs variable is a list variable and you want to assign what you showed as a list, you could put the following content in lvs.tfvars:

lvs = ["lv_opt_san"]

If instead lvs is a string variable and you want to assign a string containing JSON array syntax, you could alternatively write this:

lvs = "[\"lv_opt_san\"]"

You could then pass this to Terraform by adding -var-file=lvs.tfvars.


However, for the rest of this comment I’m going to assume that creating a .tfvars file isn’t acceptable for some reason, and so talk about both how Terraform interprets -var values and how we can use shell quoting to ensure that Terraform sees the string you intended to send it.

The first interesting thing to know about the -var command line option is that Terraform’s interpretation of it depends on what type argument you specified for your input variable in the variable "lvs" block. If you specify type = string then Terraform will take the sequence of characters you provided and use it directly as a string, without any further parsing:

variable "lvs" {
  type = string
}

However, if you specify a complex type constraint, such as type = list(string), then Terraform will try to parse the -var value you provided as Terraform language syntax and then interpret the result as a list of strings:

variable "lvs" {
  type = list(string)
}

I mention that mainly just as context, because it means that the same value -var=lvs=... can be interpreted in two different ways depending on the type constraint. That doesn’t change how we need to escape the value to avoid misinterpretation by the shell, but it can affect how Terraform treats the result once we do find a suitable way to escape the values in the shell.

Moving on to the shell escaping, then: when I’m trying to figure out a shell escaping problem I like to practice with the echo command first because that allows me to quickly see how the argument I provided was parsed by the shell, and then trust that the shell will behave the same way passing the same sequence of characters to Terraform. With that in mind, I tried out each of your test cases in turn to make sure I would see the same results you did. (Unfortunately because you didn’t mark your examples as “code” in the forum the forum formatting has changed them a little, and so for some of them I had to guess what you’d intended)

$ echo lvs=['"lv_opt_san"']
lvs=["lv_opt_san"]
$ echo lvs=["\"lv_opt_san\""]
lvs=["lv_opt_san"]
$ echo lvs=["\\"lv_opt_san\\""]
lvs=[\lv_opt_san\]
$ echo lvs=[""lv_opt_san""]
lvs=[lv_opt_san]
$ echo lvs=['\"lv_opt_san\"']
lvs=[\"lv_opt_san\"]
$ echo lvs=["\"lv_opt_san\""]
lvs=["lv_opt_san"]
$ echo lvs=["\"lv_opt_san\""]
lvs=["lv_opt_san"]
$ echo lvs="[\"lv_opt_san\"]"
lvs=["lv_opt_san"]
$ echo lvs="[\"lv_opt_san\"]"
lvs=["lv_opt_san"]

Some of my results agree with your results, but some of them do not. Indeed, six of my attempts seemed to produce the result you were expecting, although I also noticed that some of your inputs seemed to be duplicates – perhaps a result of the forum formatting? – and so I’m not sure I fully replicated all of your cases.

Reducing the above down to just the ones that produced your desired result, we can see these:

$ echo lvs=['"lv_opt_san"']
lvs=["lv_opt_san"]
$ echo lvs=["\"lv_opt_san\""]
lvs=["lv_opt_san"]
$ echo lvs=["\"lv_opt_san\""]
lvs=["lv_opt_san"]
$ echo lvs=["\"lv_opt_san\""]
lvs=["lv_opt_san"]
$ echo lvs="[\"lv_opt_san\"]"
lvs=["lv_opt_san"]
$ echo lvs="[\"lv_opt_san\"]"
lvs=["lv_opt_san"]

I would add to this one more option, which is the one I usually use when writing -var arguments for Terraform on a Unix shell:

$ echo 'lvs=["lv_opt_san"]'
lvs=["lv_opt_san"]

All of these working examples are relying on the shell behaviors for either single quotes or double quotes. I usually prefer to use single quotes in situations like this because it means that no additional escaping is needed at all unless the final value must include a literal ' character. The Terraform language does not use ' as a significant character, so it would only arise if a literal string value needed to include it.

Could you try these examples with echo as I have run above and see if you get the same results? If not, could you share the results you saw so I can see how yours are different? When you share command output or source code, please mark it as code in the forum by using the <> button in the editor toolbar, because that then avoids problems where the editor might change the characters you entered for cosmetic reasons.

Thanks for the response.

It worked as expected on my shell

   [root@san vra]# echo 'lvs=["lv_opt_san"]'

   lvs=["lv_opt_san"]

I have my lvs variable defined as following in my .TF file

     variable "lvs" {
          description = "lv name"
        }

I have applied my TF from CLI as follows,

[root@san vra]# /usr/local/bin/terraform apply -auto-approve  -var 'lvs=["lv_opt_san"]'

But it sourced my variables as [lv_opt_san] to my vRA deployment.

Hi @sandeep444033,

If the shell escaping is working as expected for echo then it should be working for Terraform too. That makes me suspect that the problem is not to do with the command line argument processing at all, but is happening somewhere else in your configuration instead.

How does your configuration make use of this var.lvs variable? I’m not familiar with vRA at all, so I don’t really know what happens to this setting after Terraform receives it, but I suspect that this string is passing through another layer of shell parsing somewhere else which is then causing the quotes to be lost.

Thanks for the response @apparentlymart.

Please find my below findings,

As advised I tried to pass variables form a varfile but did not make any difference.

[root@san vra]# cat vars.tfvars
lvs = ["lv_opt_san"]
 
/usr/local/bin/terraform apply -auto-approve -var-file=vars.tfvars

While I interestingly found out that this might be the way vRA provider works, I feel that all resource configuration declarations are by default expecting string type parameters and I guess we cannot over ride these.

My variable declaration is as follows in my deployment.TF file,

[root@san vra]# cat vra-deployment.tf | grep -A3 lvs
variable "lvs" {
  description = "lv name"
  type    = list(string)
  default = ["lv_opt_san"]
 
 ## VM PROPERTIES ##
    configuration = {
                       
                        "Activated_Ansible_Template_Extra_Var_lvs" = var.lvs
     
    }

When I try to do a terraform run I see following error,

[root@san vra]# /usr/local/bin/terraform apply -auto-approve
 
Error: Incorrect attribute value type
 
  on vrdeployment.tf line 104, in resource "vra7_deployment" "myDeployment":
104:     configuration = {
 
Inappropriate value for attribute "configuration": element
"Activated_Ansible_Template_Extra_Var_lvs" = var.lvs: string required.

I see it is not honoring my lvs variable declaration as list of strings.

If I try to change variable type as string in my deployment file,

[root@san vra]# cat vra-deployment.tf| grep -A3 lvs
 variable "lvs" {
  description = "lv name"
  type    = string
  default = ["lv_opt_san"]
 
     ## VM PROPERTIES ##
        configuration = {
                       
                        "Activated_Ansible_Template_Extra_Var_lvs" = var.lvs
 
}

My deployment fails with following error (as expected, as I am passing an array),

[    root@san vra]# /usr/local/bin/terraform apply -auto-approve
     
    Error: Invalid default value for variable
     
      on vra-deployment.tf line 75, in variable "lvs":
      75:   default = ["lv_opt_san"]
 
This default value is not compatible with the variable's type constraint:
string required.

Hi Sandeep,
I am trying to use provider as VRA.
I am receiving the error when I run command “terraform init”
Can you share me the procedure that you have followed to setup VRA provider.

Error:
PS C:\Users\chandra\Desktop\Terraform\Provider-VRA> terraform init

Initializing the backend…

Initializing provider plugins…

  • Finding latest version of hashicorp/vra…

Error: Failed to query available provider packages

Could not retrieve the list of available versions for provider hashicorp/vra:
provider registry registry.terraform.io does not have a provider named
registry.terraform.io/hashicorp/vra

If you have just upgraded directly from Terraform v0.12 to Terraform v0.14
then please upgrade to Terraform v0.13 first and follow the upgrade guide for
that release, which might help you address this problem.

Did you intend to use vmware/vra? If so, you must specify that source address
in each module which requires that provider. To see which modules are
currently depending on hashicorp/vra, run the following command:
terraform providers

@dollachandu

What version of Terraform are you using?
Can you paste your provider.tf here?