How to escape double-quotes in the local-exec provisioner

Terraform
1

Hi, everybody,

I’m trying to execute a command with the Terraform “local-exec” provisioner but this command contains double quotes (") and I can’t escape them to run the command correctly.

I have reduced my problem to a simpler one, here is my code :

resource "null_resource" "Hello_World" {
  provisioner "local-exec" {
    command = "echo \"Hello World\""
  }
}

I would like the result of this command to be :

"Hello World"

But the result is :

\"Hello World\"

Execution :

null_resource.Hello_World: Destroying... [id=2381325006212666147]
null_resource.Hellgo_World: Creating...
null_resource.Hello_World: Destruction complete after 0s
null_resource.Hellgo_World: Provisioning with 'local-exec'...
null_resource.Hellgo_World (local-exec): Executing: ["cmd" "/C" "echo \"Hello World\""]
null_resource.Hellgo_World (local-exec): \"Hello World\"
null_resource.Hellgo_World: Creation complete after 0s [id=5337014350841532259]

Could you please tell me how to properly escape double-quote and not end up with \ in my final result.

Thank you in advance.
Hugo

2

Hi @HugoSecteur4!

The result you’re seeing here is confusing to me. The first log line reports this:

Executing: ["cmd" "/C" "echo \"Hello World\""]

The backslashes we see here are a result of Terraform presenting the argument in quotes itself and thus escaping the ones in the string for display only. So the literal value of that argument as seen by the command interpreter should be echo "Hello World", as you wanted.

However, internally then Terraform must call the CreateProcess Windows API function to launch the command, because you seem to be using Windows. The interface of that function requires adding the quotes back, because on Windows command arguments are passed as a string, so the effective call to the Windows API would then be as follows:

  • lpApplicationName as cmd
  • lpCommandLine as cmd /C "echo \"Hello World\""

The escaping backslashes must be re-introduced here because the second argument to cmd contains spaces and must therefore be quoted itself.

What normally happens next on Windows is that the command interpreter (cmd.exe) parses the third argument and itself calls CreateProcess to run the given program, and then dealing with the quotes in the remaining arguments is up to that final program. However, echo is a special command built in to the command interpreter, so in this case the command interpreter would interpret the command line directly itself.

My suspicion is that the strange result you see here is due to the fact that echo is a command interpreter builtin and so it doesn’t process its arguments in the standard way: it just takes verbatim what it is given as a single string, rather than tokenizing it into several separate arguments as would normally happen when running an external program.

With all of that said, I think this odd behavior is likely unique to the echo command in particular, and that if you were running a standard Windows command line application you’d see the behavior you expected.

However, I don’t have a Windows system handy to test, so it’s equally likely that I’m misreading the situation. I’d suggest trying this out with whatever actual program you need to run to see if you get a different result. Unfortunately on Windows not all commands/applications comply with the standard convention for command line processing, and in that case those programs may not be compatible with Terraform’s local-exec provisioner unless you write a wrapper program (which could e.g. be just a .cmd script, or anything else) that can then pass the arguments in the unique way that program is expecting.

1 Like
3

Hi @apparentlymart and thank you for your complete answer !
You were right, the problem come from the windows command interpreter: cmd.

I finally understood that the escape character of the cmd.exe of windows is not \ but is ^.

So to get the result I wanted, I would have to send the command: echo ^"Hello World^" but this is apparently not possible with Terraform :

Error: Missing newline after argument

  on main.tf line 3, in resource "null_resource" "Hello_World":
   3:     command = "echo ^"hello world^""

An argument definition must end with a newline.


Error: Unsupported operator

  on main.tf line 3, in resource "null_resource" "Hello_World":
   3:     command = "echo ^"hello world^""

Bitwise operators are not supported.

To solve the problem,I simply changed the interpreter to which the command is sent in the local-exec to PowerShell:

resource "null_resource" "Hello World" {
  provisioner "local-exec" {
    command = "echo \"hello world\""
	interpreter = ["PowerShell", "-Command"]
  }
}

The result is then correct. On the other hand, I still haven’t managed to get the same result with cmd.exe

Thank you !
Hugo

2 Likes
4

Thanks for the follow-up, @HugoSecteur4!

Indeed, I think unfortunately it will not be possible to generate the expected sequence of characters in this case.

Terraform on Windows can only run programs that parse their command line arguments using either CommandLineToArgvW or an equivalent function like the one embedded in the Microsoft Visual C++ runtime library. Terraform has to decide on some mechanism of escaping the quotes in order to pass them correctly to the cmd /C argument, and it chooses the format that is most commonly supported in Windows command line tools which tend to be written (either directly or indirectly) in C or C++, for the broadest possible compatibility.

I’m glad that you were able to make it work with PowerShell. PowerShell’s command line processing has some challenges of its own when running external software with it, but as long as you are running built-in PowerShell cmdlets the standard quote escaping used by Terraform should be compatible with the PowerShell parser.

If you’re interested (this is definitely not required reading to use Terraform!), there are lots more details on the difficulty of consistently handling command line arguments in David Daley’s article How Command Line Parameters Are Parsed. That article primarily discusses typing arguments out at the command prompt; the problem is exacerbated when another piece of software like Terraform is intermediating, because Terraform itself has its own escaping rules and then the command must pass though CreateProcess to get to the command interpreter before the steps described in that article can begin. :confounded:

1 Like
5

Using the Powershell interpreter is a good workaround for local development, but is there any way we could achieve the same effect using a cross platform interpreter since I am working together with MacOS users and am facing the same issue.

6

Hi @brechtpallemans,

As far as I know there is no shell/interpreter that is common across a fresh install of both MacOS and Windows, but there are builds of Unix shells like bash available for Windows, and I believe there is also a PowerShell release for Mac OS.

One of the disadvantages of running external software using local-exec is that it’s subject to the unique quirks of whatever platform Terraform is currently running on. It can be difficult to ensure that such a configuration is convenient for everyone to run.

In a cross-platform environment, I’d recommend not using local-exec provisioners or any other mechanism that runs external software on the local system via a shell, and instead make your configuration use only resource types from portable Terraform providers. If you need to do something that can’t be handled by either a feature in an existing Terraform provider or via built-in Terraform language features, you could choose to develop a provider yourself and compile it both for Windows and MacOS, so both of your user cohorts can benefit from it.

Another option is to standardize on running Terraform on remote virtual machines rather than on your local development machines. You can then choose a standard execution environment that everyone will use, and write your Terraform configurations with that in mind. The remote operations feature of Terraform Cloud is a relatively easy way to get started with this, because once you’ve set it up all of your Terraform operations will happen on a Linux system run by HashiCorp.

7

Thank you for your tips!
I suggest using PowerShell Core interpreter:
interpreter = ["pwsh", "-Command"]
This way it will work on both Windows and MacOS - tested :slight_smile: