Executing powershell script as part of external provider

Hello Terraform users,

I am trying to run a powershell script (using powershell core) within an external provider as follow:

data "external" "powershell_execution_cloud" {
  program = ["pwsh", "./customscripts/SBusConnectionStringCloud.ps1 ${terraform.workspace} ${azurerm_resource_group.fusion.name} ${var.service_bus_type_cloud}"]
}

The script SBusConnectionStringCloud.ps1 echo’s a JSON output and when executed on a terminal windows, executes without an error:

➜ pwsh ./customscripts/SBusConnectionStringCloud.ps1 eu-feature eu-feature-westeurope standard
{
  "connection_string": "Endpoint=XXXYYYYY"
}

For some reason, Terraform fails with the following error:

	* data.external.powershell_execution_cloud: 1 error occurred:
	* data.external.powershell_execution_cloud: data.external.powershell_execution_cloud: failed to execute "pwsh": The argument './customscripts/SBusConnectionStringCloud.ps1 eu-feature eu-feature-westeurope standard' is not recognized as the name of a script file. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.

We are running Terraform version 0.11 and its being executed on Linux. The reason why we want to use pwsh instead of Bash is because our build servers are running windows. External provider of version 1.2.0 is utilized.

Has anyone faced a similar problem before and was able to determine the root cause?

Hi @kishorebasisth,

I think this is the result of a difference in how command line arguments are sent between programs on Windows vs. programs on Unix systems.

On a Unix system, the convention is for arguments to be passed as an array of strings where each argument is a separate string. The program argument to the external data source is following that convention: each element of that list is passed to the child program as a single argument, and so what you wrote there is as if you’d written the following at a Bash prompt:

pwsh './customscripts/SBusConnectionStringCloud.ps1 eu-feature eu-feature-westeurope standard'

Notice that everything except the initial pwsh here is in single quotes, so it’s all a single argument as far as pwsh is concerned. It’s therefore trying to find a file literally called ./customscripts/SBusConnectionStringCloud.ps1 eu-feature eu-feature-westeurope standard, which doesn’t exist and therefore the command fails.

Windows has a different convention: all of the arguments to a program are presented to that program as a single string, and it’s the program’s own responsibility to decide how to parse that string into separate arguments. There is no single correct way to construct a multi-argument command line on Windows, but in practice many applications use either the CommandLineToArgvW function or the similar (but not quite identical) functionality built into the Visual C++ runtime library.

Because the external provider needs to make some decision about how to present the arguments, it assumes that the given program will interpret arguments by one of the two methods mentioned in the previous paragraph, and so it will concatenate all of the given elements of program (except the first one, identifying the program to run) into a single string, with quotes around any items that contain spaces.

How the target program interprets the resulting string is not generally defined, so in this case what matters is how PowerShell itself interprets its command line string. I don’t know how PowerShell does that, but it does seem like it does it in such a way that allows it to understand the four space-separated tokens in your string as separate arguments, and thus see that ./customscripts/SBusConnectionStringCloud.ps1 is the script you intend to run and that the other tokens are arguments to that script.

If I’m right in my assumptions above, I think the answer would be to write the program argument in the way the data source is expecting, with each argument represented as a separate element of the list:

data "external" "powershell_execution_cloud" {
  program = [
    "pwsh",
    "./customscripts/SBusConnectionStringCloud.ps1",
    terraform.workspace,
    azurerm_resource_group.fusion.name,
    var.service_bus_type_cloud,
  ]
}

This should then produce the same result as if you were to run the following command in Bash, on a Unix system:

pwsh ./customscripts/SBusConnectionStringCloud.ps1 eu-feature eu-feature-westeurope standard

Now the script filename is separated from the others, and so pwsh should see that you intend to use ./customscripts/SBusConnectionStringCloud.ps1 and find that script.

On Windows, the external provider will now construct a slightly-different shaped command line arguments string to pass to pwsh, with all of the elements of program concatenated together with spaces:

pwsh ./customscripts/SBusConnectionStringCloud.ps1 eu-feature eu-feature-westeurope standard

That should then work on Windows too, as long as the Windows build of PowerShell understands that there are four separate arguments there. I think it would be strange if it did not, but I’ve not tested it so I can’t be sure.

The generalization of this advice is: there should be one element of program for each distinct argument that you’d type at the command line. If one of your arguments contains a space, you should include that space in one of the strings, and then the external provider will automatically handle those spaces in a way that is appropriate for the operating system where it’s running.

@apparentlymart Thank you very much for the help and reply. I was suspecting the way these arguments are being passed to be the actual issue. Let me try out your suggestion.

Again, thank you very much for the suggestion and the explanation. Your explanation is very through and in-depth.