I’m new to Terraform - I’m writing a PHP script to generate a .tf file based on user input and execute the tf via PHP’s shell_exec.
But then I run the PHP script executable terraform apply - the UI sits for a long time without me knowing the progress.
Is there some sort of indication I can get from terraform apply post each action so that I can show which action was applied (eg. subnet created) and as a result show a HTML5 progress bar ?
This way the user knows that the webpage doesn’t seem to load forever.
If you are trying to build a new UI layer on top of Terraform Core then you might prefer to consume the raw events that the UI normally reacts to, which you can get in JSON stream format by running with an additional option:
terraform apply -json
Working with this will require you to run Terraform in a way where the Terraform process runs in the background with its stdout handle connected to a pipe that your caller is then reading from. You’ll probably want to buffer one line at a time and then feed each line into a JSON parser to get the data.
I’m not familiar with these PHP functions (both shell_exec and exec) but just from reading the docs I think unfortunately these functions are not sophisticated enough to implement what I described above, because they seem to always block until the program is complete and then return the entire output as a string. To implement this in a streaming manner will require a different approach where the program runs in the background and you get a file handle to read from to gradually capture the output. I’m not sure if PHP has a function like that.
Thanks to the comments on the PHP docs I see that there is also proc_open which seems like it could be a better fit for consuming Terraform’s output in a streaming way. I think you could also use popen for a simpler usage pattern, because in JSON mode Terraform is non-interactive and so you only need to read from it.
Although I’d also caution that running Terraform directly from your web server process might be risky because PHP’s request timeout might cause the request to be aborted before Terraform is finished, and if you restart your web server while Terraform is running that might also cause you some problems.
A typical way that people have implemented web wrappers around Terraform is to run Terraform through a job queue system and have the job stream the Terraform output into persistent storage somewhere, and then the web UI will just read from that storage and quickly parse whatever is there so that the web requests are independent of the Terraform worker processes.
I was thinking more like this (keeping in mind that my PHP knowledge is decades out of date, so this is probably not totally right ):
$f = popen("/usr/bin/terraform apply -json tfplan", "rb");
while (($line = fgets($f)) !== false) {
$event = json_decode($line);
// Then do something with $event,
// which is describing one event
// emitted by Terraform Core.
}
fclose($f);
You can learn about what format those $event objects will be in from the documentation about Machine-readable UI. You can use a subset of the events Terraform emits to detect what operations are currently in progress and what has already completed.
I don’t think this data is sufficient for a progress bar in particular, because there isn’t information about how many events there will be in total and so there’s nothing too divide by to get a percentage complete, but you can still show e.g. how many operations have already completed and how many are currently active.
Is there a way I can know the total number of JSON lines in the output before start ? so that if there are 7 JSON lines (as in the example) I can show a progress indicator of100% in 7 intervals.
If you have previously created a plan file you can extract details of what would be done (the JSON output from terraform show) and use that to compare with the lines output from apply as things are completed.
):