Cleaning up failed build with create_before_destroy

Terraform
1

I have an intermittent scenario, where a terraform deploy will fail while provisioning a server. The server has create_before_destroy = true, but because of the error (below) I end up with two resources - the original server (tainted), and the new server (failed).

e.g. communication failed, however droplet was created (now I have 2x instances).

digitalocean_droplet.web: Creating...
digitalocean_droplet.web: Still creating... [10s elapsed]

Error: Error waiting for droplet () to become ready: strconv.Atoi: parsing "": invalid syntax

  on main.tf line 47, in resource "digitalocean_droplet" "web":
  47: resource "digitalocean_droplet" "web" {

A couple of questions:

  1. is this second (new) instance tracked by Terraform, or is the reference to this instance lost?

  2. how would I re-apply my plan (i.e. recover and proceed with deployment), so that I only end up with one server instance?

  3. how would I roll back this plan (i.e. backout deployment), so that I only end up with the one original server instance?

Here’s the script I’m using to run terraform:

terraform init -input=false
terraform workspace select ${TF_WS}
terraform taint digitalocean_droplet.web || true
terraform plan -var-file=${TF_WS}.auto.tfvars -input=false -out=tfplan
terraform apply -input=false tfplan

(deployment is occurring on a CI/CD server - can provide TF config if required)

2

There is no rollback in Terraform, because reasons (no point discussing them here), there is only roll forward.

When I get into that sort of situation it’s usually because DO is having issues. If they are, I’ve learn’t just to hold back until the issue is resolved. I then use terraform refresh to make sure terraform is happy, and then do my plan and apply again. Unless I’ve dicked around with DO via the UI, terraform (I believe) will remove the extra instance. YMMV.

1 Like