I have a scenario where I want to be able to run certain steps in a pipeline, and not have that pipeline fail due to errors that will come up. Specifically it’s in reference to transitioning from an aws_s3_bucket_object (AWS provider 3) to an aws_s3_object (AWS Provider 4) and using state rm and terraform import in pipeline steps (see here for discussion on that particular topic.
I have steps I want to put in a pipeline where I need to have the below:
terraform state rm <aws_s3_bucket_object_1> <aws_s3_bucket_object_2> ... ## remove `aws_s3_bucket_object`s from state
terraform import aws_s3_object.object_1 <S3 URI 1>; terraform import aws_s3_object.object_2 <S3 URI 2>;.... # import those S3 objects as `aws_s3_object`s
terraform apply --auto-approve
The issue is, I ideally would like to keep the pipeline running after this first successful run so that it isn’t broken.
If I try to remove from state something that is no longer in state(which will be the case after the first successful run of the above), or import something that is already managed by Terraform, those each give errors:
# can't remove an object no longer in state, since I already did it before
$ terraform state rm <aws_s3_bucket_object_1>
│ Error: Invalid target address
│
│ No matching objects found. To view the available instances, use "terraform state list". Please modify the address to reference a specific instance.
╵
# can't import an object a second time, since I already did it before and TF already manages it
$ terraform import aws_s3_object.object_1 <S3 URI 1>
│ Error: Resource already managed by Terraform
│
│ Terraform is already managing a remote object for aws_s3_object.my_cool_object_2. To import to this address you must first remove the existing object from the state.
Is there a good, at least somewhat clean way in my steps to handle the above errors so that no matter what happens on the rm and the import - whether they have been removed yet from state or imported from AWS S3 - I can always get to that next terraform apply step? Right now I’m looking into the try function, and am not sure if that is the route yet.
It sounds like the errors you are describing are being handled by whatever software you are using to run this pipeline, so I don’t think anything in the Terraform language can help here: you’ll need to find some way to tell your pipeline software to continue running even if these commands return errors.
If your pipeline software is running these commands as a Unix shell script (e.g. a bash script) then one way to do that is to use the || operator in the shell to tell the shell to run some other command (which always succeeds) if the first command fails. For example:
terraform state rm example.example || echo >&2 "Ignoring rm failure"
This will tell your shell that if the terraform state rm ... command fails (returns a nonzero exit status) then it should run echo >&2 "Ignoring rm failure", which is a command that should always succeed as long as it’s possible to write output to the standard output stream.
This is all just shell syntax rather than anything specific to Terraform, so I’d recommend reading the documentation for your shell for more details, but the summary is:
|| is similar to “logical or” in other programming languages, but in a shell true or false is decided based on exit status: exit status zero is “true” and any other exit status is “false”. If the left command fails then the shell will run the right command and use its exit status as the final status for this entire command line. (Note: This is not the same as |, which is the pipe operator and does something very different.)
>&2 means “redirect the standard output handle to the current standard error handle”. File descriptor “2” is the usual place to send error messages, so this is making the echo command send the message to the error stream instead of the output stream.
Because this command line doesn’t redirect the output from Terraform itself, I’d expect the output on failure to include both the error from Terraform and the additional message from echo, and then execution should continue, like this:
│ Error: Invalid target address
│
│ No matching objects found. To view the available instances, use "terraform state list". Please modify the address to reference a specific instance.
╵
Ignoring rm failure
You can customize the message in the echo command to produce whatever will make the output easiest to understand for someone reading it.
Thank you @apparentlymart! I think I’m going to opt for not modifying our pipelines abruptly for some on-off situation, and instead do things manually on the CLI and have all my commands ready for each workspace. I think this would be a better practice rather than hardcoding a unique scenario into something meant for regular use(a pipeline)