Hi all,
My name is Korinne, and I’m a Product Manager for Terraform 
We’re currently working on a project that will allow users to more easily refactor their Terraform modules and configurations, set to be generally available in Terraform v1.1. From a high-level, the goal is to use moved statements to do things like:
- Renaming a resource
- Enabling count or for_each for a resource
- Renaming a module call
- Enabling count or for_each for a module call
- Splitting one module into multiple
The above actions are declared within a module’s own source code, which allows Terraform to automatically apply the changes as part of creating a plan. This feature is currently included in v1.1.0-alpha20211006, and we’d be really grateful to get feedback from the Terraform community. You can learn more about this new feature (along with additional use cases) in its draft documentation.
Why this is important
Up until now, the primary tool for refactoring configuration has been terraform state mv, used in conjunction with corresponding changes to the configuration. While this can accomplish simple refactoring operations, module authors and Terraform Cloud practitioners had no way to coordinate these changes themselves. Here’s a quick summary of the problems our different users faced with refactoring:
-
Terraform CLI (open-source) users: Refactoring had to be done manually, by constructing commands to rename the state objects, and coordinate those changes with the configuration. This was often error-prone, with little feedback about how to resolve those errors.
-
Terraform Cloud users: General refactoring within Terraform Cloud workflows was not really possible, since the only interface is to create a plan based on changes to the configuration. In order to do any refactoring, users would need to temporarily switch to using local state with the CLI, and make the changes out-of-band from the normal workflow. This also assumes the user has admin access on the workspace, as they’ll need access to the state file.
-
Module authors: Directly refactoring modules was not possible, since the names of existing state objects were not within their control. In order to refactor a module, a new major version would be released to indicate a breaking change, and a guide to manually upgrade the state would be supplied to the end-user.
With this new mechanism, users can plan, preview, and validate multiple refactoring actions simultaneously. This allows for:
-
Easier and safer refactoring: Common tasks which may have involved many individual commands, like moving individual resources or modules into groups expanded with
countorfor_eachcan now be done in a single apply step, showing the simplified view of the prior and new resource names. Intermediate steps that resources may have passed through don’t need to be displayed, nor do changes that did not affect existing resources at all. -
Refactoring within Terraform Cloud workflows: Refactoring based on the configuration itself allows all changes to be evaluated and applied from a plan. Cloud practitioners can use the same workflows they are accustomed to for refactoring operations that were not previously possible.
-
Enabling module authors to refactor a public module: An author of a shared module can safely refactor the internals of the module without requiring manual steps be taken by the practitioner. The
movedblocks can be committed in the module source, and only when existing state with the previous resources are encountered, will the necessary move operations take place.
How to try it out
Pre-reqs
- Install Terraform v1.1.0-alpha20211006
Recommended setup
- For tutorials and use-cases, the documentation is the best place to start.
- For suggested environment setup, read on below.
We suggest first writing a small configuration that calls your real module, in a version control branch forked from the latest stable release of your module. For example, you could make a subdirectory refactoring-test inside your module’s directory and instantiate like this:
module "main" {
source = "../"
# (and then some input variable values that will
# make sense for a standalone test environment)
}
If you run terraform init and then terraform apply in this refactoring-test directory, you can then create a baseline state representing your module as currently implemented.
Once that’s successfully created, you can try editing your main module’s source code to implement the refactoring you want to try, including any moved blocks to describe the changes you’ve made. Once you have at least one moved block in your module you’ll also need to enable the config_driven_move experiment within your module, which you can do by setting the experiments argument inside a terraform block inside your module directory:
terraform {
# "config-driven move" is the project name for our first
# set of work related to refactoring, focused on moving
# objects between resource instance addresses. This
# subset of the functionality will be stabilized for v1.1,
# but we hope to continue adding other refactoring
# features in later releases.
experiments = [config_driven_move]
}
With the configuration changes made, run terraform plan to see the effect. If it worked, you should see Terraform report that some existing objects from the state moved to new addresses. If your moved blocks had from set to an address created by original terraform apply then you should see that it overrode Terraform’s usual behavior of planning to destroy the object at the old address and to create a new object at the new address.
Intended audience for Alpha feedback
Note: This feature is a bit more challenging than most to try out in a development environment, because it relates to ongoing changes to an already-existing instance of a module. We recommend instantiating your updated new module against an empty Terraform state, so these features won’t have an impact.
-
Maintainers of shared modules: Specifically shared modules that are used by multiple callers, especially public modules in the Terraform Registry. It’d be great if you could think of some refactoring you’ve either done in the past, or hope to do in the future – and try implementing those changes in a test-only environment.
-
Terraform Cloud/Terraform Enterprise users: If interested, please reach out to me (kalpers@hashicorp.com) for more details on how to test this. At the moment, we recommend trying this out with local state only, and setting up local credentials. We also can help ensure this isn’t used in production.
How to share feedback
- For this first round of feedback, please comment directly on this post. At this early stage, we’re aiming to keep all feedback and bug reports separate from those in the current release of Terraform. That being said, we’re looking for all failure and success stories – all feedback is valuable to our work.
- As we get closer to the stable release of these features, we’ll close this topic and ask for feedback as separate issues in GitHub.
Scope and known limitations
This first set of functionality also intentionally has a limited scope so that we can solve a subset of problems now, rather than waiting for a perfect design for all possible problems. In particular, the following are intentionally out of scope for this first iteration:
-
Moving objects between states/configurations. These features are only for moving objects between addresses in a single state/configuration.
-
Moving objects between modules packaged separately. The first iteration only allows moving objects between modules that are distributed together in the same repository or other module package, to avoid creating a new kind of external compatibility constraint for shared modules. We want to gain experience with the process of refactoring within a family of modules maintained together by the same team first, before trying to solve the more complicated problem of transferring responsibility to other module packages possibly maintained by other teams.
-
Dynamically-decided source and destination addresses. The first iteration focuses only on the more straightforward case of moving between two addresses we can write down statically in the configuration.
If you have modules where you need the out-of-scope capabilities described above, we’d still love to hear the details about your use-cases if you’d be willing to share them. However, we want to be explicit that we don’t intend to address that particular sort of feedback until a later release.
Thanks in advance for your testing efforts and feedback! If you’d also be interested in jumping on a call after trying it out, feel free to comment below and I’ll follow-up with you. We really appreciate your time on this, and excited to see how you like it.
