Create resource dynamically from a non-interactive environment

Terraform
1

We are working in a big enterprise product in which we require to generate new infrastructure with Terraform each time that a client creates an account in our API REST.

How would be the best approach to achieve this goal with Terraform?

Thank you

2

Hi @Yamilquery,

Exactly how that would work will depend on the details of what exactly you want to create on a per-customer basis, if and how that interacts with any infrastructure shared between clients, and so forth.

However, I can offer some general information that might help you with your design:

  • If the infrastructure you’re creating will be broadly the same for each client then I’d try to represent it as a reusable Terraform module with Input Variables that provide any client-specific information needed to make the infrastructure unique to the client.
  • It sounds like each client account has its own separate lifecycle, so I expect you’ll want to have an entirely separate Terraform state for each client. I expect your number of clients is in principle unbounded, in which case I’d recommend using an entirely separate backend configuration per customer, as opposed to one backend configuration with a workspace per customer, because Terraform does not generally expect to find hundreds, thousands, or more workspaces for a single backend configuration.
  • You’ll need to plan for what happens when the “template” for per-customer infrastructure changes when existing accounts are already provisioned. Perhaps you’ll need some process to queue up background updates to get the existing customers updated to the latest infrastructure, which suggests having the shared module representing that infrastructure be versioned and tracking which version is currently selected on a per-customer basis.

My initial instinct, assuming the above set of “pseudo-requirements” seems right, would be to write the shared module I mentioned and then develop some custom automation on your end which wraps Terraform.

That automation could, when launched for a particular client, generate on the fly a temporary root module configuration in Terraform’s JSON language variant that selects the right version of the shared module, configures the backend in a suitable way for that specific customer, and passes in to the module whatever discriminator data it needs to produce customer-specific infrastructure. That generated root module might look like this:

{
  "terraform": {
    "backend": {
      "s3": {
        "//": "(assuming you're using the AWS S3 backend)",
        "path": "customers/CUSTOMER-ID/terraform.tfstate",
        "bucket": "awesomecorp-terraform"
      }
    },
    "module": {
      "main": {
        "//": "Using the module registry protocol here, but any source could work",
        "source": "terraform.example.com/customer/aws",
        "version": "1.2.0",
        "customer_id": "CUSTOMER-ID"
      }
    }
  }
}

You could then run terraform apply -auto-approve against this to force Terraform to apply the updates non-interactively. Of course, you’ll need to test the module carefully during development to make sure that applying it without first reviewing the plan will be safe for all customers. To ease that, you might choose to make some other constraints on your custom system such as that upgrading the module for a particular customer must be done one version increment at a time, never skipping ahead, so that you only have to test that the module can safely upgrade from its prior version, not from an arbitrary historical version.

If you are always consistent with how you construct the backend configuration to ensure that it will find the state in the same place every time then it should just automatically find the prior state for each customer automatically. Be sure that each customer has its own distinct storage location.

The approach I’m describing here is assuming that the module to be run will be fully under your control and customers will not be able to provide arbitrary additional Terraform configuration to run. Terraform does not behave as a sandbox itself, so if you are accepting arbitrary Terraform configuration from end-users you’d need to also develop suitable isolation to ensure that customers cannot impact your base infrastructure and each other’s infrastructure.

The details are of course for you to figure out based on your specific requirements, but hopefully the above is a helpful starting point. Here are some more references as additional context for some of the topics I’ve mentioned above:

  • Running Terraform in Automation has recommendations for running Terraform in automation scenarios, including non-interactive scenarios. There are some special command line arguments and environment variables discussed there that might be needed in your case, depending on how you choose to build your automation wrapper.
  • Module Registry API is the protocol Terraform expects for Terraform-specific module registries. You could choose to implement some or all of this protocol to serve your shared customer module internally as I illustrated above, or you could use one of the other module source types and handle publishing multiple versions some other way, such as by publishing each one at a distinct location.

I hope that helps!

1 Like