How do I separate infrastructure per customer?

My use case is a little different than what Terraform is designed for. I want to set up a service similar to Runcloud, Cloudways etc. So potentially thousands of customers each having one or more VPS servers. I want to use Terraform for setting up and tearing down the server for each customer on any one of maybe 4 major cloud providers. Just like how Runcloud and Cloudways do it. So I basically want to use Terraform as the abstraction layer between 4 or 5 different major cloud providers instead of having to connect to each providers API separately and then having to write my own abstraction layer.

How would I do something like this using Terraform? Terraform is designed for infrastructure described in a few state files. I am looking to do tiny micro infrastructures (mostly just one or 2 VPS servers per customer) described in potentially thousands of state files, presumably one for each customer. I don’t think terraform workspaces is the right separation method because the separation is not strong enough.

My current thinking is to create a separate remote state bucket folder for each customer. That customer folder will probably just contain the customers state file which will typically describe just one or more fairly generic VPS servers. Will this work even though Terraform is not really designed with this sort of use case?

The other part of it I haven’t bothered to mention is that I will also combine it with Ansible for setting up apps inside the server once Terraform sets up the server and installs SSH keys and then provides Ansible with the server IP address. That should not be a problem.

Right now, the plan is to run all the command line stuff using Jenkins. So the logical workflow will be:

My User CP website > My Backend > Jenkins > Terraform + Ansible > (Google Cloud or AWS or DigitalOcean or Linode) VPS server(s)

Hi

Thanks for your interest in Terraform, it is a great infrastructure as code platform.

After carefully reading your needs, I believe that Terraform is not the tool for your user case.

First at all, Terraform is not an abstraction layer between Cloud providers, actually if you describe your infrastructure in Terraform for lets say AWS, the same description doesn’t work for Azure.

Second, Terraform is great to manage complex dependencies between infrastructure elements, a small 2 persistent server deployment could be much better deployed using another platform. Your clients probably expect their VPS to have a long happy life, in a cloud native environment servers don’t usually enjoy that privilege.

Have you seen cPanel & WHM products?

Probably Terraform could be a great solution for setting up all your infrastructure, and cPanel for your setting up your clients infrastructure.

I suggest that you try a demo of Terraform to understand it better.
See:

Thanks for taking the time to read my post and respond. Perhaps terraform is not the correct tool. However, Cpanel and WHM are a very different use case than what I have in mind. What I am doing is something similar to RunCloud.io, Cloudways.com, Gridpane.com and Forge.laravel.com.

Although Terraform is technically not designed for cloud provider abstraction it does do that as a consequence of the Providers plugins. It has plugins for all the major providers. I simply need to point to a different github repo to use Terraform on a different cloud provider. Generic terraform scripts for different providers are easily found on the internet. That is much easier than connecting directly to individual API’s and abstracting everything in my backend. I think using the terraform state file as a single source of truth will also be helpful.

If I were to code direct connections to the API’s for each provider myself that is more work for each provider I add. With Terraform, it is not much more work to support 10 providers as it is for just one provider.

So after thinking about it a bit more, I realized that I need to connect directly to the APIs anyways in order to do things like select from a list of available datacenters and misc. other things. So I guess I will have to code all connections to the APIs and the abstraction myself.

Oh well, if it was easy everyone would be doing it. I might still be able to make use of Packer or Vagrant.