How to use Terraform to manage local server cluster?

I bought a few raspberry pis so i can run a local (on-premise) server cluster at my home, without the need for aws or gcloud, for web dev experiments, local continuous integration server, local media server, local dns server, and offloading long running tasks from my macbook.

I am wanting to use terraform to do the consul+nomad+podman setup and whatnot on them. Which resource provider should I use?

Historically I am somewhat familiar with terraform v0.11 for cloud resources, having attempted a secure hashistack module for scaleway back in 2018. However as an indie low-wage open-source dev, cloud experiments were too costly and too time consuming to experiment with.

The raspis are running ubuntu server and are available via ssh access.

Hi @balupton I am doing something similar to you. It’s an ongoing project I started over the holidays:

In my experience so far (which, I should say is closer to “beginner” than “knowledgeable”) you need something get the hardware to a “terraformable” state. In my understanding that basically means that you need to expose terraform providers, ie APIs. Since you’re starting from bare metal which has no APIs, there is a bootstrap problem. I got around this by using Ansible to deploy the intial layers of things I want to Terraform later, e.g. bare-bones Consul and Nomad clusters: Playbook

You can terraform these after they have been deployed, but to get them to a deployed state, you need something that can talk to the lower level of the stack, i.e. the actual OS. You could try using something like the Linux provider, but this provider uses SSH - so you anyway need to have a working OS and SSH user on the remote end. Again, we are back to a bootstrap problem.

There are several options available to bootstrap the local “datacentre” hardware (where datacentre is your bunch of pis), but I would highly recommend Ansible due to it’s readability and availability of community resources. The other options include:

  • Maas: a “metal as a service” product from Canonical.
  • Digital Rebar - will do bare-metal deploys of OS for you and works ok with Ansible and Terraform.
  • Foreman - a bit heavy for RPis, with a few external dependencies, but very complete.

For a small local setup, all of these are huge overkill – unless you’re actually trying to learn how to boostrap and manage an entire datacentre. If you goal is just to get something up to use then I would suggest going the simpler route with plain Ansible playbooks.

One thing I haven’t tried yet is to use Packer to provision the boot images for your pis, and pre-configure your Consul and Nomad configurations on them with appropriate provisioners. Then, when you boot up the pis, they should already have podman installed, and should auto-join Consul and Nomad clusters as desired. It’s not Terraform, but at least it’s simple :slight_smile:

1 Like