Managing state when creating infrastructure with both Terraform and "Pure" Ansible

Terraform
1

I’ve got a handful of “pure” ansible projects that run on Jenkins CI to fully manage (create and provision) elements of our SW Dev infrastructure which currently resides in an on-prem vSphere datacenter. Eventually I plan to migrate these projects to Terraform, as we improve and scale our infrastructure.

The kind of infrastructure I’m talking about here is Jenkins clusters, artifact management systems, individual development environments, server and serverless file shares for various caching and mirroring purposes, secrets management, and SCM.

My question is regarding the management of tf state; the “actual” resource state caching is a primary reason for me to move to TF. All vSphere resource usage is currently managed in static Ansible inventory. Although we can use dynamic vSphere inventory plugin to determine “actual” state in “pure” ansible, significant logic and complexity is required in our playbooks to do what TF is doing, and I prefer the idea of keeping infrastructure creation (tf) separate from provisioning (ansible).

I understand that I can just migrate all my ansible infrastructure projects to TF, and each of these could share remote state so they can be run independently and maintain loose coupling. But In the short term, before I’ve moved everything over to TF, will using a hybrid approach, using TF AND Ansibele to create infrastructure, likely lead to problems and conflicts in vSphere resources?

The main goal in sharing remote state would be to ensure that my infrastructure creation projects do not exceed resources available.

With a TF-only approach, each disparate TF project would be run separately, but would be fully conscious of the actual infrastructural “big picture”, and so could be sure to fail in the planning stage when resources are not available of other conflicts arise.

Conversely, if I’m creating infrastructure through “pure” Ansible outside the context of TF it seems without implementing extra logic to analyze resources before using them, I’m creating some inconsistencies for TF to have to deal with.

Sorry if this question isn’t 100% coherent - I guess I’m looking for advice given the situation I’m in. The more I learn about TF, and the more hassle I have to deal with writing infrastructure creation Ansible playbooks using the powerful but buggy and poorly documented VMware.community collection, I’m coming to realize that I would be best off migrating all infrastructure management to TF.

I would love to hear your opinions, suggestions and stories about your journey towards better IaC. Thanks.

2

Hi @timblaktu - I have experienced a bit of the same confusion you may be experiencing when confronted with two seemingly conflicting configuration management tools (Ansible and Terraform). There is no silver bullet, but I have found it useful to re-evaluate what should be considered “infrastructure” and what should be considered “resources”.

In my thinking, resources are deployed into infrastructure. The obvious kind of resource would be a VM or container. Ansible acts on resources, Terraform acts on infrastructure.

This is a fairly arbitrary separation, but is born out of the primitives available to the respective tool. Comparing Terraform providers and Ansible modules one might be confused by the overlapping parts, where there is both an Ansible module and a Terraform provider for that particular thing. Once I looked at it the other way around, it becme easier to decide who owns what: what has only Ansible modules and what has only Terraform providers.

Another thing to consider is what should be immutable. Terraform will converge the state of infrastructure resources when the declaration changes, but making a decision to have only immutable images helped clarify which tool owns what resources.

It may not be appropriate for you, but having a model where Ansible is only invoked by Packer to build immutable images – considered resources – to be deployed into infrastructure controlled by Terraform, has proven neat and easy to communicate.

1 Like
3

Thanks for that @brucellino1! Indeed, in the last week, I have banged my head on using ansible for IaC (with vSphere specifically) for the last time. Essentially I have come to the same conclusion as you - keep IaC and CM separate. I went on a rampage researching using Terraform with Ansible and eventually I stumbled on Pulumi, which I now plan to port all my “pure ansible IaC” projects to ASAP to clean things up (and to prevent head-banging related to trying to use ansible vmware collections).

And yes, what I’m doing is very much creating immutable images (templates, really) using packer/ansible. I was using Ansible to clone them in various sets and use ansible further to add more “layers” of config to animate them in different ways: Jenkins nodes, web/file servers, etc…

1 Like