I have wondered for a long time why there is no template(), and instead, the only way to interpolate templates in dynamic text is by reading it from a file. It seems that a more generic template() function would be just as readable for file input by simply adding a call to the existing file() function in the string input field.
This feels like an opinionated choice that Hashicorp made and I do not understand why we are limited to not being able to interpolate dynamic content unless it specifically is read from a file (not a input variable, not an output from a data/resource, not parsed from a datastructure deserialized via yamldecode(), etc…)
Why templatefile() and not the more generally useable, and just as readable, template() (optionally with a file() call as an arg)?
The template_file data source which templatefile replaced caused an uncountable number of people to be confused by having to provide a string template containing a string template and therefore having to negotiate multiple levels of escaping.
That data source never intentionally supported dynamic template evaluation and only did so as a side-effect of its implementation. But the fact that it did support it meant that it was then hard to use for anyone using it for the purpose it was intended for, which was to render a fixed template hard-coded into the program.
Therefore Terraform v0.12 addressed that problem in two ways:
The templatefile function reads directly from disk, thereby ensuring that the template is always immediately evaluated and we never get into the confusing situation of having a string containing literal template sequences.
The inline template syntax provides all of the same capabilities directly inside expressions in .tf files, for situations where the template is not complex enough to warrant factoring it out into a separate file.
You are right that it was an intentional decision to explicitly not support dynamic template evaluation. That is because it effectively amounts to an “eval” function – dynamically evaluating arbitrary expression strings built at runtime – and that is a troublesome capability for a language like Terraform which relies heavily on static analysis in order to support validation and planning prior to making actual changes in remote systems.
Although we have intentionally not supported dynamic evaluation of templates constructed at runtime, we are interested in supporting the less troublesome variant of statically declaring a template in one part of the configuration and then passing it to another part (e.g. a separate module) for the final rendering. The distinction here is that the template is still compiled statically and passed around as a special “compiled template” data type, instead of as a string. I wrote a prototype of that a while ago to prove that it’s possible, but our attention has been elsewhere and so we’ve not prioritized final implementation of that yet.
If you’re interested in this capability then I’d encourage you to share some details about your specific use-case in the issue representing this request:
Hi - thanks for the thoughtful and thorough response!
I appreciate what you mean with multiple levels of escaping being confusing, which is something shell programmers notoriously struggle with.
I also, in general terms, appreciate how Hashicorp has been restrained with Terraform features in order to somewhat sandbox users and promote readable and maintainable configs. Given the way Terraform phases and evaluates planning, there are already some situations that can be frustrating to cleanly implement (thinking, for_each keys depending on data outputs for example) - so I can see how dynamic eval type logic could get people into trouble.
One pattern I use quite a bit is making Terraform configs with “config data” broken out into YAML, which lets us keep our Terraform modularized & general (which cleanly documents dependency relationships in IaC), gives us a clean boundary between “config data” and logic, and allows for very user-accessible “gitops” flows for engineers to submit config changes without understanding Terraform. But this does mean we sometimes struggle with ways to represent config data in a way that still works within the Terraform paradigm/flow - this is one area where being able to define reusable/parseable templates that reference other bits of config data is a very useful way to deduplicate or establish default (but overridable) relationships. YAML anchors are also useful for this, but very limited as well.
Anyways, I appreciate your thoughts, and I’ll contribute to the issue! If I am understanding your description of static templates that are passable, but can be validated/parsed before plan phase, it would probably work great in my use case.