Parse HCL treating variables or functions as raw strings hashicorp/hcl

Hi,

I’m using the official HCL Go module https://pkg.go.dev/github.com/hashicorp/hcl/v2@v2.3.0/gohcl to parse some Terraform files that contain Terraform GitHub Provider configuration. I didn’t find a way to parse attributes as raw strings without being forced to provide the decoder with a Context that knows how to resolve any function or variable referenced.

I was wondering if it is possible to use https://pkg.go.dev/github.com/hashicorp/hcl/v2@v2.3.0/gohcl?tab=doc#DecodeBody with a nil Context and therefore be able to use raw values instead of having to interpret things like ${locals.example.[count.index]}.

The goal is to be able to rewrite the .tf files from a script removing a resource if it matches a given pattern.

Sorry for posting it in Terraform subforum, I didn’t find a specific one for HCL questions.
Kind regards!

Hi @jimen0,

Since your goal is to edit configuration, I think the best way to get this done is using the hclwrite package, which is designed to allow you to work directly with HCL native syntax constructs. gohcl, by contrast, is for decoding HCL data into normal Go values, which requires expression evaluation as you’ve seen. Also, reading data into gohcl and then writing it out again will be lossy, because the Go data structures can’t preserve details like the ordering of the arguments in the input, any comments that are present, etc.

This hclwrite package isn’t as mature as HCL’s main parser/decoder because none of the major applications using HCL 2 have needed it yet, but I did personally use it in a side-project related to Terraform and it seems to be broadly working, notwithstanding some edge-cases you can see in the issues in that repository at the time of writing.

I think you could do what you want here with the following steps using hclwrite:

  • Read one .tf file using hclwrite.ParseConfig, producing an hclwrite.File object.
  • Use the Body method on that object to obtain the root body of the file. (That is, the construct containing the top-level blocks.)
  • Iterate over the result of the Blocks method to visit each of the blocks in the file in turn:
    • Use Block.Type() to get the block type, and continue if the result is anything other than "resource", since you said you want to remove resources.
    • Use Block.Labels() to get the labels for the block, which for a Terraform resource block will be a pair of strings: the resource type name and the resource name. (If you want to be robust against invalid input here, you should check first to make sure the result has length 2.)
    • Check whether the two labels match the pattern you are looking for. If not, continue.
    • Call Body.RemoveBlock on the main body, passing in the current block, to remove it.
  • Finally, call File.Bytes() on the original file (now modified in-place by the work so far) to obtain the modified source code. You could perhaps check if it’s different than what you originally loaded and overwrite the original file with the new content if so.

If you want to process an entire module, you can repeat the above for each .tf file in a directory. The side-project I mentioned earlier might serve as a good starting-point, because it also works with all of the .tf files in a particular directory and so maybe you can just replace the cleanBody function with your own logic like the above to get something working quickly.

The hclwrite package’s types encapsulate the raw tokens that an input file is built from and provider a higher-level API for manipulating it, which has the advantage that if you don’t modify a particular body at all then it will be preserved exactly, aside from whitespace changes caused by the pretty-printer. In particular, it will preserve any comments in the unmodified regions, and retain the ordering of attributes within the blocks, both of which would usually be discarded during normal HCL parsing.

Just built a small proof of concept using hclwrite in like 20 minutes and it is simpler and less error prone that my previous approach.
Thank you so much, @apparentlymart for the detailed explanation, the link to a real world example and the recommendations.

Thank you! We can close this thread as resolved now :smiley: