How to Extract AST

Terraform
1

Hello Team,
I am trying to write a golang code to get the abstract syntax tree (paser, grammar) for terraform. Could you please help me on how I can do that?

Thank you,
Megh Shetty

2

Hi @MeghvShetty,

The syntax tree is an implementation detail that isn’t exposed as a public API. However, there may be a different way to achieve what you are trying to do; can you share a little more about your underlying goal? Thanks!

3

Hi,
sure I can I am a security researcher doing research on SAST based approach. My goal is to get AST and find vuln against a rule set that I have defined.

4

Hi @MeghvShetty,

If you have already decided that analyzing the syntax is the only available option for your goal then I think you will need to essentially reimplement Terraform’s configuration loader in your own codebase using the lower-level hclsyntax library, which is what Terraform uses internally for the first level of parsing, before then doing higher-level analysis.

If you pass the contents of a .tf file to hclsyntax.LoadConfig then you will get a syntax tree describing the raw structure of the file.

5

Thank you, Do you have any other suggestion or best day approach this ?
I want to build a SAST scanner for terrafrom code.

6

Can you give an example of one kind of vulnerability you are aiming to detect?

7

Ensure that Cloud SQL database Instances are not open to the world. More of pattern matching too maybe

8

That’s going to be very tricky, since a full solution would require evaluating arbitrarily complex Terraform expressions. But perhaps you’re only looking to statically catch simpler cases?

It may be that this project: GitHub - hashicorp/terraform-config-inspect: A helper library for shallow inspection of Terraform configurations may be of use to you, at least as some inspiration.

9

Hello,
is hclsyntax.LoadConfig is a function within the hclsyntax? I am referring to the go lang official documentation but couldn’t find the function.

thank you,
Megh Shetty

10

hclsyntax.LoadConfig don’t exit any more in the package

11

I think focussing on the precise naming of a specific function in a low-level library (I guess perhaps ParseConfig was meant instead of LoadConfig?) is taking your attention away from the real issue here…

There is no ready to use library code that will expose a Terraform AST.

If you intend to go further down this path, you will inevitably need to either copy or emulate large chunks of code from either Terraform itself, or the terraform-config-inspect library I previously linked to.

If you look through the code of terraform-config-inspect, the amount of re-implementation of code from Terraform itself that that library has needed, is evidence in support of my assertion.

If you are truly prepared to undertake a project of this complexity, then the source code of Terraform and/or terraform-config-inspect will be useful examples to you.

Conversely, if you feel that looking through those codebases for inspiration is not something you are willing or able to do, then I think you should abandon this project, as you don’t seem to be prepared for the complexity that awaits you if you proceed.

You previously said that you wanted to:

however you did not go into further detail about what that actually meant in terms of Terraform configuration.

It wouldn’t be that hard to craft a checker for people writing something like:

resource "somekindofcloud_sql_database" {
  open_to_world = true
}

but that’s a toy example… do you also aim to detect something like:

variable "open_to_world" {
  default = true
}

resource "somekindofcloud_sql_database" {
  open_to_world = var.open_to_world
}

?

Without knowing where you want to draw the line about the capabilities of your checker, it is difficult to know what advice to give.

Another complexity to consider: Most Terraform configuration is written in HCL, but technically Terraform will also read a JSON format, designed for programmatic construction of input from languages that cannot easily write HCL… do you aim to support that?

In conclusion: If you elaborate further about your aims and requirements, it may in return become possible to give further advice… but at the moment, you seem to be betting on there being one function in a library somewhere that will solve everything … and there isn’t.