Parser from terraform to json and vice versa

Terraform
1

Hi,

We have requirement to edit .tf files and providing back to the user in one of our application. The language of our application is PHP but we can use CLI to parse files.

The approach we are taking to achieve this requirement is mentioned below:

  1. Take .tf files as input
  2. Parse .tf files in JSON so that application can easily read and edit JSON.
  3. Parse this JSON back to .tf files and provide to user

Now we are able to parse .tf files in JSON using this great library: GitHub - alecthomas/hcl: Parsing, encoding and decoding of HCL to and from Go types and an AST.
But unable to find a solution which can parse JSON to .tf files.

We tried to use below methods but is not upto the satisfaction.

For the sake of initial development, we are trying very basic outputs.tf file but our goal is to edit way more complex and full fledged terraform files.

outputs.tf:

output "virtual_machine_id" {
  value = azurerm_linux_virtual_machine.main.id
}

output "private_ip_address" {
  value = azurerm_network_interface.main.private_ip_address
}

outputs.json (from alecthomas/hcl with slight modification, removed ${} from expression):

{
    "output": {
        "private_ip_address": [
            {
                "value": "azurerm_network_interface.main.private_ip_address"
            }
        ],
        "virtual_machine_id": [
            {
                "value": "azurerm_linux_virtual_machine.main.id"
            }
        ]
    }
}

We need help to achieve this goal as we are new to GoLang and are unable find a satisfactory solution.

Thank you in advance.

2

hi @prashant.masuria
did you find any answer to converting json to hcl2 format?
i tried hclparse.NewParser().ParseJSONFile() but it only returns json back
i didnt find any convertor like json2hcl
Thanks in advance

3

It is not possible to write a generic converter from JSON to HCL.

The JSON presentation of HCL, only stores enough information to be read, provided the reader knows the schema of the HCL block types being read.

For Terraform, that means it can only be successfully turned back into full HCL, with knowledge of the Terraform core language, and all of the Terraform providers (at their correct versions!) used in the configuration.

4

Indeed… we can see case where that’s true in the above example:

{
    "output": {
        "private_ip_address": [
            {
                "value": "azurerm_network_interface.main.private_ip_address"
            }
        ],
        "virtual_machine_id": [
            {
                "value": "azurerm_linux_virtual_machine.main.id"
            }
        ]
    }
}

Without any awareness of the Terraform language structure, the above could represent two different HCL structures:

output "private_ip_address" {
  value = "azurerm_network_interface.main.private_ip_address"
}

output "virtual_machine_id" {
  value = "azurerm_linux_virtual_machine.main.id"
}

output {
  private_ip_address = {
    value = "azurerm_network_interface.main.private_ip_address"
  }
  virtual_machine_id = {
    value = "azurerm_linux_virtual_machine.main.id"
  }
}

Terraform itself can decode the JSON because Terraform knows how its output blocks work. For another tool to successfully translate Terraform language from JSON to HCL will require that tool to have the same information Terraform does about the expected structure.

For output blocks that information is built in to Terraform itself because they are always the same. But for the contents of resource, data, and provider blocks the provider plugin is the one describing the schema, so a fully general transformer would need access to the provider schemas too.

I would generally suggest not trying to use translations between the syntaxes in the way described here. The intended model is that machine-generated configuration is always .tf.json and human-generated configuration is always in .tf, and Terraform itself can successfully read both of those without any conversion. There should rarely be any need to convert the syntax before sending it to Terraform, because Terraform already supports both syntaxes fully.

If you do need to make machine-driven transformations to .tf files that are normally edited by humans then the robust way to do that is to manipulate the HCL native syntax directly using the hclwrite Go package. Using that directly will require at least part of your application to be written in Go, but there is a third-party tool hcledit which exposes some of the hclwrite operations in an executable program, and you might be able to make the changes you want to make using that tool.

Converting from native syntax to JSON and back is not a viable strategy because it is always lossy. For example, the conversion to JSON will lose details about source formatting and locations of comments because JSON itself does not support comments. Modifying the native syntax source code directly can preserve more of the source detail by surgically changing only the parts that need to be changed, rather than regenerating the whole file.