I’m using the HCL v2.4.0 to parse some custom HCL files. Overall I’ve been very happy with it. HCL seems to be a good “syntax” that works well for what I need.
I have situation where I want (1) either the blocks in order or (2) I’d like to get back their range information so I can sort them myself. I’m currently doing something like this (code is approximate):
type HCLFile struct {
Services []Service `hcl:"service,block"`
Apps []Database `hcl:"app,block"`
}
var hclfile HCLFile
parser := hclparse.NewParser()
f, diags := parser.ParseHCL(src, fileName)
diags = gohcl.DecodeBody(f.Body, nil, &hclfile)
I have a requirement to go through these in order. I’m assume each type goes into their respective array in order. But I’m not sure how to determine the overall order.
I saw this thread on HCL and the reply from @apparentlymart led me to his syntax cleaner.
That led me to look at a pattern using the hclwrite package and something like hclwrite.File.Body().Blocks(). That returns them all in order and I can use the Type() method to figure out what I have.
Is there a more elegant way to this? Also, is it possible to get back the Position information of each block? I can see Position down deeper in the data structures but I don’t see a way to bring it back with the blocks. I can get it back for attributes but I don’t see a way for the block itself.
(And sorry for a non-Terraform question in the Terraform forum but I didn’t see where else to put it)
Hi @billgraziano,
The higher-level abstractions provided by the gohcl and hcldec packages make some assumptions about what is significant and what is not which means that unfortunately they cannot represent all possible situations.
However, the lowest-level HCL decoding API does preserve overall ordering of blocks and so you should be able to get what you want here by taking a hybrid approach of decoding the toplevel with the low-level API while still being able to use gohcl to decode the contents of those blocks. Something like this:
rootSchema := &hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{ Type: "service" },
{ Type: "app" },
},
}
content, diags := f.Body.Content(rootSchema)
// (handle diagnostics in some way...)
for _, block := range content.Blocks {
switch block.Type {
case "service":
var service Service
diags := gohcl.DecodeBody(block.Body, nil, &service)
// (handle diags and do something with "service"...)
case "app":
var app Database
diags := gohcl.DecodeBody(block.Body, nil, &app)
// (handle diags and do something with "app"...)
}
}
The BodyContent.Blocks field is a flat slice of pointers to hcl.Block objects in the original order given in the source, so you can then iterate over that slice to interleave the processing of blocks of different types.
There’s more information on this low-level API in the manual section Advanced Decoding with the Low-level API. This API is the one that gohcl and hcldec are both implemented in terms of, and exposes all of the details of the HCL infoset.
1 Like
Thanks for the reply. That looks interesting. I’ll see what I can build using this.
Just wanted to pop back in and say Thank You @apparentlymart. It looks like I can make it do what I need based on your comment. It gets me back different types in the proper order and with the Range.
I did find one thing weird though. It doesn’t seem to populate the labels. I can see them in the Labels field but they aren’t populated into the structure. Full working code is below. Maybe you can spot something obvious I did wrong 
It may be that’s a quick of mixing API levels. It’s easy for me to work around. Mostly posted this to say Thank You!
package main
import (
"fmt"
"log"
"github.com/davecgh/go-spew/spew"
"github.com/hashicorp/hcl/v2"
"github.com/hashicorp/hcl/v2/gohcl"
"github.com/hashicorp/hcl/v2/hclparse"
)
func main() {
type Service struct {
Name string `hcl:"name,label"`
Address string `hcl:"address"`
}
type App struct {
ID string `hcl:"id,label"`
Port int `hcl:"port"`
}
type HCLFile struct {
Services []Service `hcl:"service,block"`
Apps []App `hcl:"app,block"`
}
rootSchema := &hcl.BodySchema{
Blocks: []hcl.BlockHeaderSchema{
{
Type: "service",
LabelNames: []string{"name"},
},
{
Type: "app",
LabelNames: []string{"id"},
},
},
}
src := `
service "abc" {
address = "1.1.1.1:80"
}
app "id-37" {
port = 8080
}
service "xyz" {
address = "1.1.1.1:1230"
}
`
parser := hclparse.NewParser()
bb := []byte(src)
f, diags := parser.ParseHCL(bb, "test.hcl")
if diags.HasErrors() {
log.Fatal(diags.Error())
}
content, diags := f.Body.Content(rootSchema)
if diags.HasErrors() {
log.Fatal(diags.Error())
}
//spew.Dump(content)
//spew.Dump(content.Blocks[0])
fmt.Printf("Blocks: %d\n", len(content.Blocks))
for k, block := range content.Blocks {
fmt.Println("-------------------------------------------------")
fmt.Printf("key: %v block.type: %v\n", k, block.Type)
fmt.Println("DefRange:", block.DefRange)
fmt.Println("TypeRange:", block.TypeRange)
fmt.Println("LabelRanges:", block.LabelRanges)
fmt.Println("Labels:", block.Labels)
switch block.Type {
case "service":
var svc Service
diags := gohcl.DecodeBody(block.Body, nil, &svc)
if diags.HasErrors() {
log.Fatal(diags.Error())
}
// spew.Dump(svc)
fmt.Println("Name (why empty?):", svc.Name)
fmt.Println("Address:", svc.Address)
case "app":
var app App
diags := gohcl.DecodeBody(block.Body, nil, &app)
if diags.HasErrors() {
log.Fatal(diags.Error())
}
// spew.Dump(svc)
fmt.Println("ID (why empty?):", app.ID)
fmt.Println(fmt.Sprintf("Port: %d", app.Port))
}
}
}
Hmm, yes… that’s because block.Body is the content of the block, rather than the block itself, and so it doesn’t have access to the block labels.
It doesn’t look like we have an API to decode the isolated block itself into a Go struct using gohcl right now, so with the API as it currently stands I think you’d need to manually copy the values from block.Labels into the appropriate fields of your struct after gohcl.DecodeBody returns.
gohcl could potentially offer another function gohcl.DecodeBlock to decode the block as a whole, including its labels, into the struct value. I don’t have time to work on that right now (I’m currently focused on Terraform project work) but if you’re motivated to work on it we could talk more about it in an issue in the HCL repository.
2 Likes