How terraform resolves its graph

Terraform
1

I’m not sure where the best place to post this question is.

I’ve been playing with writing a few different DSL config languages using HCL. I have all the basics working for my purposes but I’m curious how to introduce shared dependencies between blocks or if that’s even possible without refering to terraform core codebase in some way.

for example

auth "file" {
  username = "b0bu"
  value = "/tmp/auth"
}

auth "token" {
  username = "b0bu"
  value = "12345"
}

auth "keyvault" {
  url = "https://keyvault"
  value = "12345"
}

thing1 {
 // other config 
  auth = auth.file
}

thing2 {
  // other config 
  auth = auth.token
}

It’s occured to me that all of the things I want to manage in this way may require different credentials either because they are edge cases or because they’re hosted on multiple platforms. I’ve played with the idea of just embedding a static auth = {} map or block in each thing block. There’s also the issue of “auth” requring different attributes depending on what “backend” it is. Whether it’s a file, vault or azure keyvault etc what’s required will be different.

Is there a starting point for me to start thinking about how to implemented this? Even at it’s most basic functionality. I’m trying to understand where hcl stops and terraform begins it might not be practical for me to pursue. But right now my knowledge on it is bordering on zero.

2

Hi @b0bu,

Terraform tracks dependencies via references in the configuration. So in your example config, thing1 has a reference to auth.file, so a graph edge is created between the two nodes. The same for thing2, which refers to auth.token, so an edge is added to ensure auth.token is evaluated before thing2.

Even explicitly added dependencies via depends_on work the same way, the depends_on attribute just creates a location to put more references, which will cause those edges to be created when constructing the graph.

3

There’s a little more info on how Terraform uses the HCL API to achieve this sort of inter-block dependency in this documentation section:

https://hcl.readthedocs.io/en/latest/go_patterns.html#interdependent-blocks

4

@jbardin thank you very useful. I have a lot to learn about how this is implemented.

@apparentlymart thanks, that was a good start I read all the docs, as a newbie still a bit confused about some things. But given the following language

// thing.hcl
thing {
    name = "some_name"
    ...
    file {
        user = auth.file.local.user // file.local.user haven't decided
        path = auth.file.local.path
    }
}

auth "file" "local" {
    user = "b0bu"
    path = "/tmp/token"
}

I managed to at least get a traversal for user and path, but not sure what to do with them. The cut down implementation of the graph bit is still missing. I’m trying to fight the urge to “resolve” auth.file.local.user by building up an internal struct of the file thing.hcl where auth is just procedurally built first, pulling the values out and working off of that bypassing the graph. The idea is to try and understand though, so feel like I’d it’d be missing the point.

        ....
		case "thing":
			thingContent, thingContentDiags := block.Body.Content(thingAuthBlockSchema)
			diags = append(diags, thingContentDiags...)
			if thingContentDiags.HasErrors() {
				log.Fatalf("error parsing 'thing' body against thingAuthBlockSchema %v", diags)
			}

			for _, authBlock := range thingContent.Blocks {

				switch authBlock.Type {
				case "file":
					var fileAuthBlockSchema = &hcl.BodySchema{
						Attributes: []hcl.AttributeSchema{
							{
								Name:     "user",
								Required: true,
							},
							{
								Name:     "path",
								Required: true,
							},
						},
					}
					fileAuthContent, fileAuthContentDiags := authBlock.Body.Content(fileAuthBlockSchema)
					diags = append(diags, fileAuthContentDiags...)
					if fileAuthContentDiags.HasErrors() {
						log.Fatalf("error parsing body against fileAuthBlockSchema %v", diags)
					}

					// get dependencies ???
					if attr, exists := fileAuthContent.Attributes["user"]; exists {
						traversal, travDiags := hcl.AbsTraversalForExpr(attr.Expr)
						diags = append(diags, travDiags...)
						if travDiags.HasErrors() {
							log.Fatalf("traversal error for file auth 'user' attr %v", diags)
						}

	                _ = traversal // what do
					}

				case "keyvault":
					fmt.Println(true)
				}
5

Hi @b0bu,

I think you are on the right track.

I think the main thing you’ve got not quite right here is using hcl.AbsTraversalForExpr(expr) to analyze the expression, instead of the expr.Variables() method.

hcl.AbsTraversalForExpr makes a stronger assertion: that the expression must be just a traversal-shaped expression, without any other operators such as string templating or arithmetic.

expr.Variables() instead searches the tree of expressions starting at expr to see if any “traversals” are present, and returns all of the ones it finds.

Once you have a bunch of hcl.Traversal objects, the next thing Terraform does is try to raise the level of abstraction to its own “address” objects in package addrs, using the logic here:

The graph that Terraform constructs for itself internally is actually based on these Terraform-specific address types rather than the HCL traversals directly. That’s not necessarily the only way to do it, but it’s the approach we have the most experience with.

The actual expression evaluation happens much later in a separate step. That part’s actualy kinda hard to observe in Terraform in isolation because it’s split over a few different subsystems, but perhaps the best starting point is here, where Terraform once again analyzes all of the expressions to figure out what to put into the hcl.EvalContext and then fetches all that data to create the HCL-level scope:

Terraform ends up constructing a separate hcl.EvalContext for each expression (or, in some cases, whole block) being evaluated, which is productive in Terraform because there can potentially be many different objects in scope at different phases. It might not be necessary for you to do that, but I’m not sure.

I’m sorry that Terraform is quite a complicated example to learn from, since all of this stuff is woven around the main execution engine. I don’t have any more direct example to hand to share with you, though. :confounded:

6

@apparentlymart Thanks for the info. I’ve been going through source in my free time but there is a lot of abstraction that’s kind of distracting from the problem I’m trying to solve. I have no state file or graph and since I only have 2 block types in my example which can be dependent, It’s hardly the scale of terraform.

I’ve decoded the auth blocks and know what their values are, I have a nested file block which has “references” to auth. Simplistically is it a hack to want to decode that file block using the auth values as evalContext? In this case I feel like I haven’t “resolved” anything.

Trying to understand the expression types has been a challenge. I’ve gone down the route of traversals := attr.Expr.Variables() for auth.file.local.user and path. Now I know a lot about the expression but I don’t know how to use the traversal type to do anything with it. I feel like I’m overcomplicating it, but it might just be complicated.

From what I’ve read traversals are used to provide range information in diagnostics, “error somewhere here” …and somehow used in evaluating what a.b.c’s actual value should be. I can see from what you’ve sent that terraform creates references from these containing range information. I’m yet to have my eureka moment on what it’s actually doing with it though.