Module depends_on modifies the called module

Terraform
1

Hi All

been working on terraform for only a short time, but something seemingly straightforward has be most confused in behavior. Could be something simple that I misunderstand, but hoping for some assistance.

I have a couple of projects (both happen to be vsphere ones). I have stripped out provider references which seems self defeating in the re-usable code aspect as it makes them now either a module or a standalone project but I digress.

So both work fine. One creates a win DC and one creates a win member server. I cannot of course have the member server being created at the same time as the DC as the DC will not be up for it to join.

So in my parent TF plan, I simply call the DC module, then set a depends_on to the DC module within the second module which creates the member server.

The issue is when doing so, I then get errors relating to the module containing the depends_on (in this case one relating to not defining the drive0 size), but this is declared from the clone template in fact.

If I remove the depends_on clause, both module calls will work fine just in parallel rather than sequential. Am I missing some impact regarding the use of depends_on? I have done much reading but cannot seem to see anything that would cause this issue.

Simplified code snip which will fail:

provider “vsphere” {
user = var.vsphere_user
password = var.vsphere_password
vsphere_server = var.vsphere_server
allow_unverified_ssl = true
}
module “DC_Builder”{
source = my_source_git_module
#variables to pass to the module here
}
module “Winserver_Builder” {
source = my_source_git_module
#variables to pass to the module here
depends_on = [module.DC_Builder]
}

Thanks for any input

2

Hi @terraform_cw,

I think to answer this question we will need to see exactly what error messages you saw in the configuration with depends_on.

depends_on between entire modules can significantly change the dependency graph by adding a large number of new dependency edges (everything in one module now depends on everything in the other) so change in behavior is not surprising to me but if you share the exact error message I can hopefully explain exactly what happened in this case and possibly suggest a different approach that won’t hit the same problems.

Thanks!

3

thanks @apparentlymart

the error I get is this:

tf error
tf error765×137 4.51 KB

but this is just referencing the defaults from the template being cloned with the lines:

disk {
        label            = "disk0"
        size             = "${data.vsphere_virtual_machine.template.disks.0.size}"
        eagerly_scrub    = "${data.vsphere_virtual_machine.template.disks.0.eagerly_scrub}"
        thin_provisioned = "${data.vsphere_virtual_machine.template.disks.0.thin_provisioned}"
    }
4

Also, not sure if it has a relevance, but if I define:

depends_on = [module.DC_Builder.vsphere_virtual_machine]

instead of the module itself, it will run through but will not wait for the creation of the resource and again generate both in parallel

5

Thanks for that extra information, @terraform_cw.

The error message you saw here seems to be one returned by the provider itself rather than by Terraform, and so unfortunately the details of exactly what’s going on here are not very familiar to me; I don’t have much experience with the hashicorp/vsphere provider in particular.

However, I do have a guess as to what might be happening:

when you tell Terraform that a module depends on another module, that is a shorthand for saying that everything in one module depends on everything in another, and that “everything” includes data blocks like your data "vsphere_virtual_machine" "template" block.

This means that if there are any planned changes for any resources in module "DC_Builder" then Terraform sees that it mustn’t try to read from that data resource until the apply step, because otherwise it might read stale information from before making those changes and therefore the plan would become invalid in the middle of the apply step.

Normally this wouldn’t be a problem because Terraform will track data.vsphere_virtual_machine.template.disks.0.size as an “unknown value” (shown as (known after apply) in the plan UI) and ask the provider to create the most complete plan it can based on that partial information.

However, I think the hashicorp/vsphere provider might have a bug where it’s treating an unknown value for size as if you haven’t set a value for that argument at all, and so it’s returning an incorrect validation error.

If I’m right about this (which I’ve not confirmed) then unfortunately the real fix for this would be in the vSphere provider itself, changing it to treat an unknown value as meaning to skip the validation until a later step. You could report this in the provider’s repository to give the provider developers an opportunity to study it further and understand what’s going on here.

Without changing the provider I think you will unfortunately need to restructure your configuration so that the size of the disk is always known during the planning phase. That means that the data "vsphere_virtual_machine" "template" block must not depend on any other resource blocks. Because depends_on in a module block is the same as adding a dependency for everything inside the module, this means that you must not use depends_on in the module block and must instead specify the dependencies more precisely.

I don’t know what the relationship is between module "DC_Builder" and module "Winserver_Builder", so I can’t give specific advice on how to structure this, but the general idea would be to make your first module export output values for the specific objects that the second module will refer to, and then pass those output values into input variables in the second module. Terraform will then see each of those connections as an individual dependency, and so the dependency graph can be considerably more precise than a whole-module depends_on can be, and in particular you can hopefully avoid the data resource having any resource dependencies at all.

6

I too have encountered this behaviour of depends_on.

@apparentlymart , do you think it might be worth filling a feature request issue for a new resources_depend_on meta-argument for modules? It would work exactly like depends_on, but only act on resources, not data blocks, within the module on which it was placed.

I suppose some people might question the purity of the construct, but I think it is just shorthand for a particular kind of dependency relationship which users often happen to want.

Meanwhile, I think I know a way that @terraform_cw can get the desired behaviour now… Recently, by chance, I discovered that depends_on has an undocumented feature, that it can depend not just on resource addresses, but also variable values.

Therefore, if an output variable is added to the DC_Builder module, holding some piece of generated data from the VM resource, and the other module made to depend on the output variable value, I believe the desired behaviour will occur.

7

Hi @maxb,

If I’m understanding correctly what you are suggesting, I think that what you described is essentially just the first half of what I had proposed.

That is: you’d still need the output value in the first module, but you could skip declaring the input variable in the second module, because everything in the second module would then depend indirectly on only whatever that particular output value depends on.

I don’t see any reason why that cannot work, but I’m also not sure I see why it’s better than passing in whatever output value your first module is returning. depends_on is intended only for situations where the dependencies don’t follow the natural data flow, but if you are able to export something that represents the dependency then the dependencies do seem to follow the natural data flow and so there’s no reason to use depends_on rather than just passing the values and allow Terraform to automatically infer the correct order of operations.

8

You’re right to question my suggestion - in my enthusiasm to share what I’d discovered about depends_on being able to accept values as well as resource references, I’d overlooked that the placement of the data sources here is not quite the same as in the situation I previously experienced, so what I said above isn’t a full solution.

As you said, here, it is actually necessary to pass the value into the second module as an input variable, so that the dependency can be linked only to the resources inside it.

But there is still a place where using depends_on to point at a value rather a resource address could be useful: the abbreviated code snippet is somewhat suggestive that the Winserver_Builder doesn’t currently need to make use of any values from DC_Builder, so there may be a lack of obvious way to wire up the dependency via natural data flow.

In this case it may be useful to write:

  depends_on = [var.some_value_from_DC_Builder]

inside Winserver_Builder

Separately, do you have any feedback on the resources_depend_on feature I proposed, as a way to make this kind of module sequencing easier for users?

9

Thank you both for the suggestions, the data values indeed may be the issue.

Assuming that is the case, I may actually remove the data sections from the modules and put them into the calling plan to test. Failing this I will look at adding some variable as a dependency. Regarding your query on relationships between the two, there actually is none, i.e. Winserver_builder doesnt require any outputs from DC_builder it just needs to have completed its build before it initiates.

I will try the move of the data blocks and see what happens. Thanks again

10

@apparentlymart leading from your suggestions, I can confirm it looks to be an issue with the vsphere provider.

Having modified my module source to have the drive0.size attribute passed to it rather than attempting to pull this from the template, it works as expected.

Thanks for the suggestion, I will raise it over in the vsphere repo for the devs to investigate

Appreciate the help