Newbie Help - Modules not re-downloaded when updated in git repo

Hi,

I might be being stupid here, or not following some guideline as I’ve largely learnt this at need during our move from manual building to automated building in GCP.

Essentially, I’m writing a load of modules which I then upload to bitbucket, and we have terraform fetch them and run them.

However, terraform doesn’t seem to ever bother updating a module it’s already fetched, even when it’s quite clear from the git status that it’s changed. Is this by design, or am I missing some step which I should include for when I’ve updated a module and it needs refreshing?

I know about terraform get -update however this fetches all the modules, which generally means it takes awhile and it’s re-fetching modules which haven’t changed. I’ve tried various invocations of this command hoping I’d figure out a way to fetch just the module I know has updated but I haven’t yet managed this. Currently, I’m removing the directory manually, then re-running terraform init which pulls down the missing module however this feels rather clunky to me, and I’m surprised terraform doesn’t at least acknowledge that the git repo is newer than the local module?

Is this something really obvious I’m missing, or does no one know and this isn’t the ‘normal’ behaviour?

Is it the standard behaviour and I’m doing something wrong?

Hi @djsmiley2k,

As you’ve seen, Terraform doesn’t have any particular awareness of Git: from Terraform’s perspective, Git is just like any of the other module installation sources in that it gives Terraform a tree of files to copy into the cache directory under .terraform/modules. After that, Terraform will only change what’s in there if you ask it to upgrade, as you saw, because as far as Terraform can tell the files from that remote package are already installed.

I can understand that this behavior would be frustrating when you’re currently trying to develop both modules at the same time. Terraform’s understanding of module packages is that they are immutable until the source address changes, which evidently doesn’t hold for a Git repository under active development where the definition of “HEAD” is constantly changing.

With that said, I think your current strategy of deleting the directory in question and re-installing it is a viable one, albeit clunky. Another option is to cd into the directory and run git pull as you normally might for a Git repository, because Terraform typically installs the module there by running git clone.

However, I’ll note that this won’t be true in all cases: if you have two references to exactly the same source address from two module blocks in your configuration then Terraform will try to optimize fetching the second one by just re-using the files already downloaded the first time, but that results in the second directory just being a plain directory containing the files from the “package”, not a Git working directory as would be true for a direct download. In that case, you would indeed need to delete each of them individually, or use the command line options you’ve already learned about to tell Terraform to ignore what’s already installed.

1 Like

Thank you, that fully answers my question :slight_smile:
I’m guessing there’s no command line options to easily tell terraform to re-fetch a specific module, or mark it so that it should be refetched?

I did read about the taint options, but they seem to be related to the resources that have been deployed, rather than the source for the modules themselves.