Hello. Trying to upgrade our Jenkins deployment from 0.12 to 0.13. This code deploys many copies of Jenkins in a Kubernetes cluster. We wanted the config to be as DRY as possible so we are using Terragrunt.
The main module source is used by all of the instances.
I’ve added the version.tf to the module source repo
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
rancher2 = {
source = "terraform-providers/rancher2"
}
}
required_version = ">= 0.13"
}
But when I do a terragrunt init I still get the upgrade error
Initializing provider plugins...
- Using previously-installed terraform-providers/rancher2 v1.10.0
- Using previously-installed -/kubernetes v1.12.0
- Using previously-installed -/rancher2 v1.10.0
- Using previously-installed hashicorp/kubernetes v1.12.0
- Finding latest version of hashicorp/rancher2...
Error: Failed to install providers
Could not find required providers, but found possible alternatives:
hashicorp/rancher2 -> terraform-providers/rancher2
If these suggestions look correct, upgrade your configuration with the
following command:
terraform 0.13upgrade .
[terragrunt] 2020/08/19 11:29:29 Hit multiple errors:
exit status 1
If i add the same file to the instance directory the error goes away, which does not make a lot of sense to me as the init is copying that file into the .terraform directory of the target instance. Seems like TF should see the file and just use it.
Am I missing something really simple here?
Again, trying to be as DRY as possible so copying that file to every instance is not desired, even though it’s probably not going to change for a while, it’s still repeating the same config many places for no good reason.
Not sure if the above is enough data, let me know if more detail is needed.
Hi @dsargent3220! Sorry for this rough edge in the upgrade process.
Could you run terraform providers in the same directory where you ran terraform init and share its output? That command should explain where all of the different dependencies are being detected from and might therefore explain where this remaining incorrect dependency on hashicorp/rancher2 is being detected from.
You bet, here it is.
PS C:\source\jenkins-configuration\jenkins-demo> terraform providers
Providers required by configuration:
.
├── provider[registry.terraform.io/hashicorp/rancher2]
├── provider[registry.terraform.io/hashicorp/kubernetes]
└── module.jenkins
├── provider[registry.terraform.io/hashicorp/kubernetes]
└── provider[registry.terraform.io/terraform-providers/rancher2]
Providers required by state:
provider[registry.terraform.io/-/rancher2]
provider[registry.terraform.io/-/kubernetes]
and here is the file created in the .terraform directory with full directory transversal
PS C:\source\jenkins-configuration\jenkins-demo> cd .\.terraform\
PS C:\source\jenkins-configuration\jenkins-demo\.terraform> dir
Directory: C:\source\jenkins-configuration\jenkins-demo\.terraform
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 8/19/2020 10:27 AM modules
d---- 8/19/2020 10:27 AM plugins
-a--- 8/19/2020 10:27 AM 636 terraform.tfstate
PS C:\source\jenkins-configuration\jenkins-demo\.terraform> cd .\modules\
PS C:\source\jenkins-configuration\jenkins-demo\.terraform\modules> dir
Directory: C:\source\jenkins-configuration\jenkins-demo\.terraform\modules
Mode LastWriteTime Length Name
---- ------------- ------ ----
l---- 8/19/2020 10:27 AM jenkins -> C:\source\jenkins-configuration\terraform-jenkins-configuration\
-a--- 8/19/2020 11:29 AM 175 modules.json
PS C:\source\jenkins-configuration\jenkins-demo\.terraform\modules> cd .\jenkins\
PS C:\source\jenkins-configuration\jenkins-demo\.terraform\modules\jenkins> dir
Directory: C:\source\jenkins-configuration\jenkins-demo\.terraform\modules\jenkins
Mode LastWriteTime Length Name
---- ------------- ------ ----
d---- 5/5/2020 1:54 PM .vscode
d---- 7/23/2020 10:59 AM examples
-a--- 5/7/2020 2:29 PM 29 .gitignore
-a--- 8/13/2020 9:27 AM 19064 answers.tpl
-a--- 7/23/2020 3:26 PM 1797 container_repositories.tf
-a--- 7/23/2020 10:59 AM 7405 credentials.tf
-a--- 6/18/2020 4:45 PM 2290 deploy.sh
-a--- 6/18/2020 4:45 PM 1953 Dockerfile
-a--- 8/13/2020 9:30 AM 4500 main.tf
-a--- 5/7/2020 2:29 PM 7234 README.md
-a--- 7/23/2020 10:59 AM 1150 secrets.tf
-a--- 7/23/2020 10:59 AM 2190 skaffold.tf
-a--- 7/23/2020 10:59 AM 7904 variables.tf
-a--- 8/19/2020 10:34 AM 204 versions.tf
PS C:\source\jenkins-configuration\jenkins-demo\.terraform\modules\jenkins> cat .\versions.tf
terraform {
required_providers {
kubernetes = {
source = "hashicorp/kubernetes"
}
rancher2 = {
source = "terraform-providers/rancher2"
}
}
required_version = ">= 0.13"
}
I think I see why it’s not happy, but don’t get WHY it wants rancher2 from terraform-providers/ in the module and hashicorp in the local.
@apparentlymart, forgot to mention in you above reply, not sure if you got it or not, sorry about that.
In addition though, what I don’t get is the same versions.tf file placed in the instance directory fixes things. ???
Also, if a copy of module repo and instance repo would help, i can bundle them up and attach them. let me know.
Hi @dsargent3220,
Based on what you’ve shared here, it seems like Terraform found a reference to a “rancher” provider in your root module, but because it didn’t find a required_providers block mentioning that name it assumed you meant hashicorp/rancher for that module.
Provider requirements are a per-module idea. Terraform uses each module’s declared source address to understand if two modules are using the same provider or if they are using two different providers that happen to have the same local name. Therefore each module that includes a mention of the rancher provider will need to also state what source address to use for that provider, because the rancher provider is not an “official” one (maintained by a team at HashiCorp).
It seems like you already tried putting a required_providers block in your root module and saw it work. That would’ve been the answer I would’ve suggested, but I think you’re also unsure as to why the root module depends on that provider. Are you saying that the root module is depending on that provider even though it contains no direct mentions of that provider? If so, that seems like a bug.
No, I think you nailed it. Now that I understand it is per module it makes sense. I just means I have to update 20 different source repositories and od pull requests on them. Kind of a bummer. Would be cool if we could indicate the source modules required_providers applies to calling modules. (not sure I have the terminology correct there)
Thanks for helping me understand this.
Ahh, nevermind, I can generate the file using terragrunt.
Do you know if there are any plans to build Terragrunt functionality into TF? Kind of a pain to have to rely on a second app to avoid repetition.