We have discovered that it is not supported in azurerm to deploy both a subscription and resources within the subscription in a single module. This is because the subscription ID must be known (passed to the providers block) during terraform init.
This behavior is a bit inconsistent and we thought we had solved the issues, but we’ve learned that in 4.x the inconsistencies will be corrected and this will become required in all cases.
Reference: https://github.com/hashicorp/terraform-provider-azurerm/issues/17979
" We don’t recommend creating new a subscription and adding resources to that subscription in the same module or apply operation. The reason it works with Azure CLI authentication is that the subscription ID is sourced at plan time - which means the plan is working against the wrong subscription - and because the correct subscription is yet to be created, the plan happens to be somewhat correct. But this is just coincidence. When authenticating as a service principal without specifying an existent subscription ID, the plan fails because the subscription ID cannot be determined. This is the expected behaviour.
The coincidental behaviour of planning with the wrong subscription ID when using CLI authentication, is likely to change in version 4.0 of the provider, and as such as don’t recommend relying on this undocumented behaviour."
So we are left trying to find the best pattern to stitch together one module to manage subscriptions, and another to manage the resources inside of them. We’ve come up with a few ideas, but are wondering what others have found as a good pattern?
- Using az_api instead of azurerm - this is how Microsoft worked around this issue. We are using sub-modules which are referenced hundreds of times built using azure_rm, and I don’t always trust az_api to be idempotent or make changes as expected as reliably as azure_rm so I would rather avoid switching to az_api if possible.
- Terragrunt - Split into two modules, and use its feature of dynamically generating the providers block to take the output of the subscription module and update the subscription ID in the providers block for the other modules. It seems Terragrunt can also call multiple modules in a single operation and simplifies that challenge.
- Cross state file composition using remote state. The subscription module’s state file will contain the subscription ID, so look this up using remote_state and set in the providers block on the module which creates resources inside the subscription. This may or may not work as technically I think it’s still an unknown value at plan time since it needs to retrieve the data from remote state?
- Use something other than Terraform to deploy the subscriptions themselves, and have that process automatically update the providers block in our repos to include the subscription ID & add import blocks to import the subscription into a a single Terraform module.
- How to create Azure Subscription and deploy into it in same stream - I think this might be relying on the “inconsistent behavior” which might break in azure_rm 4.x? We also found when authenticating as a user account we don’t have an issue (as the poster suggested), but it didn’t work as a service principal in CICD. The comments on the GitHub issue indicate this will stop working in azure_rm 4.x.