Hello all,
we are using Terraform v0.12.29 to manage a group of VMs on Azure using azurerm 2.10.0.
Whenever we run “apply”, Terraform wants to recreate the VMs:
It seems to me, that this is related to the public and private IPs, since these are marked with “-”.
The configuration of the public IP and network interface looks like this:
resource "azurerm_public_ip" "pip" {
count = var.public_ip ? 1 : 0
name = "${local.name_prefix}-pip"
location = var.location
resource_group_name = var.resource_group_name
#allocation_method = "Dynamic"
allocation_method = var.public_ip_dynamic ? "Dynamic" : "Static"
sku = "Basic"
tags = var.common_tags
}
resource "azurerm_network_interface" "nic" {
name = "${local.name_prefix}-nic"
location = var.location
resource_group_name = var.resource_group_name
count = var.nic == [1] ? 1:0
ip_configuration {
name = "nic-config"
subnet_id = var.subnet_id
private_ip_address_allocation = "Dynamic"
public_ip_address_id = var.public_ip ? azurerm_public_ip.pip[0].id : null
}
tags = var.common_tags
}
resource "azurerm_linux_virtual_machine" "vm" {
name = "${local.name_prefix}-vm"
location = var.location
resource_group_name = var.resource_group_name
network_interface_ids = var.nic == [1] ? [azurerm_network_interface.nic[0].id] : var.nic
size = var.vm_size
admin_username = var.admin_username
computer_name = "${local.name_prefix}-host"
custom_data = var.custom_data
}
Anyone have an idea what we may be doing wrong?
Best regards and thanks in advance!
Hi @SBiesinger ,
I’m not very familiar with Azure in particular so I’m not sure exactly what might cause this, but from reading your screenshot of the plan output I notice that the only attribute showing a change that corresponds with an argument in your configuration is custom_data
; the others are all attributes the provider populates itself after the apply operation succeeds.
Unfortunately, the provider has marked custom_data
as being sensitive and so we can’t see exactly what has changed. However, a typical reason for an argument to show changes even though nothing has changed in the configuration is if the remote API is normalizing the data in some way that the provider isn’t accounting for.
It’s generally a provider’s responsibility to decide for each change in the configuration whether it represents normalization (a different way of writing the same thing), or a meaningful change (the new value means something different), but sometimes the remote API has normalization behavior that the provider doesn’t know about yet and so it can be mis-classified as a meaningful change.
From referring to the Azure provider documentation I see that custom_data
is commonly a multi-line string. Therefore I wonder if the Azure API is normalizing the string you’ve given in a way that the Azure provider doesn’t expect, such as converting the newline characters from the Unix standard to the Windows standard or vice-versa.
The configuration snippet you shared doesn’t show how var.custom_data
is populated so I can’t be sure if this is the correct explanation, but I hope it gives you a useful lead for further debugging. If you have access to the raw Terraform state snapshots for this configuration you could also look there to see what value is stored for custom_data
and see if it exactly matches your var.custom_data
value. The state snapshots are in JSON format, so you should hopefully be able to just search in there for custom_data
and find the relevant attribute.
Hi @apparentlymart ,
thank you very much, custom_data was in fact causing the issue.
For anyone else coming across this problem, it seems to be related to these issues:
opened 03:25PM - 04 May 20 UTC
closed 10:20AM - 19 May 20 UTC
upstream/terraform
service/virtual-machine
<!---
Please note the following potential times when an issue might be in Terra… form core:
* [Configuration Language](https://www.terraform.io/docs/configuration/index.html) or resource ordering issues
* [State](https://www.terraform.io/docs/state/index.html) and [State Backend](https://www.terraform.io/docs/backends/index.html) issues
* [Provisioner](https://www.terraform.io/docs/provisioners/index.html) issues
* [Registry](https://registry.terraform.io/) issues
* Spans resources across multiple providers
If you are running into one of these scenarios, we recommend opening an issue in the [Terraform core repository](https://github.com/hashicorp/terraform/) instead.
--->
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
### Terraform (and AzureRM Provider) Version
```
Terraform v0.12.24
+ provider.azurerm v2.8.0
+ provider.external v1.2.0
+ provider.null v2.1.2
+ provider.random v2.2.1
+ provider.template v2.1.2
```
### Affected Resource(s)
* `azurerm_linux_virtual_machine`
### Terraform Configuration Files
```hcl
resource "azurerm_linux_virtual_machine" "vm" {
name = var.name
resource_group_name = var.resource_group_name
location = var.location
size = var.size
availability_set_id = var.availability_set_id
proximity_placement_group_id = var.proximity_placement_group_id
network_interface_ids = [
azurerm_network_interface.nic_0.id,
azurerm_network_interface.nic_1.id,
azurerm_network_interface.nic_2.id
]
plan {
name = "bundle2"
publisher = "paloaltonetworks"
product = "vmseries1"
}
source_image_reference {
publisher = "paloaltonetworks"
offer = "vmseries1"
sku = "bundle2"
version = "9.0.4"
}
os_disk {
name = "${var.name}-os"
caching = "ReadWrite"
storage_account_type = "StandardSSD_LRS"
}
computer_name = var.name
admin_username = var.admin_username
admin_password = var.admin_password
disable_password_authentication = false
allow_extension_operations = true
admin_ssh_key {
username = var.admin_username
public_key = var.public_key
}
custom_data = base64encode("storage-account=${data.azurerm_storage_account.bootstrap_storage_account.name},access-key=${data.azurerm_storage_account.bootstrap_storage_account.primary_access_key},file-share=${azurerm_storage_share.bootstrap_storage_share.name},share-directory=files")
}
```
### Expected Behavior
Terraform should check if `custom_data` base64 value was changed and mark the VM for redeployment only if it changed.
### Actual Behavior
Every `terraform apply`, the VM is marked for recreation even if the base64 value of `custom_data` is the same every time. When removing `custom_data` line, the VM is not recreated.
### Steps to Reproduce
1. `terraform apply`
### Important Factoids
### References
<!---
Information about referencing Github Issues: https://help.github.com/articles/basic-writing-and-formatting-syntax/#referencing-issues-and-pull-requests
Are there any other GitHub issues (open or closed) or pull requests that should be linked here? Such as vendor documentation?
--->
Similar issue but this solution is not working although it was supposed to be fixed in version 2.0:
* [#1013](https://github.com/terraform-providers/terraform-provider-azurerm/issues/1013)
opened 08:00PM - 04 Jan 18 UTC
closed 05:20PM - 24 Jun 20 UTC
enhancement
core
proposal
# Background Info
Back in #6598 we introduced the idea of _data sources_, all… owing us to model reading data from external sources as a first-class concept. This has generally been a successful addition, with some great new patterns emerging around it.
However, the current lifecycle for data sources creates some minor problems. As currently implemented, data sources are read during the "refresh" phase that runs prior to creating a plan, except in two situations:
* If any of the configuration arguments in the corresponding `data` block have `<computed>` values.
* If `depends_on` is non-empty for the data resource.
In both of the above cases, the read action is deferred until the "apply" phase, which in turn causes all of the result attributes to appear as `<computed>` in the plan.
Unfortunately both of the above situations are problematic today, as a consequence of data sources being processed during "refresh". These problems are described in more detail in the following sections.
## When Data Resource Arguments change
Because data resources are read during the "refresh" phase, references to attributes of resources are resolved from their value in state rather than their value in the resulting diff. This results in a "change lag" , where certain changes to configuration require two runs of `terraform apply` to fully take effect. The first run reads the data source with the _old_ resource values and _then_ updates the resource, while the second run reads the data source using the _new_ resource values, possibly causing further cascading changes to other resources.
This is particularly tricky for situations where a resource has custom diff logic (via the mechanism added in #14887) that detects and reports changes to `Computed` attributes that are side-effects of the requested changes, since this can result in additional value changes that are not reflected in the data source read.
The most problematic case is when an attribute is marked as `<computed>` during an update: this _should_ cause any dependent data resource to be deferred until apply time, but instead the old value is used to do the read and the computed value resulting from the change is not detected at all.
## Trouble with `depends_on`
The current behavior for `depends_on` for data resources is essentially useless, since it always results in a "perma-diff". The reason for this is that `depends_on` doesn't give Terraform enough information to know what aspect of the dependency is important, and so it must conservatively _always_ defer the read until the "apply" phase to preserve the guarantee that it happens after the resource changes are finalized.
Ideally we'd like the data resource read to be deferred until apply time only if there are pending changes to a dependent resource, but that is not currently possible because we process data resources during the "refresh" phase where resource diffs have not yet been created, and thus we cannot determine if a change is pending.
# Proposed Change
The above issues can be addressed by moving data source processing into the "plan" phase.
This was seen as undesirable during the original data source design because it would cause the "plan" phase to, for the first time, require reaching out to external endpoints. However, we have since made that compromise in order to improve the robustness of planning in #14887. In this new context, reading from data sources during plan is consistent with our goal of having Terraform make whatever requests it needs to make in order to produce an accurate, comprehensive plan. #15895 proposes some adjustments to the behavior of `terraform validate` so that it can be used as the "offline static check" command, allowing `terraform plan` to be more complex and require valid credentials for remote APIs even when the implicit refresh is disabled.
Including data source reads in the _plan_ graph means that Terraform will produce diffs for resources _before_ attempting to read data sources that depend on them, which addresses all of the problems described above: the data source arguments can be interpolated from the new values as defined in the diff, rather than the old values present in state.
In particular, the diff can be consulted in order to decide whether a data resource must be deferred until the "apply" phase, allowing any new `<computed>` values to be considered, and allowing `depends_on` to defer only if there is a non-empty diff for the referenced resource(s).
## Effect on the Managed Resource Lifecycle
This change does not greatly affect the lifecycle for _managed_ resources, but it _does_ restore the original property that the "refresh" phase is a state-only operation with the exception of provider configuration.
An interesting implication of this is that it is in principle safe to disregard any inter-resource dependencies for refresh purposes and instead construct a flatter graph where each resource depends only on its provider. This in turn can permit greater parallelism in read calls, and more opportunities for API request consolidation once #7388 is addressed.
## Effect on `terraform refresh`
Since data sources are currently processed in the "refresh" phase, the `terraform refresh` command currently updates them. This can be useful in situations where a root module output depends on a data source attribute and its state is being consumed with `terraform_remote_state`.
Moving data source reads to the plan phase will mean that `terraform refresh` will no longer update them. The change proposed in #15419 can partially mitigate this by making data resource updates -- and corresponding output updates -- an explicit part of the normal `terraform apply` flow, which is a more desirable outcome for the reasons described in that issue.
To retain the ability to _only_ update data resources, without applying other changes, we can add a new argument to `terraform plan` (and, by extension, `terraform apply` with no explicit plan file argument) `-read-only`, which produces a reduced plan graph that _only_ includes the data resources.
Bringing data source refresh into the main plan+apply workflow is superior to the current `terraform refresh` approach because it allows the user to evaluate and approve the resulting changes to outputs, rather than just blindly accepting these updates and potentially disrupting downstream remote state consumers.
For users that still want to accept data source updates without a confirmation step, the command line `terraform apply -read-only -auto-approve` would be equivalent to the current `terraform refresh` behavior.
The accepted workaround until the release of Terraform v0.13 seems to be to ignore changes to custom_data as per so:
lifecycle {
ignore_changes = [custom_data]
}
once again many thanks to @apparentlymart for the quick an helpful reply!