I have read few posts suggesting to wait after each run as a work around. Is there any way to put sleep in every run of for_each loop. Or, would be grateful for any better suggestions on this issue.
I’m not sure since you didn’t share the entire error message, but I think this is an error message where Terraform is reporting incorrect behavior in the part of the provider, rather than incorrect input configuration, in which case the solution would be to fix the provider to properly implement the provider protocol, so that this error message cannot appear.
If you aren’t sure how to interpret the error message then it would help if you could share the full error message and then I’d be happy to try to give a more specific suggestion.
Thank you so much for helping. This is the only error message I’m getting
Error: Provider produced inconsistent result after apply
When applying changes to
module.db-tst-1.custom_azuread_member.db-writer-users["xxx@company.com"],
provider "provider[\"pqrs.company.com/company/custom\"]"
produced an unexpected new value: Root resource was present, but now absent.
I’m sorry I didn’t see the notification about your response before.
The situation this message is reporting is that during the planning step the provider returned a non-null object, representing intent to create or update the object, but then when asked to apply that change the provider returned a null object, representing that the object doesn’t exist at all.
Without access to the provider’s source code I can’t be sure about why that would be, but one possible reason it could occur if you are using SDKv2 to implement the provider is if you didn’t call d.SetId with a non-empty ID string during the Create function. Having a non-empty ID is how SDKv2 represents an object existing – and, conversely, having an empty string ID represents that the object is missing, e.g. in the Read function. Therefore you must always set a non-empty ID before returning in Create or else the SDK will return null to indicate that the object doesn’t exist, and then Terraform Core in turn will return this error because the earlier plan said that an object would be created and therefore a missing (null) object is not a valid outcome.
the custom provider creates the resource properly in an initial apply
the resource gets deleted outside of Terraform
on the next apply, the custom provider resource Read is called, and it returns ID “” because the resource doesn’t exist
Terraform errors
2022-04-14T12:09:33.107-0400 [WARN] Provider "example.com/gcp/custom-gcp-cluster" produced an unexpected new value for custom-gcp-cluster_cert.jenkins_ca during refresh.
- Root resource was present, but now absent
Terraform does not call the custom resource Create. Instead it continues, and then downstream dependent resource apply fails.
This particular message is an internal warning to help us with debugging rather than an end-user-oriented message. Arguably it should really be an [INFO] rather than a [WARN] because the whole point of “refresh” is to detect changes which would then be reported by this message.
But with that confusing log level aside, you don’t need to worry about that particular warning in this case: if something has been deleted outside of Terraform then it’s correct behavior for the provider to detect it during refresh and report it as absent.
The cause for concern here was that there was drift in the first place, not that the provider detected it. This warning is there basically just so that when someone asks a question or reports a bug that Terraform is proposing something unexpected in the plan diff we can use the trace logs to see if it was a real drift detected by the provider or if there was some sort of bug inside Terraform.
If you did also see an actual error – something printed to the terminal even when the debug logs are disabled, with Error: in front of it, then that would be unexpected. If that’s true, it would help if you could share that error message, rather than the hidden warning from the logs.
Are there two unrelated “Root resource was present, but now absent” messages then - one in the log, and another that shows up as an error (not in the log)?
I have the case where it is only in the log. After your reply above I started to understand the original post here was similar, but not my case.
I was confused because it sounded like my case occurred as a bug in Create, which wasn’t getting called at all. Now I have the understanding these are separate scenarios with the same error text.
I tracked down my issue. There was a lack of dependency creating a timing problem. Terraform was destroying, then creating the “first” resource, and a “second” resource dependent on the first got processed at the same time, and its Read had returned id “” because its dependency was missing, causing that warning.
I added depends_on and haven’t seen the issue since.
Oh, sorry! I too got confused here because we were discussing a few different things in this topic and I got them a bit blended together in my head.
The important difference between the “Provider produced inconsistent result after apply” error and the “unexpected new value for … during refresh” warning is that a provider producing a final result which disagrees with its earlier plan is always indicative of a bug in the provider, whereas an unexpected new value during refresh is something changed in the remote system and so it is correct behavior for a provider to detect and report that, since making a best effort to detect that sort of drift is what the refresh phase is for. It’s included in the logs despite that because sometimes users ask “why did Terraform propose this change when I didn’t change my configuration?” and so we can use the logs to figure out that something changed in the remote system, rather than it being a Terraform bug.
A general guideline is that anything you can only see with TF_LOG=... set is intended primarily to help Terraform developers to diagnose bug reports without access to the reporter’s computer, and that anything which an end user should be concerned about ought to always be in the main UI output that appears by default.
The “Provider produced inconsistent result after apply” is a funny edge case of that rule, because really the message is directed at the developer of the provider rather than at the user, but the provider’s misbehavior in that case is significant enough that Terraform must not proceed – to do so would cause downstream operations to receive different inputs than they were given during planning, which could cause the final result to significantly diverge from what was proposed in the plan. So in that case as a compromise we have Terraform halt with an error which basically instructs the user to pass the error on to a provider developer to debug it further, because the problem is typically outside of the user’s direct control.
Error: Provider produced inconsistent result after apply
│
│ When applying changes to azurerm_synapse_linked_service.des-linked-service["test"], provider "provider[\"registry.terraform.io/hashicorp/azurerm\"]" produced
│ an unexpected new value: Root resource was present, but now absent.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
As the error message says, this is a bug in the provider you are using and so I would suggest that you open an issue in the provider’s repository to report it.
I think the following existing issue might be describing the same problem you have seen here, in which case I would suggest adding your reproduction case to this issue rather than opening a new one: