Strange problem setting up a storage account

[paul.boardman]

I’m using terraform cloud to create a simple storage account (SA) in our subscription.

Create storage account for network watcher logging

resource “azurerm_storage_account” “sa-net-wat-op” {
name = “nameremoved”
resource_group_name = azurerm_resource_group.ops-netwatch-rg.name
location = var.region_ne
account_tier = “Standard”
account_replication_type = “LRS”
}

I have several Workspaces set up and use a Cloud Agent running on an Azure Container as I have a locked down environment.
There is a storage Registry module but that does not have any variables set up and my .tf file is not referencing it at all.

The SA creates but the terraform side does not complete. It times out and says the creation fails.

And this on the Container Agent

I can create the same SA in the subscription manually. And as I say, the SA gets set up anyway but any subsequent runs will fail as TFC keeps hanging when checking this as part of its run.
I can create other resources without any problems in this Workspace, so not an Agent issue or permission issue.
I can use other Workspaces to create this SA in the same subscription without any problems, so again not RBAC permission.
I can create this SA in other subscriptions as well.

I’m leaning towards the Workspace but I cant see how or what might be happening to cause this and my initial google searches has not proved that fruitful.

[ExtelligenceIT]

I don’t have an answer for you, but I have seen similar ‘issues’ when dealing with storage private endpoints and self-hosted agents with Azure DevOps related to traffic routing and DNS resolution - even if the SA in question is not set up with Private endpoints (Storage account networking in some circumstances can be somewhat strange under the covers). I have a few suggestions that may help you diagnose the issue:

• If you create the storage account via the Portal, what does the FQDN of the storage account endpoints resolve to from your containerised cloud agent? Does it try and retrieve the details from a private-link subdomain, or do you get the ‘public’ IP of the storage account?

• Can your cloud-agent, when it is running, actually route to/reach the IP address that is returned for the storage account that you manually provision?

Unfortunately the AzureRM provider does not exclusively use the ARM API, via the resource manager control-plane, to manage Storage accounts. It needs to be able to contact the storage account directly (the ‘data-plane’ ). Especially when managing / provisioning the sub-resources such as containers, queues, etc.

Not being able to access the storage account for some reason (DNS resolution/routing/firewalling) from the agent would meant that the initial creation of the storage account (which happens via the Resource Manager API) is successful, but when the later steps that the provider attempts to configure the sub-resources start, they are unable to access the storage account directly and fail.

Good luck - I would be interested when you find the reason and the resolution.

Happy Terraforming

[paul.boardman]

Thanks for responding. I’ll take a look at this.

However, because I can create the storage account via another workspace using the same Agent, I’m not so sure its networking.

But its something to look into though and gives me something to check.