Error "Could not find ADLS Gen2 Token" when creating a credential passthrough mount via an Azure Dev Ops pipeline using an SP

Terraform Providers
1

I’ve used Terraform to create two Azure Databricks workspaces each with a single high concurrency cluster. I’ve set up Azure Dev Ops pipelines using YAML to deploy the Terraform code and the pipelines use a Service Connection to connect to Azure. The Service Connection is an Azure Resource Manager type using service principal authentication. The service principal has Owner RBAC permissions on the Azure subscription and is in the admin group in the Databricks workspaces.

I’m now trying to use the databricks_mount resource to create mounts to a Gen2 Azure Data Lake Store that was also created in the same Terraform code base using the same service principal. The service principal has Storage Blob Data Contributor RBAC permissions on the data lake storage account.

For one workspace I’m using a service principal credential to mount to the lake, as described here and that is all working as expected.

However, for the second workspace I need to mount using credential pass though. I’ve used the example described here and have managed to get it working when applying the Terraform code locally (where I’ve authenticated using az login via Azure CLI). However, when trying to deploy the same Terraform code via the YAML pipeline I receive the error…

Error: Could not find ADLS Gen2 Token

My provider block looks as follows…

provider "databricks" {
  host  = azurerm_databricks_workspace.databricks_read.workspace_url
  alias = "read"
}

… but I’ve tried a few other formats as described here but always end up with the same error.

Has anyone out there managed to get this working using a service principal to deploy to the code?

2

@SPea-1 it’s a known issue for service principals and AAD passthrough and it doesn’t work… please try using either client secret based mount.