How to create Azure Databricks Linked Service with existing pool Id and additional Cluster Settings using Terraform

I have a Job Pool in Databricks and want to create a linked service using that. I tried using azurerm_data_factory_linked_service_azure_databricks with instance_pool. But it doesn’t provide an option to include the following:

  • Cluster Spark environment variables, Databricks init script, Cluster Log Dbfs Destination
  • Python Version

How to create this linked service using terraform?

Few more interesting points:

  • I’ve mentioned min and max workers but it still creates with Worker options as Fixed instead of Autoscaling.