I have a Job Pool in Databricks and want to create a linked service using that. I tried using azurerm_data_factory_linked_service_azure_databricks with instance_pool. But it doesn’t provide an option to include the following:
- Cluster Spark environment variables, Databricks init script, Cluster Log Dbfs Destination
- Python Version
How to create this linked service using terraform?
Few more interesting points:
- I’ve mentioned min and max workers but it still creates with Worker options as Fixed instead of Autoscaling.