Permissions errors trying to use /Azure/terraform/quickstart/301-service-fabric-apim

i255d · December 16, 2019, 8:31pm

I needed to convert this code to 12 first. Then I got permissions errors using my service principle account so I added Application Read and Write all applications, Read and write all users’ full profile. Now I am down to two errors:

azurerm_key_vault_certificate.client: Creating…

azurerm_key_vault_certificate.cluster: Creating…

Error: keyvault.BaseClient#CreateCertificate: Failure responding to request: StatusCode=403 – Original Error: autorest/azure: Service returned an error. Status=403 Code=“Forbidden” Message=“Access denied. Caller was not found on any access policy.\r\nCaller: appid=APRIVATENUMBER;oid=919444dd-4e81-4a64-baee-c1f918155614;numgroups=0;iss=https://sts.windows.net/7556c224-7d5d-4ed3-aa32-1e7bf7db3a65/\r\nVault: tfq-demo-tfquick-sbx-kv;location=westus2” InnerError={“code”:“AccessDenied”}

on keyvault.tf line 62, in resource “azurerm_key_vault_certificate” “cluster”:

62: resource “azurerm_key_vault_certificate” “cluster” {

Error: keyvault.BaseClient#CreateCertificate: Failure responding to request: StatusCode=403 – Original Error: autorest/azure: Service returned an error. Status=403 Code=“Forbidden” Message="Access denied. Caller was not found on any access policy.\r\nCaller: appid= APRIVATENUMBER;oid=919444dd-4e81-4a64-baee-c1f918155614;numgroups=0;iss=https://sts.windows.net/7556c224-7d5d-4ed3-aa32-1e7bf7db3a65/\r\nVault: tfq-demo-tfquick-sbx-kv;location=westus2" InnerError={“code”:“AccessDenied”}

on keyvault.tf line 120, in resource “azurerm_key_vault_certificate” “client”:

120: resource “azurerm_key_vault_certificate” “client” {

i255d · December 16, 2019, 8:33pm

Here is the terraform plan output after it errors out. 5 to add down form 31 or 32.

Here is the plan output.
I have converted this to 12.

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:

create

Terraform will perform the following actions:

azurerm_api_management_backend.sf will be created

resource “azurerm_api_management_backend” “sf” {
- api_management_name = “tfq-demo-tfquickstart-sandbox-apim”
- id = (known after apply)
- name = “service-fabric-backend”
- protocol = “http”
- resource_group_name = “demo-tfquickstart-sandbox-rg”
- resource_id = “https://tfq-demo-tfquickstart-sbx-sf.westus2.cloudapp.azure.com:19080”
- url = “fabric:/fake/service”
- service_fabric_cluster {
  - client_certificate_thumbprint = (known after apply)
  - management_endpoints = [
    - “https://tfq-demo-tfquickstart-sbx-sf.westus2.cloudapp.azure.com:19080”,
      ]
  - max_partition_resolution_retries = 3
  - server_certificate_thumbprints = (known after apply)
    }
    }

azurerm_key_vault_certificate.client will be created

resource “azurerm_key_vault_certificate” “client” {
- certificate_data = (known after apply)
- id = (known after apply)
- key_vault_id = “/subscriptions/ APRIVATENUMBER/resourceGroups/demo-tfquickstart-sandbox-rg/providers/Microsoft.KeyVault/vaults/tfq-demo-tfquick-sbx-kv”
- name = “service-fabric-client”
- secret_id = (known after apply)
- tags = (known after apply)
- thumbprint = (known after apply)
- vault_uri = (known after apply)
- version = (known after apply)
- certificate_policy {
  - issuer_parameters {
    - name = “Self”
      }
  - key_properties {
    - exportable = true
    - key_size = 2048
    - key_type = “RSA”
    - reuse_key = true
      }
  - lifetime_action {
    - action {
      - action_type = “AutoRenew”
        }
    - trigger {
      - days_before_expiry = 30
        }
        }
  - secret_properties {
    - content_type = “application/x-pkcs12”
      }
  - x509_certificate_properties {
    - extended_key_usage = [
      - “1.3.6.1.5.5.7.3.1”,
      - “1.3.6.1.5.5.7.3.2”,
        ]
    - key_usage = [
      - “cRLSign”,
      - “dataEncipherment”,
      - “digitalSignature”,
      - “keyAgreement”,
      - “keyCertSign”,
      - “keyEncipherment”,
        ]
    - subject = “CN=mtcdenver”
    - validity_in_months = 12
    - subject_alternative_names {
      - dns_names = [
        
        “sfdemosandbox.denvermtc.net”,
        ]
        }
        }
        }
        }

azurerm_key_vault_certificate.cluster will be created

resource “azurerm_key_vault_certificate” “cluster” {
- certificate_data = (known after apply)
- id = (known after apply)
- key_vault_id = “/subscriptions/APRIVATENUMBER/resourceGroups/demo-tfquickstart-sandbox-rg/providers/Microsoft.KeyVault/vaults/tfq-demo-tfquick-sbx-kv”
- name = “service-fabric-cluster”
- secret_id = (known after apply)
- tags = (known after apply)
- thumbprint = (known after apply)
- vault_uri = (known after apply)
- version = (known after apply)
- certificate_policy {
  - issuer_parameters {
    - name = “Self”
      }
  - key_properties {
    - exportable = true
    - key_size = 2048
    - key_type = “RSA”
    - reuse_key = true
      }
  - lifetime_action {
    - action {
      - action_type = “AutoRenew”
        }
    - trigger {
      - days_before_expiry = 30
        }
        }
  - secret_properties {
    - content_type = “application/x-pkcs12”
      }
  - x509_certificate_properties {
    - extended_key_usage = [
      - “1.3.6.1.5.5.7.3.1”,
      - “1.3.6.1.5.5.7.3.2”,
        ]
    - key_usage = [
      - “cRLSign”,
      - “dataEncipherment”,
      - “digitalSignature”,
      - “keyAgreement”,
      - “keyCertSign”,
      - “keyEncipherment”,
        ]
    - subject = “CN=mtcdenver”
    - validity_in_months = 12
    - subject_alternative_names {
      - dns_names = [
        
        “sfdemosandbox.denvermtc.net”,
        ]
        }
        }
        }
        }

azurerm_service_fabric_cluster.default will be created

resource “azurerm_service_fabric_cluster” “default” {
- add_on_features = [
  - “DnsService”,
    ]
- cluster_code_version = (known after apply)
- cluster_endpoint = (known after apply)
- id = (known after apply)
- location = “westus2”
- management_endpoint = “https://tfq-demo-tfquickstart-sbx-sf.westus2.cloudapp.azure.com:19080”
- name = “demo-tfquickstart-sf”
- reliability_level = “Bronze”
- resource_group_name = “demo-tfquickstart-sandbox-rg”
- tags = (known after apply)
- upgrade_mode = “Automatic”
- vm_image = “Windows”
- azure_active_directory {
  - client_application_id = “dc7b7144-891c-4363-b938-516aef045bcc”
  - cluster_application_id = “8fff89ef-5f58-4093-b521-ee78fde62ab9”
  - tenant_id = APRIVATENUMBER
  }
- certificate {
  - thumbprint = (known after apply)
  - thumbprint_secondary = (known after apply)
  - x509_store_name = “My”
    }
- client_certificate_thumbprint {
  - is_admin = true
  - thumbprint = (known after apply)
    }
- diagnostics_config {
  - blob_endpoint = “https://tfqdemotfquickstartsfsbx.blob.core.windows.net/”
  - protected_account_key_name = “StorageAccountKey1”
  - queue_endpoint = “https://tfqdemotfquickstartsfsbx.queue.core.windows.net/”
  - storage_account_name = “tfqdemotfquickstartsfsbx”
  - table_endpoint = “https://tfqdemotfquickstartsfsbx.table.core.windows.net/”
    }
- fabric_settings {
  - name = “Security”
  - parameters = {
    - “ClusterProtectionLevel” = “EncryptAndSign”
      }
      }
- fabric_settings {
  - name = “ClusterManager”
  - parameters = {
    - “EnableDefaultServicesUpgrade” = “True”
      }
      }
- node_type {
  - client_endpoint_port = 19000
  - durability_level = “Bronze”
  - http_endpoint_port = 19080
  - instance_count = 3
  - is_primary = true
  - name = “default”
  - application_ports {
    - end_port = 30000
    - start_port = 20000
      }
  - ephemeral_ports {
    - end_port = 65534
    - start_port = 49152
      }
      }
      }

azurerm_virtual_machine_scale_set.default will be created

resource “azurerm_virtual_machine_scale_set” “default” {
- automatic_os_upgrade = false
- id = (known after apply)
- license_type = (known after apply)
- location = “westus2”
- name = “demo-tfquickstart-vmss”
- overprovision = false
- resource_group_name = “demo-tfquickstart-sandbox-rg”
- single_placement_group = true
- tags = (known after apply)
- upgrade_policy_mode = “Automatic”
- boot_diagnostics {
  - enabled = true
  - storage_uri = “https://tfqdemotfquicksvmsssbx.blob.core.windows.net/”
    }
- extension {
  - name = “ServiceFabricNodeVmExt_vmDefault”
  - protected_settings = (sensitive value)
  - provision_after_extensions =
  - publisher = “Microsoft.Azure.ServiceFabric”
  - settings = (known after apply)
  - type = “ServiceFabricNode”
  - type_handler_version = “1.0”
    }
- identity {
  - identity_ids = (known after apply)
  - principal_id = (known after apply)
  - type = (known after apply)
    }
- network_profile {
  - ip_forwarding = false
  - name = “NetworkProfile”
  - primary = true
  - ip_configuration {
    - application_gateway_backend_address_pool_ids =
    - application_security_group_ids =
    - load_balancer_backend_address_pool_ids = [
      - “/subscriptions/ APRIVATENUMBER/resourceGroups/demo-tfquickstart-sandbox-rg/providers/Microsoft.Network/loadBalancers/demo-tfquickstart-lb/backendAddressPools/ServiceFabricAddressPool”,
        ]
    - load_balancer_inbound_nat_rules_ids = [
      - “/subscriptions/ APRIVATENUMBER/resourceGroups/demo-tfquickstart-sandbox-rg/providers/Microsoft.Network/loadBalancers/demo-tfquickstart-lb/inboundNatPools/demo-tfquickstart-nat-pool”,
        ]
    - name = “IPConfiguration”
    - primary = true
    - subnet_id = “/subscriptions/ APRIVATENUMBER/resourceGroups/demo-tfquickstart-sandbox-rg/providers/Microsoft.Network/virtualNetworks/demo-tfquickstart-vnet/subnets/demo-tfquickstart-sf-subnet”
      }
      }
- os_profile {
  - admin_password = (sensitive value)
  - admin_username = “hdiadmin”
  - computer_name_prefix = “sfvm”
    }
- os_profile_linux_config {
  - disable_password_authentication = (known after apply)
  - ssh_keys {
    - key_data = (known after apply)
    - path = (known after apply)
      }
      }
- os_profile_secrets {
  - source_vault_id = “/subscriptions/ APRIVATENUMBER/resourceGroups/demo-tfquickstart-sandbox-rg/providers/Microsoft.KeyVault/vaults/tfq-demo-tfquick-sbx-kv”
  - vault_certificates {
    - certificate_store = “My”
    - certificate_url = (known after apply)
      }
      }
- os_profile_windows_config {
  - enable_automatic_upgrades = true
  - provision_vm_agent = true
    }
- sku {
  - capacity = 3
  - name = “Standard_D1_v2”
  - tier = “Standard”
    }
- storage_profile_data_disk {
  - caching = “ReadWrite”
  - create_option = “Empty”
  - disk_size_gb = 10
  - lun = 0
  - managed_disk_type = (known after apply)
    }
- storage_profile_image_reference {
  - offer = “WindowsServer”
  - publisher = “MicrosoftWindowsServer”
  - sku = “2019-Datacenter-with-Containers”
  - version = “latest”
    }
- storage_profile_os_disk {
  - caching = “ReadWrite”
  - create_option = “FromImage”
  - managed_disk_type = “Standard_LRS”
  - vhd_containers =
    }
    }

Plan: 5 to add, 0 to change, 0 to destroy.

i255d · December 16, 2019, 8:43pm

Here is the code from Keyvault.tf:

resource “azurerm_key_vault” “cluster” {

name = “{var.dns_prefix}-{substr(var.name, 0, 12)}-${var.environment_short}-kv”

location = azurerm_resource_group.default.location

resource_group_name = azurerm_resource_group.default.name

tenant_id = data.azurerm_client_config.current.tenant_id

enabled_for_deployment = true

enabled_for_disk_encryption = true

enabled_for_template_deployment = true

sku_name = “standard”

access_policy {

tenant_id = data.azurerm_subscription.current.tenant_id

object_id = var.client_object_id

certificate_permissions = [

  "create",

  "delete",

  "deleteissuers",

  "get",

  "getissuers",

  "import",

  "list",

  "listissuers",

  "managecontacts",

  "manageissuers",

  "setissuers",

  "update",

]

key_permissions = [

  "backup",

  "create",

  "decrypt",

  "delete",

  "encrypt",

  "get",

  "import",

  "list",

  "purge",

  "recover",

  "restore",

  "sign",

  "unwrapKey",

  "update",

  "verify",

  "wrapKey",

]

secret_permissions = [

  "backup",

  "delete",

  "get",

  "list",

  "purge",

  "recover",

  "restore",

  "set",

]

}

resource “azurerm_key_vault_certificate” “cluster” {

name = “service-fabric-cluster”

key_vault_id = azurerm_key_vault.cluster.id

certificate_policy {

issuer_parameters {

  name = "Self"

}

key_properties {

  exportable = true

  key_size   = 2048

  key_type   = "RSA"

  reuse_key  = true

}

lifetime_action {

  action {

    action_type = "AutoRenew"

  }

  trigger {

    days_before_expiry = 30

  }

}

secret_properties {

  content_type = "application/x-pkcs12"

}

x509_certificate_properties {

  # Server Authentication = 1.3.6.1.5.5.7.3.1

  # Client Authentication = 1.3.6.1.5.5.7.3.2

  #extended_key_usage = ["1.3.6.1.5.5.7.3.1"]

  extended_key_usage = [

    "1.3.6.1.5.5.7.3.1", # Server Authentication

    "1.3.6.1.5.5.7.3.2", # Client Authentication

  ]

  key_usage = [

    "cRLSign",

    "dataEncipherment",

    "digitalSignature",

    "keyAgreement",

    "keyCertSign",

    "keyEncipherment",

  ]

  subject_alternative_names {

    dns_names = ["sfdemosandbox.denvermtc.net"]

  }

  subject            = "CN=mtcdenver"

  validity_in_months = 12

}

}

resource “azurerm_key_vault_certificate” “client” {

name = “service-fabric-client”

key_vault_id = azurerm_key_vault.cluster.id

certificate_policy {

issuer_parameters {

  name = "Self"

}

key_properties {

  exportable = true

  key_size   = 2048

  key_type   = "RSA"

  reuse_key  = true

}

lifetime_action {

  action {

    action_type = "AutoRenew"

  }

  trigger {

    days_before_expiry = 30

  }

}

secret_properties {

  content_type = "application/x-pkcs12"

}

x509_certificate_properties {

  # Server Authentication = 1.3.6.1.5.5.7.3.1

  # Client Authentication = 1.3.6.1.5.5.7.3.2

  #extended_key_usage = ["1.3.6.1.5.5.7.3.1"]

  extended_key_usage = [

    "1.3.6.1.5.5.7.3.1", # Server Authentication

    "1.3.6.1.5.5.7.3.2", # Client Authentication

  ]

  key_usage = [

    "cRLSign",

    "dataEncipherment",

    "digitalSignature",

    "keyAgreement",

    "keyCertSign",

    "keyEncipherment",

  ]

  subject_alternative_names {

    dns_names = ["sfdemosandbox.denvermtc.net"]

  }

  subject            = "CN=mtcdenver"

  validity_in_months = 12

}

}

i255d · December 18, 2019, 4:35pm

It turned out that two variables needed to be modified, I don’t think it is mentioned in the readme.md.

variable “client_object_id” {
default = “ed267c3b-ce3e-418e-8734-072bff32728d”
}

variable “application_id” {
default = “0938d8bc-3351-4bcc-ddb5-113c2218ff0d”
}
These values need to be obtained from your service principal account.

Topic		Replies	Views
Adding a Keyvault Secret after Keyvault Creation fails with 403 Forbidden Access Error Terraform azure	0	2322	May 31, 2022
AZURERM - Error with KEYVAULT Creation while creating ML Workspace Azure	0	903	September 8, 2023
Terraform provider registration failure (Azure) Azure	3	8069	June 15, 2023
Azurerm_key_vault_secret --> InnerError={"code":"ForbiddenByRbac"} Azure vault , azure	1	1163	July 16, 2023
Create a management group Terraform	4	1694	December 4, 2020