Well, thanks for your help !
Yes you absolutely got my issue. So I tried to reproduce based on your idea.
It seems ok to create the private zone and set private_dns_zone_id within azurerm_kubernetes_cluster.
Now when I try to create the dns record and then provision the aks cluster I get an error:
Failure sending request: StatusCode=0 – Original Error: Code=“BadRequest”
Message="DNS record for FQDN subdomain ‘aks-private’ already exists on private dns zone
So it doesn’t seems possible create this dns_A_record resource before aks cluster I believe.
As a workaround I tried setting up another entry with different prefix but the issue now would be related to the SSL certificate not issued for that specific subdomain.
I’ll try to sum up the situation as best as I can.
What I’m currently trying to achieve is to setup a private aks cluster within a hub-spoke topology where I have vpn gateway, application gateway, private dns zone and hub vnet on 1 subscription and then on a different subscription, the spoke vnet and aks cluster working with aad-pod-identity, agic and external dns.
The main drawback is that I have limited number of IP available to expose resources from azure through vpn. I can’t use aks kubenet plugin because I work across multiple subscription and maintaining route tables doesn’t seems possible across subscriptions.
So I came up with the idea of having 1 subnet for aks, from which pods will get their ips, and another subnet to expose resources through vpn.
Pods get let’s say “private ips” and accessed through the application gateway. Vnets are peered so connectivity between appgw and pods is ok.
Now since the Aks cluster is given a specific subnet, the private endpoint associated to the kube apiserver is automatically within this subnet. So if I want to interact with the kube api-server privately I need a private endpoint linked to the kube api-server Private Link.
And then comes the best part, dns resolution and SSL.
As I mentioned when you set a dns_prefix parameter. This prefix is used to generate the certificate, if I want to interact with that api-server, the only option that I see is to update the dns alias with the IP of the new private endpoint.
At the current time everything seems to work. The only “manual and hacky thing” is this managed DNS record.
That’s quite a long explanation but you asked
Hopefully that makes sense.
I’m open for suggestions, better ideas …