Unable to create windows image in Azure using packer, getting "forbidden" error

We would like to have an Azure DevOps pipeline to create 2 custom images, for linux and for windows. The linux image is being created successfully, but we struggle to get the windows image to work using WinRM. We use packer 1.11.

As soon as the temp VM gets deployed and WinRM tries to connect we keep getting the following error:

2025/01/29 09:12:06 packer-plugin-azure_v2.2.1_x5.0_linux_amd64 plugin: 2025/01/29 09:12:06 Waiting for WinRM, up to timeout: 10m0s

==> azure-arm.MPS_Builder: Waiting for WinRM to become available…

2025/01/29 09:12:06 packer-plugin-azure_v2.2.1_x5.0_linux_amd64 plugin: 2025/01/29 09:12:06 [INFO] Attempting WinRM connection…

2025/01/29 09:12:06 packer-plugin-azure_v2.2.1_x5.0_linux_amd64 plugin: 2025/01/29 09:12:06 [DEBUG] connecting to remote shell using WinRM

2025/01/29 09:12:06 packer-plugin-azure_v2.2.1_x5.0_linux_amd64 plugin: 2025/01/29 09:12:06 [ERROR] connection error: unknown error Post "https://172.27.57.22:5986/wsman": Forbidden

2025/01/29 09:12:06 packer-plugin-azure_v2.2.1_x5.0_linux_amd64 plugin: 2025/01/29 09:12:06 [ERROR] WinRM connection err: unknown error Post "https://172.27.57.22:5986/wsman": Forbidden

Connectivity on tcp/5986 works fine, it seems related to configuration and/or authentication.

Any ideas what to check? Thank you.