Changing subscription with new service princiaple

Terraform Providers Azure
1

i have a new directory with a simple main.tf:

provider "azurerm" {
  version = "=2.5.0"
  ARM_CLIENT_ID="removed"
  ARM_CLIENT_SECRET="removed"
  ARM_SUBSCRIPTION_ID="removed"
  ARM_TENANT_ID="removed"
}

terraform {
  backend "azurerm" {
resource_group_name  = "westus_5_i_rnd_rg"
storage_account_name = "westusdsorndstor"
container_name       = "vnets"
key                  = "5.infra.terraform.tfstate"
  }
}

# Create a resource group

resource "azurerm_resource_group" "cluster" {
  name     = "${var.azlocname}_${var.teamid}_${var.shortuse}_${var.enviro}_rg"
  location = var.location
  tags = {
Environment = var.enviro
Owner = var.team
Team = var.team
  }
}

and there is a basic variables.tf in this directory as well:

variable "networkSpace" {
  type = string
  description = "cidr address space for vnet"  
  default = "removed"
}
variable "azlocname" {
  type = string
  description = "cidr address space for vnet"  
  default = "westus"
}
variable "teamid" {
  type = string
  description = "cidr address space for vnet"  
  default = "5"
}
variable "shortuse" {
  type = string
  description = "cidr address space for vnet"  
  default = "w"
}
variable "enviro" {
  type = string
  description = "cidr address space for vnet"  
  default = "rnd"
}

i had created a new service principle
following these steps

this new service principle only has access to a different subscription

but when i run the terraform init

/terraform$ terraform init

Initializing the backend…

Error: Failed to get existing workspaces: Error retrieving keys for Storage Account “dsoterraformstate”: storage.Acc
ountsClient#ListKeys: Failure responding to request: StatusCode=404 – Original Error: autorest/azure: Service retur
ned an error. Status=404 Code=“ResourceGroupNotFound” Message=“Resource group ‘RG-AZDSO-WUS’ could not be found.”

but this storage account and resource group referenced above were in a different terraform project with a different service principle - so its no surprise it is getting a 404, but why is it not picking up and looking for the storage account and resource group names from the terraform{backend{}} in this main.tf?

did i miss something?