Create a cloud resource according its availability of GPU resources in a region

Terraform Providers Google
1

Hello there,
We are experiencing issue to get resources with GPU.
I would like to know if you could provide a way to ask terraform to create a ressource in the first region of a list of regions, sorted by preference ?
Or to get a data ressource to know the availability of GPU for instance.
Thx

2

Other than compute_machine_types, I don’t think there’s anything even close that’s accessible via a data source.

You’d probably have to either write something external that checks quotas, and even in that case, you could have available quota but still not have capacity.

The docs for the above (see example about Machine Family preference give you an example of how you could sort of accomplish this.

A kind of silly example like this might accomplish part of what you’re trying:

data "google_compute_machine_types" "us_central1_a" {
  project = var.project
  filter  = "name = \"a3-highgpu-4g\""
  zone    = "us-central1-a"
}

data "google_compute_machine_types" "us_central1_b" {
  project = var.project
  filter  = "name = \"a3-highgpu-4g\""
  zone    = "us-central1-a"
}

data "google_compute_machine_types" "us_central1_f" {
  project = var.project
  filter  = "name = \"a3-highgpu-4g\""
  zone    = "us-central1-f"
}

data "google_compute_machine_types" "asia_northeast2_a" {
  project = var.project
  filter  = "name = \"a3-highgpu-4g\""
  zone    = "asia-northeast2-a"
}

locals {
  zones = toset(coalescelist(
    [for mt in data.google_compute_machine_types.us_central1_a.machine_types : "us-central1-a" if mt.name == "a3-highgpu-4g"],
    [for mt in data.google_compute_machine_types.us_central1_b.machine_types : "us-central1-b" if mt.name == "a3-highgpu-4g"],
    [for mt in data.google_compute_machine_types.us_central1_f.machine_types : "us-central1-f" if mt.name  == "a3-highgpu-4g"],
    [for mt in data.google_compute_machine_types.asia_northeast2_a.machine_types : "asia-northeast2-a" if mt.name == "a3-highgpu-4g"],
  ))
}

output "test" {
  value = local.zones
}

However, just because the instance type exists in a zone doesn’t mean you’ll have quota and capacity. You could also randomize the zone to use, and try applying multiple times.

3

Hi,
Thank you for your answer. You show here how to know in which region some compute instances type are available.

But this is not my request. I know where there are such resources, I have the quota ok in the project, BUT unfortunately, Google has no more resources of this kind in some of theses regions.
So I’m looking for a way (or it’s a feature request) to deploy a GCE ressource according their availability.
In other word.

  • set a variable with a list of region where I know the resource is said available by google
  • terraform try to create the resource in region A,
  • if the creation failed because of stock out sent from Google, then terraform keep try in another region of the list instead of stopping the job

Thank you

4

Right, I understood what you meant, which is why I said that it might accomplish part of what you’re trying to do, and that you’d probably need to do something custom – as best I know, there’s not something that can solve your problem directly within Terraform / Google provider-- you’d likely have to write something custom and integrate with it somehow.

Even if you did write something custom, I’m not sure Google provides any information on capcity / stockout issues via API at all, so if the instance type exists in the region, and you have quota for it, there may not really be a way to know what’s available other than trying to create and seeing if it fails.

While you could randomize the region and just keep applying until success, I don’t think there’s an easy way in Terraform to, say, retry a failed apply with a fallback parameter.