Plugin framework: Use a custom type for API where "" and `null` are the same?

Terraform Providers Plugin Development
1

I’ve currently got an attribute definition like this:

"description": schema.StringAttribute{
  Optional:            true,
  Validators:          []validator.String{stringvalidator.LengthAtLeast(1)},
},

It’s Optional because neither the API, nor the accompanying web UI require the user to submit a description.

Though the API permits an omitted description attribute in POST it will return "" (rather than null) for this attribute in response to GET. There is no distinction between null and "" in the API.

The “no empty strings allowed” validator exists to eliminate ambiguity about what we should commit to the state during Read(): The user could not have set an empty string, so types.StringNull() is the best choice when the API returns "".

It’s just occurred to me that a custom string type which considers null and "" to be semantically equal would give my users a little more flexibility: It would allow them to either omit the description attribute, or to set an empty string.

Am I on the right track here? Any concerns about using a custom type in this way?

There are other attributes with similar behavior:

  • integer attributes where 0 is a special “I don’t care” value returned by the API when the attribute was omitted - I currently forbid 0 from the config via validation.
  • set attributes like aliases (set of strings) - I currently disallow empty sets via validation because the API doesn’t distinguish between [] and null.
2

Hi @hQnVyLRx,

While possible, from a Terraform perspective, I would suggest not allowing the ambiguity in the configuration at all if you can help it. While you can ignore an attribute change between null and "" in isolation, Terraform still considers those distinct values, hence experienced users will also expect those to be distinct values. If you only ever accept null and save null to the state, then you can hide this inconsistency for users rather than expose it as part of your provider’s API too.

Since this is an optional attribute, which form the provider returns during plan and apply doesn’t matter, you must return the same value which was found in the configuration. During read however you don’t have the configuration, and it helps to be consistent so as to not show changes happening outside of Terraform.

3

Thanks, @jbardin.

I was thinking the same way when I initially put this together (eliminate ambiguity!), but I’m wondering if I found the right balance with user-facing ergonomics.

In the case of the aliases attribute (set of strings which currently must never be empty for the same reasons), I’ve been contacted by users who are confused that they can’t do this:

resource "something" "x" {
  aliases = <some calculated value which might be an empty set>
}

And are annoyed that they have to do something like this:

locals { aliases = <some calculated value which might be an empty set> }
resource "something" "x" {
  aliases = length(local.aliases) > 0 ? local.aliases : null
}

Is this a big deal? Probably not. But the ergonomics of Hashicorp tooling is generally so good, so I try to honor that wherever I can.

During read however you don’t have the configuration, and it helps to be consistent so as to not show changes happening outside of Terraform.

Wouldn’t a custom type which considers null to be semantically equal to empty string (or set/list/whatever) accomplish the goal of not showing changes?

I haven’t tried it yet, so I’m not sure what I’m missing.

One more wrinkle:

Each resource in my provider has a corresponding data source which uses the same model struct. If I relaxed the validator so that “empty” values were permitted in the configuration, what value should be returned by the data source: null or empty? Semantic equality can’t help here, so I’d have to make a choice. While it seems less likely to be problematic (or even noticed), that choice could also lead to a bit of user confusion.

This data source wrinkle seems like another nudge toward “leave it alone, strict validation was the right choice”.

Anyway, I really appreciate the sounding board.

4

IMHO requiring

aliases = length(local.aliases) > 0 ? local.aliases : null

is more consistent, because I expect an empty set and a null set to be different values, but I can see how some users might find that inconvenient.

Wouldn’t a custom type which considers null to be semantically equal to empty string (or set/list/whatever) accomplish the goal of not showing changes ?

Oh yes, it does look like the the framework’s implementation compares the prior state during read and modifies the response accordingly. This will prevent changes in almost all cases. The inconsistencies arise from when these if the fields are referenced elsewhere, because you can now have one of two values. Granted you shouldn’t need to reference optional attributes, so this applies more to optional|computed.

The data source consistency is also something to consider, which is similar to the computed case above. I would expect the data source to return the same values as the managed resource, but if the resource is only consistent with what was last set in the config, the data source obviously can’t always match it. Settling on a normalized form of the state value can make sure these always match.

Anecdotally I’ve also found that many API’s which aren’t strict about how null and empty values are handled are often internally inconsistent about their handling. You might find paths which return one or the other in certain cases, and normalizing them all can help with processing the responses when you need to communicate with other systems (Terraform for example) which may not have the same relaxed semantics. In short, you might be able to remove “nil or empty” checks throughout the code, which could be missed in various spots (panics from providers hitting a nil value where something which was not usually nil were common in the in the old SDK, though the framework fixes a lot of that)

5

Thanks, @jbardin. This has been helpful.

6

I’ve defined a resource block without specifying a value for a required attribute. When I try to apply the configuration, Terraform’s API returns an empty string (“”) for that attribute in the response to a GET request. Terraform’s data model doesn’t differentiate between null and an empty string, causing ambiguity in handling the response.