Returning nested maps from provider

Before 0.12, if my provider returned maps where nested keys were dot-separated, eg d.Set("something", map[string]{"foo.bar.baz":"hello"}), a user could then access it like "${myprovider.some_id.someting.foo.bar.baz}".
As of 0.12, with the same provider code, it seems the ens user TF code would instead need to be myprovider.some_id.something["foo.bar.baz"].

Is there something I can do in my code to keep the old structure?

Hi @carlpett,

The old behavior was an unintended consequence of the way Terraform 0.11 handled references, caused by an ambiguity in how map values were stored in the state. This bug was fixed in Terraform 0.12 to resolve that ambiguity and thus allow for .foo.bar.baz to be parsed as if it were ["foo"]["bar"]["baz"] instead, as the documented language grammar suggested it should be.

There is no way to use the old buggy syntax in Terraform 0.12. A map key is always a single string and never interpreted as multiple separate parts.

The type system changes in Terraform 0.12 allow a provider to, in principle, declare an attribute as being a map of map of map of string, which would allow the chained attribute syntax you mentioned to work by returning nested maps instead. Unfortunately, because we’re still using the Terraform SDK written for Terraform 0.11 and prior (to allow the cross-compatibility between both versions) there is no way for a provider to currently do that in practice, or else it would be incompatible with Terraform 0.11. Development on an updated SDK designed around the 0.12 data model is starting now, and will eventually allow providers to make use of this.

In the meantime, nested maps cannot be permitted because the 0.11 provider protocol and state snapshot format have no way to serialize them.

1 Like

Thanks @apparentlymart! I suspected as much. Will wait for the new SDK then :slight_smile:
Small question regarding the incompatibility with 0.11, though - I was under the impression that after updating the SDK to become compatible with 0.12 the provider already became incompatible with 0.11? Is this a misunderstanding, and the new binaries I build for 0.12 are actually still 0.11-compatible?

Yes, any provider plugin currently released with 0.12 support at the time I write this also has protocol support for both 0.11 and 0.10.

The SDK still primarily talks the 0.11 protocol and 0.12 support is implemented currently via an adapter layer to the 0.11-oriented API. The future work I was referring to aims to change that such that the SDK is primarily using the 0.12 protocol and can support the new features that allows; at that point, 0.11 compatibility will no longer be possible because those new features cannot be represented in the 0.11-compatible wire protocol.

1 Like

@apparentlymart do you know if this API is read for public usage yet?

I’m trying to do something described here. I created another post also: Provider Schema Types Map of Maps

I think this is the tracking issue for making the SDK support the full Terraform 0.12 type system: