Pass arbitrary maps to provider functions

Terraform Providers Plugin Development
1

I am writing a function in a custom provider. It accepts an arbitrary map as a DynamicParameter, does some logic, and returns another arbitrary map. I know that the map will be known by the time of function’s execution. I do not know the structure of the map beforehand, and it is deeply nested.

I am currently stuck trying to reflect a Terraform value into a Go value. My goal is to have this work:

var arg SomeType
_ = req.Arguments.Get(ctx, &arg)
m := make(map[string]interface{})
_ = arg.As(ctx, &m)
  1. I have attempted to implement a CustomType of DynamicTypable with a CustomValue of DynamicValuable, to no avail; in the implementation of
func (i CustomValue) As(ctx context.Context, target interface{}) diag.Diagnostics {}

, which receives attr.Value, fields of CustomValue.UnderlyingValue() are unexported and thus unaccessable through reflection. Reflection code is:

func valueToMap(ctx context.Context, val interface{}) (map[interface{}]interface{}, diag.Diagnostics) {
	data := make(map[interface{}]interface{})
	varType := reflect.TypeOf(val)
	if varType.Kind() != reflect.Struct {
		err := fmt.Errorf("not a struct")
		return nil, diag.Diagnostics{
			diag.NewErrorDiagnostic(
				"Value Conversion Error",
				fmt.Sprintf("An unexpected error was encountered trying to convert the value. This is always an error in the provider. Please report the following to the provider developer:\n\nError: %s", err.Error()),
			),
		}
	}

	value := reflect.ValueOf(val)
	for i := 0; i < value.NumField(); i++ {
		if !value.Field(i).CanInterface() {
			//Skip unexported fields
			continue
		}
		fieldName := varType.Field(i).Name
		if varType.Field(i).Type.Kind() != reflect.Struct {
			data[fieldName] = value.Field(i).Interface()
		} else {
			data[fieldName], _ = valueToMap(ctx, value.Field(i).Interface())
		}

	}
	return data, diag.Diagnostics{}
}
  1. I have attempted to use asgotypes.GoPrimitive from terraform-plugin-go-contrib
var dynValue types.Dynamic
if err := req.Arguments.Get(ctx, &dynValue); err != nil {
	resp.Error = function.ConcatFuncErrors(err)
	return
}
var m asgotypes.GoPrimitive
switch value := dynValue.UnderlyingValue().(type) {
case types.Map:
	diags := value.ElementsAs(ctx, &m, true)
	if diags.HasError() {
		resp.Error = function.ConcatFuncErrors(resp.Error, function.FuncErrorFromDiags(ctx, diags))
		return
	}
default:
	resp.Error = function.NewArgumentFuncError(0, "should be map")
	return
}

but it errored with

cannot reflect tftypes.Map[tftypes.Object["this": ...] into a struct, must be an object.

Been stuck on this task for a few days, went through tftypes, basetypes, reflect packages in terraform-plugin-framework repo… I really do not want to jsonencode() the map on Terraform’s side before passing it to the function as a string.

2

If I understand correctly, I think the confusion here is stemming from the fact that maps cannot contain arbitrary types, all values must be of the same type. If the function accepts a dynamic type, unless the caller is careful to match the structures of all values, the result must be an object rather than a map.

This is especially true if the caller is constructing the value in the configuration, as the {...} object syntax will default to an object type unless converted to a map, either explicitly or via a type conversion during assignment. Since the type is dynamic however, there can’t be any type conversion.

3

Thank you for your reply!
The fact that Map contains values of the same type is why I’ve set the parameter as Dynamic, because, as far as I understand, I cannot use Object either. Object parameter should have AttributeTypes set, which I do not know. I have some arbitrary map/object (read from YAML file with yamldecode(file(filepath))). How can I pass it to the provider function? Or in other words, how can I convert it into a Go value?

4

If the map values are actually all of the same type, then it would be easier to declare their type in the schema rather dealing with a dynamically typed object. If the values do vary at all in their structure (which I assume since you are using an empty interface type in the example map), you would still need this to be a dynamic type.

Unless you specifically declare the parameter as a map type though, the underlying type of the value coming from Terraform is usually going to be an object. The caller could be actually assigning a map, because there is no constraint on the type, so you must take that into account too, but it will probably be the less common path.

I’m not as familiar with the provider implementation here, so someone else may be able to step in and give more specific advice. If you are dealing with the terraform type system, you will have to take each value, decide if it’s an object (or a map), and recursively iterate over the attributes. I’m not sure if decoding into a map[string]any is supposed to work in that case, since it’s implemented slightly differently from the Terraform side of things.

5

Hey there @marsskop :wave: ,

So a good chunk of the reflection utilities (ElementAs for maps, As for object, etc.) that exist in framework for converting framework types (attr.Value) into Go types are based off knowing the type beforehand (confirming what you observed in your original post). This is also true for the underlying terraform-plugin-go/tftypes package that custom type implementations have access to as well.

As an alternative, you can recursively walk through the dynamic underlying value by inspecting it’s type then determining what to do with each type, as I believe you started to try with asgotypes.GoPrimitive. The general rule is unless you already know the full type, you won’t be able to extract complex values like objects with the reflection utilities that exist in framework ATM.

I’d write out an example, but there actually is a good example that exists in the kubernetes provider that does this type of recursive walking through the data, manifest_encode:

Now the example I linked is trying to build a YAML string, but in the process it builds out a map[string]any where it doesn’t know the type beforehand.

Let me know if that helps!

2 Likes
6

Thank you! I’ve implemented recursive walking similar to the one in kubernetes provider, works perfectly in my case!

1 Like