Whole-provider-process synchronization

rickhlx · July 12, 2019, 1:14pm

Would like to know more about whole-provider-process sync as mentioned here

I’m running into an API limitation requiring me to set parallelism to 1.

rickhlx · July 12, 2019, 8:00pm

@apparentlymart is this something you could point me in the right direction for?

apparentlymart · July 15, 2019, 4:48pm

It’s been a while since I wrote that comment, but I believe I was thinking of a generic synchronization primitive rather than anything Terraform-specific. For example, locking a Mutex for the duration of each operation would ensure that within that single provider process only one operation can be in progress at the same time, which would create a similar effect to setting the parallelism option to 1.

That mutex could be kept inside the object returned from Configure that is passed in as meta to resource type arguments, for example. That would then make it available to all resource type implementations in the provider.

rickhlx · July 16, 2019, 3:53pm

Thanks @apparentlymart!

This was very helpful, ended up using a version of how the AWS provider implements Mutex locks. Linking some

github.com

terraform-providers/terraform-provider-aws/blob/91fd73606fa7d8dbbe29bffec5010dd9ad341a1b/aws/provider.go#L1066


	if v, ok := d.GetOk("forbidden_account_ids"); ok {
		for _, accountIDRaw := range v.(*schema.Set).List() {
			config.ForbiddenAccountIds = append(config.ForbiddenAccountIds, accountIDRaw.(string))
		}
	}


	return config.Client()
}


// This is a global MutexKV for use within this plugin.
var awsMutexKV = mutexkv.NewMutexKV()


func assumeRoleSchema() *schema.Schema {
	return &schema.Schema{
		Type:     schema.TypeSet,
		Optional: true,
		MaxItems: 1,
		Elem: &schema.Resource{
			Schema: map[string]*schema.Schema{
				"role_arn": {
					Type:        schema.TypeString,

github.com

terraform-providers/terraform-provider-aws/blob/761927fa2909d5aefd9c780d565c3f41192cd093/aws/resource_aws_security_group_rule.go#L151


				ValidateFunc: validateSecurityGroupRuleDescription,
			},
		},
	}
}


func resourceAwsSecurityGroupRuleCreate(d *schema.ResourceData, meta interface{}) error {
	conn := meta.(*AWSClient).ec2conn
	sg_id := d.Get("security_group_id").(string)


	awsMutexKV.Lock(sg_id)
	defer awsMutexKV.Unlock(sg_id)


	sg, err := findResourceSecurityGroup(conn, sg_id)
	if err != nil {
		return err
	}


	perm, err := expandIPPerm(d, sg)
	if err != nil {
		return err

My provider is working great now.

Thanks,

Topic		Replies	Views
Per-provider parallelism options for disabling Plugin Development	2	431	August 22, 2023
ReadWrite Lock in a Provider? Plugin Development	7	254	April 13, 2024
Locking to avoid multiple like resources being created concurrently? Plugin Development	2	1506	March 5, 2021
SDKv2 Provider - Caching API Responses Plugin Development	2	371	February 7, 2023
Provider plugins that live for the duration of a terraform run Terraform	2	733	August 6, 2019

Whole-provider-process synchronization

Related topics