Terraform not using s3 backend lock when executed in AWS CodeBuild

Terraform
#1

Hello,

I’m using terraform to create the infrastructure of a project where terraform is executed in a container in AWS CodeBuild.

When I run terraform in my development machine, I see the following message:

Acquiring state lock. This may take a few moments…

However, when terraform is executed in AWS CodeBuild, I don’t see such a message and, in fact, I’ve seen two parallel executions happening without any of them failing because of not being able to get the lock (one eventually failed because a resource already existed).

I haven’t been able to reproduce the problem running a container locally from the same base image using the same IAM role (i.e. in this case the lock is acquired). I’ve also tried with terraform 0.13.4 and 0.14.4 just in case to no avail.

What could be the reason for terraform not to acquire the lock while the same container in my development machine does?

Best regards,
Javier

#2

Hi @jcollado

which backend are you using?

#3

Hello @tbugfinder,

I’m using the s3 backend, the terraform configuration block is like this:

terraform {                                                                                                                                                                  
  required_version = ">=0.13.0"                                                                                                                                              
  required_providers {                                                                                                                                                       
    aws = {                                                                                                                                                                  
      source  = "hashicorp/aws"                                                                                                                                              
      version = "~> 3.1.0"                                                                                                                                                   
    }                                                                                                                                                                        
    null = {                                                                                                                                                                 
      source  = "hashicorp/null"                                                                                                                                             
      version = "~> 2.1.2"                                                                                                                                                   
    }                                                                                                                                                                        
    random = {                                                                                                                                                               
      source  = "hashicorp/random"                                                                                                                                           
      version = "~> 2.3.0"                                                                                                                                                   
    }                                                                                                                                                                        
    template = {                                                                                                                                                             
      source  = "hashicorp/template"                                                                                                                                         
      version = "~> 2.1.2"                                                                                                                                                   
    }                                                                                                                                                                        
  }                                                                                                                                                                          
  backend "s3" {                                                                                                                                                             
    bucket         = "<bucket_name>"                                                                                                                             
    dynamodb_table = "<dynamodb_table_name>"                                                                                                                                             
    encrypt        = true                                                                                                                                                    
    key            = "<object_key>"                                                                                                                                           
    kms_key_id     = "<kms_key_id>"
    region         = "<aws_region>"                                                                                                                                             
  }                                                                                                                                                                          
}
#4

Can you execute TF_LOG=TRACE terraform init within the CodeBuild environment? You might double check backend credentials and ensure to pass those properly.

#5

@tbugfinder It turns out that I was confused by the lack of the Acquiring state lock. log.

I’ve seen in the trace logs that dynamodb GetItem and PutItem operations calls were made as expected. Also, I’ve tried to reproduce the problem by launching multiple builds at a time and got some Error acquiring the state lock messages back, so I’m guessing that what I thought were parallel executions actually weren’t.

I’ve taken a look at the terraform source code and, despite I’m not a go programmer, I believe that somehow l.ui is nil in the AWS CodeBuild environment when these lines are executed:

Anyway, this was really a no issue. Thanks anyway, for the help in troubleshooting the problem.

1 Like