Within the context of the command terraform init
from an AWS CodeBuild instance of image hashicorp/terraform:0.12.19
, the module fails to download.
The source terraform template references a module package source in a s3 bucket.
module "outbound" {
source = "s3::https://s3.amazonaws.com/my-modules/outbound-0.0.1.zip"
...
The command fails with this output:
Error: Failed to download module
Could not download module "outbound" (main.tf:17) source code from
"s3::https://s3.amazonaws.com/my-modules/outbound-0.0.1.zip":
NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
This should be valid per the documentation:
The s3 endpoint is correct because the region is us-east-1
.
The CodeBuild project has an IAM role with correct permissions to s3.
The code works from a local CLI command terraform init
.
How do I get more information from the command output?
What am I doing wrong fetching the module package from s3?
I have tried many different s3 URLs.
The environment has the variable that would supply the task_role
to terraform: ECS_CONTAINER_METADATA_URI=http://169.254.170.2/v3/40292219-6c1c-424e-b072-8456e6ea8165
Hi @outthought,
Looking at the relevant code in the upstream library Terraform uses to fetch modules, it looks like it supports only three AWS credentials sources:
- The
AWS_ACCESS_KEY_ID
and AWS_SECRET_ACCESS_KEY
environment variables.
- The
~/.aws/credentials
file (or equivalent on other platforms)
- EC2 instance profile auth via the metadata service.
The Terraform AWS provider and S3 backend code checks for an additional environment variable AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
and, if present, adds an HTTP credentials source that seems to be intended to work within ECS, based on the log line the provider/backend produces in that case:
[INFO] ECS container credentials detected, RemoteCredProvider added to auth chain
So it looks like in order for the module installer to support ECS-style authentication the upstream go-getter
library would need to be adjusted to use logic similar to Terraform. Since go-getter
is a shared library common to several other applications, I imagine it would need some API changes to allow the calling application to customize the credentials configuration, and then maybe Terraform could configure it using the same implementation it uses for the S3 backend.
1 Like
To summarize, at present, terraform init
inside of AWS CodeBuild will fail to download modules from an S3 module source.
Thank you for the insight.
There are two workarounds I will mention in case they are helpful to someone.
One, a public s3 bucket and object, along with http
or https
module source (not prefix s3::
).
Two, a github module source with the AWS CodeBuild buildspec attribute git-credential-helper
.
Here are the details of option Two:
Terraform template module source:
module "outbound" {
source = "github.com/MyOrganization/outbound?ref=0.0.1"
...
buildspec.yml
:
version: 0.2
env:
git-credential-helper: yes