Context: I have a use case where my provider needs to be able to upload a particular file to a destination via my API (which my provider internally issues calls to), when given a certain URL in a main.tf.
This file happens to exist in a Sonatype Nexus repository where it is accessible via something like wget or curl.
Current Solution: I currently manually place the needed file(s) in the same directory as my main.tf and then pass the URI of the file (via ${path.module} inside main.tf) to my provider, which internally uses os.Open(uri) to create a variable for the file object.
Problem: Essentially, I need to be able to pass a URL to my custom provider so that it can perform an os.Open() against the file fetched via the URL rather than a local file.
Attempted Solution/Ideas:
A. First download the file to the same directory as the module, then pass the URI to the provider exactly how I’m doing now. I’m not sure how to download files via Terraform. The http resource doesn’t seem to actually download to a file, it basically just gets the response.
B. Modify my provider to be able to read directly from a URL (i.e. work directly with the response that the HTTP resource might provide), which would be passed in as a mere string within my calling module’s main.tf, and fetch the data/file from that URL and store it in a variable inside the provider (not sure how to do this either)
What is a way to do either of these approaches?
Hi @pritster5 
In terms of fetching the file you could either incorporate Go code within your provider to fetch the file, something along the lines of (taken from How can I efficiently download a large file using Go? - Stack Overflow):
import ("net/http"; "io"; "os")
...
out, err := os.Create("output.txt")
defer out.Close()
...
resp, err := http.Get("http://example.com/")
defer resp.Body.Close()
...
n, err := io.Copy(out, resp.Body)
Alternatively, if you can use the http provider, then you could use something like the following (taken from Terraform download local file from remote URL on apply and delete file on destroy - Stack Overflow):
data "http" "aws-lb-controller-policy" {
url = "https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.0/docs/install/iam_policy.json"
request_headers = {
Accept = "application/json"
}
}
resource "aws_iam_policy" "load-balancer-controller" {
name = "AWSLoadBalancerControllerIAMPolicy"
policy = tostring(data.http.aws-lb-controller-policy.body)
description = "Load Balancer Controller add-on for EKS"
}
1 Like
Thanks for this! Does the body of the HTTP response need to be parsed into a string via tostring()? Or can I just get the raw bytes of the response body and pass that to the provider?
In my use case, I’m getting a custom file type from the nexus that is not a json, so could my code in main.tf look something like this?
data "http" "nexus_file" {
url = "link/to/file"
request_headers = {
Accept = "application/FILE_EXTENSION"
}
}
resource "provider_resource_name" "base_file" {
data = data.http.nexus_file.body
}
Or does this approach only work if the response body is of type text or JSON?
The Read-Only attributes for the http provider include response_body_base64 so you could pass that to provider_resource_name:
resource "provider_resource_name" "base_file" {
data = data.http.nexus_file.response_body_base64
}
An analogous approach is shown in the example for the random_bytes resource which exposes base64-encoded bytes.