Thinking more about what you shared before and what you’ve added here, unfortunately I think the main thing you’re looking for here lives in the gaps that the sourcebundle package delegates to its calling application. Specifically:
-
DependencyFinder delegates the problem of analyzing an already-fetched directory to see what other packages it depends on.
That is, this package expects its caller to provide logic like “find all of the .tf files, find the module blocks inside them using HCL, and then take the source and version arguments”.
-
PackageFetcher delegates the problem of actually retrieving a remote source package and placing its content into a designated directory on local disk for further analysis.
-
RegistryClient delegates the problem of speaking the Terraform Module Registry Protocol to allow translating a registry-style source address and version constraints into a remote package address that the PackageFetcher can retrieve.
sourcebundle.Builder contains the glue logic to drive the interactions between implementations of those three interfaces, but it doesn’t provide any implementations itself because real implementations of these interfaces tend to require relatively “heavy” dependencies, like HCL itself, go-getter and all of its transitive dependencies, etc.
It sounds like you already wrote something that could be turned into an implementation of DependencyFinder.
A PackageFetcher that directly wraps go-getter’s “getters” (selecting one based on sourceType) ought to be relatively straightforward to build given that sourceaddrs currently implements a subset of the go-getter address syntaxes. (This is also the place where you could, if you wanted to, optimize how exactly to fetch git repositories. Terraform Cloud’s implementation of this interface makes a shallow clone, for example, because we know that in that context there will never be any further Git operations run against the resulting work tree.)
For RegistryClient there isn’t really much reasonable variation in the implementation – it’s basically always going to be wrapping an HTTPS client making calls to two of the operations in the Terraform Module Registry protocol – but so far there isn’t a a ready-to-call example of that in any open source Go library.
I do expect that at some point we’ll have open-source libraries that either directly implement these interfaces or are at least similar enough in their level of abstraction for it to be trivial to write a wrapper implementation, but I also don’t expect that to come in the very near future because we’re currently focused on using this thing as a vehicle for the early private preview of Stacks, and so the flexibility to change it in response to feedback is more important.
If you’re willing to implement all three of these interfaces then sourcebundle could work for you, but with one big caveat: it intentionally supports only a subset of the remote source address types that Terraform CLI supports today. Specifically, it supports Git repositories and HTTPS URLs that refer to gzipped tarballs, as previously mentioned. If you’d like your tool to support other source types then this implementation strategy would not be suitable.
It might grow to support other source types in future, but I don’t expect it to ever reach 100% parity with Terraform CLI because several of its supported source types are essentially technical debt at this point, and the Stacks execution model (which is what this library was written in support of) is providing an opportunity for some carefully-considered breaking changes.
) files, I am loading them up into a custom struct, then passing them to another process that goes to download them.