While it’s great to hear that pull requests are being reviewed again, my overall experience with Hashicorp open-source projects has been primarily one of frustration.
As an example of a frustrating experience I had, in Summer 2021 I mentored some students hoping to gain real-world software engineering experience. The goal of the project was for the students to make a meaningful contribution to a real-world open-source project. I feel that they did a great job on the implementation and learned a lot, but I also feel that the response to the pull request they submitted at Add flag to exclude specific resources from plan by alexl563 · Pull Request #30041 · hashicorp/terraform · GitHub has likely given them a rather negative impression of what open-source is like, and I feel partially responsible for that bad experience, as their mentor.
To explain why I feel the response to my students’ pull request wasn’t great, I have to give a bit of context.
In June 2015, a feature request was submitted for an -exclude option to match -target. The thread is https://github.com/hashicorp/terraform/issues/2253. Since then, that feature request has garnered almost 1,500 reactions (the single most upvoted issue amongst the more than 1,500 on the tracker) and 125 comments. Virtually every member of the community who has given an opinion has come out in favor of the utility of the feature (note the over 1,000 upvotes and not a single downvote, and numerous universally upvoted comments echoing the request).
Thanks to the vibrant contributor base of Terraform (even in 2015), it only took 7 days for a pull request to be submitted that would solve the problem. This is at adding exclude flag to plan and apply by nevins-b · Pull Request #2326 · hashicorp/terraform · GitHub. The PR received a brief review comment, but after that comment was addressed, there was no further feedback for several months. Another enterprising contributor rebased the PR into adding exclude flag to plan and apply to exclude specified resource from map by josephholsten · Pull Request #3366 · hashicorp/terraform · GitHub to keep up with changes from master, but despite this effort, there was no feedback from Hashicorp over the next several years, despite continued requests for updates from the community both on the PR thread and in the original feature request. After five years, the PR was closed with the simple comment that “we haven’t forgotten this request, but it’s going to take some careful design and isn’t on our immediate roadmap.”
This pattern of months- or even years-long radio silence on issues that are important to the community is one I have unfortunately seen replicated on numerous threads across Terraform’s issue tracker. Usually, when a response does come from Hashicorp, it is to say that there are internal concerns around the proposed implementation - usually ones that haven’t been communicated publicly - and that the team would prefer to address the problem in a more comprehensive way at some future date. This makes sense insofar as part of open-source maintenance is ensuring coherent design decisions make it into the product, but unfortunately it seems that many of the design discussions for Terraform are only allowed to happen internally, and the issues that are important to the community are “not prioritized” or “require further consideration” or “do not currently have a roadmap”, with no clear next action and no way for the community to help. This is especially frustrating when there is a clear and simple way to address the core community concern, but any movement in this direction is rejected in favor of a yet-to-be-designed comprehensive solution that may take years to implement.
An example of this pattern that I found particularly frustrating can be found in the feature request https://github.com/hashicorp/terraform/issues/28803, which asks for a way to disable new output from Terraform that many people found confusing. This feature request has again received virtually universal support from every member of the community who has cared to weigh in - note the 281 upvotes / 0 downvotes on the request (and many hundreds more on subsequent comments), and the 4 upvotes / 15 downvotes on the response from Hashicorp explaining the case against implementing it. I submitted one of the upvotes on this feature request, because the added output made Terraform unusable for my company’s application engineers. In the end, as one of the people responsible for keeping those engineers happy, I had to add a wrapper script around Terraform to delete the new output with a regex. I felt disgusting.
Despite numerous requests for updates or reconsideration from various community members on this feature request, there was no response for 5 months, and eventually the issue was closed with a comment essentially declining the community feedback in favor of plans to make the output more helpful in some planned future update (involving “cross-cutting” features that are “not something we can just casually implement without careful design first”). Based on the comments in the issue thread, I do not think this is the resolution that most community members were looking for, and I suspect that this would have been communicated if the thread had not been locked. For my part, that disgusting regex is still on the critical path of my company’s production infrastructure because Terraform provides no other way forward. (The update 5 months later in Terraform 1.2.0 mentioned in the final comment in that feature request is, to my understanding, only a partial solution - the language in the comment even says “in most cases” - and there is no reference to future plans to completely fix the problem.)
With all this context in mind, let’s come back to my students. I knew that pull requests sometimes take a while to be reviewed, regardless of project, but if there was anything that would receive attention for Terraform, I figured it would surely be a pull request fixing the six-year-old, number-one-most-popular feature request on the issue tracker - based on the latest master, with added and passing tests and documentation. Sure, there was a pull request from 2015 that had never received a response, but I felt sure that engagement with community pull requests must have improved sometime in the last six years, right?
Unfortunately, despite numerous supportive comments from the community, the new pull request Add flag to exclude specific resources from plan by alexl563 · Pull Request #30041 · hashicorp/terraform · GitHub - just like its predecessor - waited 2 months just to receive a comment remarking that there was “no update on when it can be merged”. It’s now been almost a year, and still the latest is “I do not have much of an update for you right now”. Following the same sentiment seen elsewhere on the issue tracker, the team “would rather address the underlying fundamental limitation directly as a product design issue”, which sounds great in principle, but in practice (given that it has been over 7 years by now with no reported progress) makes me wonder if my students will ever get to see others benefit from their work.
At first I was excited that after many weeks of effort, my students would have a positive experience closing the loop on the open-source contribution process and work with maintainers to get their code safely into an upcoming release. But as weeks stretched into months, that excitement began to wither into disappointment and frustration. I suspect many of the hundreds of commenters on this and other Terraform issue threads may have similar feelings.
While I understand the necessity for contributors to be patient, and the importance of sometimes saying no as a maintainer - I maintain dozens of open-source projects myself, have very limited bandwidth (writing this comment has taken most of my open-source time for the week), and have on occasion had to make difficult or unpopular decisions - the philosophy that I see on the Hashicorp issue tracker takes this ethos farther than I personally think is healthy for the community. The extended communication gaps on numerous popular feature requests and pull requests only compound the problem, and make it difficult to believe as a contributor that my work truly will be valued or even acknowledged.
Is there anything that can be done on Hashicorp’s side to make Terraform feel more like open-source software in the community sense, and less like software whose source happens to be open?