Since the initial version of Waypoint, the product has supported the ability to configure multiple apps within a single waypoint.hcl file. Starting with Waypoint 0.9, this functionality is deprecated and will be removed in 0.10. The vast majority of users are not using this functionality and it served mostly as a source of confusion for users. For users who are using a monorepo pattern, we plan to add better workflows for you.
With a waypoint.hcl now focused on the configuration of a single application, the concept of a project within the Waypoint data model will be removed, moving applications to the top level. This is already how users talk about using Waypoint and we are confident that it will improve the overall understanding of Waypoint as well.
We’d love to hear your comments and concerns about this change below. We’re looking to incorporate the feedback into future planning work, such as ways to improve the monorepo pattern usage.
Sure, I’ve done some name changing in the files, but the content is pretty much the same.
The first one I’m attaching is the one I mentioned with ‘n’ number of client apps. Upon waypoint up, the first app definition builds the image and pushes it to ECR. Subsequent app definitions use that same image via docker-pull.
The second has a web app and two Nomad periodic job apps that get deployed. It also makes heavy use of dynamic stanzas that allow some differing configuration for each application. The build is similar to the one above where the first app builds the image and the other two use docker-pull to fetch it.
Similar to @dwanek, I was investigating migrating our Nomad deployment tooling to waypoint. Waypoint touts itself as having it’s projects map 1:1 with a VCS repository. In that case, it’s not clear to me how I would deploy multiple processes.
For a hypothetical python codebase with two processes:
web: python web.py
worker: python worker.py
How would we specify that in waypoint now, given the deprecation of projects? The two processes are functionally the same app, just running two different processes. To me, it makes sense to at least have a project-level abstraction that ties both together.
We have been using Waypoint for our monorepo project, with a top-level ./waypoint.hcl that’s been working well for us. Having just tried to separate this into a waypoint.hcl per app (./app1/waypoint.hcl, ./app2/waypoint.hcl, etc.), a few comments:
Seems we need to change directory to deploy multiple apps. Would be nice to have a replacement for “waypoint up -app app1” like “waypoint up -dir ./app1” that we can run from the VCS root directory. I thought “-p” might do this but seems it only relates to the server target. Perhaps I am missing something as fairly new to Waypoint.
Similarly (but less importantly) something like “waypoint init -dir ./app1” would be nice instead of having to descend into each directory. Not clear how to use existing “-from-project” and “-into” when the application directory already exists.
We did have a top-level config stanza for workspace-specific env vars shared between apps. It would be nice to be able to include these common definitions somehow in the app-level waypoint.hcl files.
We also had been using a top-level variable stanza to declare input variables shared between apps. Thought having a shared var-file might help us out here, but the variable declarations also need to be shared.
Thanks again for providing feedback on the decision to remove multi-app workflows from Waypoint. We really appreciate you all taking the time to share your setups with us.
First off, we want to apologize for the hard failure that accompanied the the deprecation warning. Our intentions were to warn users with the message, but unfortunately it shipped as a config validation error instead. We will release a patch soon to update the deprecation warning and remove the error.
The team has taken your feedback into consideration and decided that we don’t want to remove multi-application orchestration from Waypoint just yet. We understand the power and value in orchestrating these applications easily without forcing users to deal with config sprawl or needing to write extra wrappers around Waypoint to orchestrate deploying multiple applications. We are exploring the possibility of disallowing the configuration of multiple applications within a single waypoint.hcl file but want to continue to enable multi-applications in Waypoint. We are very interested to hear how you all envision configuring multiple applications with Waypoint and develop our product in a way that can support your workflows.
We recently released a patch version of Waypoint (v0.9.1) today that removes the hard depreciation failure and sets it to a warning instead as intended. See the release post here: Waypoint v0.9.1 released. Thank you! <3
If this is moving forward are there any plans to allow a “Terraform-like” ability to author these across multiple files?
The huge files are going to get unwieldy and fast - and if we are focusing to an application level, then having the ability to split out the variables, configs and other items that currently make up the bulk of the waypoint.hcl file would keep things clean and readable.
We wanted to follow up with this post and say that we have decided to remove the multi-app deprecation warning from Waypoint in a future release. We intend to continue to support defining multiple applications within a single waypoint.hcl for now. We greatly appreciate your feedback on this. If there’s a missing feature that you’d like to see to support your use-case with Waypoint, please feel free to open a feature request on our GitHub. We would love to hear from you!