Since the initial version of Waypoint, the product has supported the ability to configure multiple apps within a single waypoint.hcl file. Starting with Waypoint 0.9, this functionality is deprecated and will be removed in 0.10. The vast majority of users are not using this functionality and it served mostly as a source of confusion for users. For users who are using a monorepo pattern, we plan to add better workflows for you.
With a waypoint.hcl now focused on the configuration of a single application, the concept of a project within the Waypoint data model will be removed, moving applications to the top level. This is already how users talk about using Waypoint and we are confident that it will improve the overall understanding of Waypoint as well.
Weâd love to hear your comments and concerns about this change below. Weâre looking to incorporate the feedback into future planning work, such as ways to improve the monorepo pattern usage.
Hey @evanphx, we actually use the multi-app feature quite a bit with nomad-jobspec deploys. Is there a path forward for this?
For instance, we have one waypoint âappâ config that manages the web app and another âappâ config that launches a nomad âperiodicâ jobspec.
We also have another use-case that has many apps that get launched because the code is not multi-tenant. So, in this particular case, ânâ number of clients means ânâ number of apps.
I think the later case we could refactor into the nomad jobspec, but the first case needs two different jobspecs because âperiodicâ is defined at the job level.
Sure, Iâve done some name changing in the files, but the content is pretty much the same.
The first one Iâm attaching is the one I mentioned with ânâ number of client apps. Upon waypoint up, the first app definition builds the image and pushes it to ECR. Subsequent app definitions use that same image via docker-pull.
The second has a web app and two Nomad periodic job apps that get deployed. It also makes heavy use of dynamic stanzas that allow some differing configuration for each application. The build is similar to the one above where the first app builds the image and the other two use docker-pull to fetch it.
Similar to @dwanek, I was investigating migrating our Nomad deployment tooling to waypoint. Waypoint touts itself as having itâs projects map 1:1 with a VCS repository. In that case, itâs not clear to me how I would deploy multiple processes.
For a hypothetical python codebase with two processes:
web: python web.py
worker: python worker.py
How would we specify that in waypoint now, given the deprecation of projects? The two processes are functionally the same app, just running two different processes. To me, it makes sense to at least have a project-level abstraction that ties both together.
We have been using Waypoint for our monorepo project, with a top-level ./waypoint.hcl thatâs been working well for us. Having just tried to separate this into a waypoint.hcl per app (./app1/waypoint.hcl, ./app2/waypoint.hcl, etc.), a few comments:
Seems we need to change directory to deploy multiple apps. Would be nice to have a replacement for âwaypoint up -app app1â like âwaypoint up -dir ./app1â that we can run from the VCS root directory. I thought â-pâ might do this but seems it only relates to the server target. Perhaps I am missing something as fairly new to Waypoint.
Similarly (but less importantly) something like âwaypoint init -dir ./app1â would be nice instead of having to descend into each directory. Not clear how to use existing â-from-projectâ and â-intoâ when the application directory already exists.
We did have a top-level config stanza for workspace-specific env vars shared between apps. It would be nice to be able to include these common definitions somehow in the app-level waypoint.hcl files.
We also had been using a top-level variable stanza to declare input variables shared between apps. Thought having a shared var-file might help us out here, but the variable declarations also need to be shared.
Thanks again for providing feedback on the decision to remove multi-app workflows from Waypoint. We really appreciate you all taking the time to share your setups with us.
First off, we want to apologize for the hard failure that accompanied the the deprecation warning. Our intentions were to warn users with the message, but unfortunately it shipped as a config validation error instead. We will release a patch soon to update the deprecation warning and remove the error.
The team has taken your feedback into consideration and decided that we donât want to remove multi-application orchestration from Waypoint just yet. We understand the power and value in orchestrating these applications easily without forcing users to deal with config sprawl or needing to write extra wrappers around Waypoint to orchestrate deploying multiple applications. We are exploring the possibility of disallowing the configuration of multiple applications within a single waypoint.hcl file but want to continue to enable multi-applications in Waypoint. We are very interested to hear how you all envision configuring multiple applications with Waypoint and develop our product in a way that can support your workflows.
We recently released a patch version of Waypoint (v0.9.1) today that removes the hard depreciation failure and sets it to a warning instead as intended. See the release post here: Waypoint v0.9.1 released. Thank you! <3
If this is moving forward are there any plans to allow a âTerraform-likeâ ability to author these across multiple files?
The huge files are going to get unwieldy and fast - and if we are focusing to an application level, then having the ability to split out the variables, configs and other items that currently make up the bulk of the waypoint.hcl file would keep things clean and readable.
I would like to second the comments in this thread about being able to run multiple variants of the same app/codebase.
Especially in a Ruby on Rails scenario, itâs quite common to have a few app types in Procfile. Maybe a web definition and one or multiple ones for several queues/Sidekiq/background job workers.
So either would be nice to have multiple app definitions or files within one directory and one Project/App in Waypoint as these would have the exact same set of Input Variables turned into ENV vars.
Could also be that in the app stanza we could somehow specify multiple âexecutorsâ with per executor also how many replications and resource constraints should be applied.
The last would allow a Heroku-style deployment where one waypoint up would fully deploy the whole app infrastructure in a rolling way with a new code version and/or new ENV var settings.
We wanted to follow up with this post and say that we have decided to remove the multi-app deprecation warning from Waypoint in a future release. We intend to continue to support defining multiple applications within a single waypoint.hcl for now. We greatly appreciate your feedback on this. If thereâs a missing feature that youâd like to see to support your use-case with Waypoint, please feel free to open a feature request on our GitHub. We would love to hear from you!
Given waypoints lambda support I would be really curious how many people actually have a multiple repos per lambda. I currently have 20+ lambdas in a repo. Splitting this out doesnât make sense to me (or likely my Github administrator) as every other single tool I use, golang,terraform etc. Can work with this configuration.
