Trying to understand the boundaries envisioned for Waypoint

Waypoint
1

Hi, I’ve only just started to learn about Waypoint, but I’m already extremely impressed.

I’ve lately been using Copilot, AWS’s new CLI for ECS+Fargate, and I see similarities in terms of the workflows and UX that both tools are aiming towards. So, curious, I checked out the Waypoint tutorial Deploy Waypoint to AWS Elastic Container Service and I liked what I saw there.

The workflow and UX described in the tutorial look great, but it did make me curious about the intended boundaries for Waypoint. Specifically: if Waypoint can, and does, create the ECS cluster, ECS service, ECS task definition, AWS security group, ALB target group, ALB listener, etc, as part of deployment — why does it require that an ECR repository already exist prior to starting the deployment? I mean, Waypoint can clearly create resources on the fly, as needed. And it can also clearly detect existing resources and use them. So why not have it create an ECR repository as the very first step, if one doesn’t already exist?

Similarly, near the end of the tutorial I noticed this:

NOTE : The destroy command will not delete container images in the registry or the ECS cluster itself. Visit the ECR console to manually delete the entire registry or individual container images.

This is also puzzling to me — why would Waypoint not delete the ECS cluster that it created?

I suspect this occurred to me specifically because I’ve been using Copilot recently, and Copilot will create everything needed to deploy, including a VPC, ECR repository, etc — and it will also delete everything that it created, when instructed to do so.

To be clear, I definitely understand that Waypoint is new and early and much of this might be planned and simply not yet implemented. If that’s the case, that’s totally cool. But it might also be due to the “boundaries” of Waypoint’s use cases being envisioned as not including the creation, configuration, and deletion of the target environment.

See, Copilot is meant to cover the entire development and operations workflow, from creating environments, deploying apps and services to those environments, and deleting those apps, services, and environments.

But I imagine that since Hashicorp already offers Terraform for creating, configuring, and deleting environments, perhaps Waypoint’s scope is meant to complement Terraform’s, rather than overlap it.

That’d make total sense to me; I’m just looking for some guidance/clarification on this, so as to refine my mental model of the team’s vision for Waypoint.

Thank you!

3 Likes
2

Yes, great question. I’m also interested in the relationship between terraform and waypoint as it relates to creating target infrastructure for ecs/fargate. I like how waypoint can optionally create the infrastructure, but also allows it to be configured after creating it with something like terraform. We have a large number of terraform templates for different types of fargate architectures and a tool to wrangle them. It would be nice to have a lighter-weight waypoint plugin that simply creates task definitions and deploys them to ECS services. To do that, it would only need to be configured with the cluster, service, and task definition details. Maybe I’ll write a plugin for that.

Thanks again for waypoint! I’m super excited about it.

1 Like
3

I’m curious too. Given the current design that all waypoint-entrypoints connect to the server through an external LB, it’s required for every container to have Internet access. For me on ECS on EC2, that meant provisioning a NAT gateway, and that’s not exactly trivial.

Not to mention, ALBs are not free. Just running idle, they’re $16 a month (us-west-2). If you create one per deployment per app per environment, that’s a non-trivial overhead eventually.

4

@aviflax sorry that your original question languished a bit without a response. You pretty much answered it that many of the deficiencies in deletion, etc are because it’s so early and Waypoint still has many areas of incompleteness. We do want to be clear what things are created and which things are deleted with Waypoint.

With respect to boundaries, I believe you can expect some overlap in basic provisioning capability related to app deployments with Terraform to continue as we do want simple getting started use cases to work without having to introduce users to separate required tools like Terraform.

Beyond basic use cases, I would expect Terraform and other infrastructure provisioning tools to be very complementary with Waypoint as I described in this post about comparing Waypoint and Terraform.

@nickatsegment by default waypoint-entrypoint does require the ability to make an outbound network connection to the Waypoint Server and wherever the Waypoint URL Service is hosted (currently on HashiCorp Cloud). Yet you can self-host both the Waypoint Server and Horizon (powers the Waypoint URL Service) on your own private infrastructure and private network, which means your app does not need to have outbound internet connectivity or a load balancer to enable the Waypoint Entrypoint. You may need an LB for your production app traffic, but that’s probably not a Waypoint-specific requirement.

5

Thanks!

Yet you can self-host both the Waypoint Server

My understanding is there is currently only a self-hosted option. I assume you guys are thinking about Waypoint Enterprise in the future :slight_smile:

which means your app does not need to have outbound internet connectivity

While I can imagine this could be true in a new version of Waypoint, it is currently not easily accomplished on the ECS-on-EC2 platform. You’d have to add an internal LB for the Waypoint Server, configure that as its advertised address, and ensure the entrypoints can talk to it (either by requiring they’re in the same VPC as the server, or configuring VPC peering or something, outside of Waypoint).

There’s a point though where the deployment of the Waypoint server is complex enough that you can’t capture it in a single command like waypoint install; it basically should be deployed by some sort of meta-Waypoint. Which is exactly what Spinnaker’s Halyard does (more or less). And I’m looking at Waypoint as a way to avoid the sprawling complexity of Spinnaker :smiley: Just something to keep in mind.

6

This is an aside in this thread, but I’m going to modify ECR to create the repo if it doesn’t exist, that’s a good idea as well.

7

@jbayer and @evanphx In the same spirit I’d like to check if deploying Waypoint in AWS ECS Fargate or other similar offering within GCP/Azure is an envisioned possibility to those that have smaller footprint where provisioning a EKS/GKS/AKS is too much?

This way, it would be easier and faster to anyone start playing with Waypoint with a single container deployed.

If so, what are the current constraints that limit the deployment on Kubernetes cluster only? What’d be required to get abstracted in order to achieve this simpler deployment?

I’d be interested in contributing with that if some general guidance was provided.

Keep Rocking!

8

I’m getting acquainted with Waypoint and would greatly benefit from suggestions about what infrastructure to set up in Terraform (where we currently manage everything) and what to leave to Waypoint.

For context, my goal is to deploy to ECS Fargate. In my current attempt, I’m using terraform to setup my waypoint server, and defining my app’s dns, vpc, load balancer, listener, ECS task IAM policy + role. My waypoint config specifies corresponding subnet IDs, task role name, and ALB listener ARN.

I’ve run into two issues with this approach. 1) I would like terraform outputs to be our source of truth for the subnet IDs and listener ARN so that if any of those resources are rebuilt, I don’t need to remember to update my waypoint.hcl in a separate step. 2) The terraform-defined ALB listener that I’ve pointed waypoint at is considered modified after waypoint deploys because waypoint updates the target groups on it. Therefore, re-appylying the terraform config would remove those target groups and break our environment.

I’d welcome specific suggestions for my issues and/or more general guidance about how to best work with terraform and waypoint together.

[EDIT: also found a bug filed here: ecs plugin: configuring listener_arn, listener port, protocol and cert are reset · Issue #1568 · hashicorp/waypoint · GitHub]