We released Waypoint 0.6.2!
Changelog
FEATURES:
- plugin/nomad: Introduce Nomad On-Demand Runner support for Waypoint server. Launch
tasks to build containers in short lived runners. [GH-2593]
IMPROVEMENTS:
- cli/login: Allow login from-kubernetes command to work with non-default namespace installations [GH-2575]
- serverinstall/nomad: Update install helper to always setup a Consul service
with a backend and ui service tag. [GH-2597]
BUG FIXES:
- cli: Use values from filter flags when listing deployments and releases [GH-2672]
- cli:
waypoint status and waypoint status -app no longer display destroyed deployments [GH-2564]
- core: Fix a panic where a custom Waypoint plugin would panic if the plugin
did not properly implement a Registry component with AccessInfoFunc() [GH-2532]
- serverinstall/k8s: Clean up rbac resources on uninstall, and do not error when existing rbac resources are detected during server upgrade. [GH-2654]
- ui: avoid loading all status reports [GH-2562]
- ui: improve docker reference parsing [GH-2518]
Thanks,
The Waypoint Team
Changelog - waypoint/CHANGELOG.md at v0.6.2 · hashicorp/waypoint · GitHub
Binaries - waypoint v0.6.2 Binaries | HashiCorp Releases
1 Like
@briancain Thank you for working on adding Nomad ODR support! @krantzinator mentioned in one of the last releases it’d be coming soon but I didn’t expect it this fast 
I just finished getting my runner profile set up for Nomad with a custom config (I used a JSON config file), but my remote docker builds are actually throwing a 401. Did you come across this during your testing? I can open up an issue with more details if needed! The image I’m building is actually right out of the waypoint-examples repo, node-jobspec (but I added the Git data source to my project for remote operations).
2021-11-05T01:56:43.060-0400 [WARN] waypoint: unknown stream event: job_id=01FKQAQ3N6RAMJ1VNR85MWBF52 event="&{job:{id:"01FKQAQ3N6RAMJ1VNR85MWBF52" application:{application:"example-nodejs" project:"example-nodejs"} workspace:{workspace:"default"} target_runner:{id:{id:"01FKQAQ3N6M8S70RF9PYG551J1"}} data_source:{git:{url:"https://github.com/hashicorp/waypoint-examples.git" path:"nomad/nodejs-jobspec"}} build:{} state:ERROR queue_time:{seconds:1636091793 nanos:68774150} assign_time:{seconds:1636091795 nanos:150587411} ack_time:{seconds:1636091795 nanos:154905069} complete_time:{seconds:1636091803 nanos:83326169} data_source_ref:{git:{commit:"d8e7a5c3667c3cf823466cb06f1598967cabcc24" timestamp:{seconds:1634932596}}} config:{source:FILE} error:{code:13 message:"error setting up entrypoint layer: Error starting blob upload: 401 Unauthorized"}}}"
2021-11-05T01:56:43.061-0400 [WARN] waypoint: job failed: job_id=01FKQAQ3N6RAMJ1VNR85MWBF52 code=Internal message="error setting up entrypoâś“ Building Buildpack with kaniko...
❌ Testing registry and uploading entrypoint layer
! error setting up entrypoint layer: Error starting blob upload: 401 Unauthorized
2021-11-05T01:46:45.727-0400 [TRACE] waypoint: stopping signal listeners and cancelling the context
Hey there @paladin-devops! It’s my pleasure, I’m glad you are excited about Nomad ODR support! 
With On-Demand Runners and running builds remotely, we expect the build step to push to a remote container registry. Before a container builds, Waypoint attempts to authenticate to the remote registry defined in your waypoint.hcl. Given the error, I imagine this is probably the issue. I would double check to make sure you have a registry stanza in your build step, and then make sure the credentials are right. If that doesn’t work, please open an issue on GitHub and we can take it from there. Thanks!
1 Like
Seems to be a typo in the title: Waypoing 
Thanks @shantanugadgil - got that fixed 
1 Like
Thanks @briancain ! It works! 
I incorrectly assumed it had something to due w/entrypoint due to the log.
As a result of running remote operations though, I have a lot of ODR tasks left over in Nomad (26, to be precise!). Should they be getting cleaned up automatically? I noticed that these are “service” jobs but maybe “batch” jobs could work, so the task would end after the runner completes its task?
Once a task has completed, the jobs get deregistered in Nomad through the StopTask function: waypoint/task.go at 882e8760a2613679ae7b8a8d37e3d6701aaba993 · hashicorp/waypoint · GitHub
These jobs don’t get garbage collected right away in Nomad if I remember right. You could manually clean them up with nomad system gc I think.
But now that you mention batch jobs, I think it probably makes more sense for these tasks to be started that way since they are typically short lived. That would be a good feature request!
1 Like
@briancain I took a look at that code as well as the corresponding method in the Nomad API code and it adds up to me! I just can’t explain why my jobs aren’t getting cleaned up then (even after manually kicking off garbage collection)…
These are what appear to be the final logs at the end of the runner completing a task, but then seemingly starting back up again:
2021-11-05T19:36:59.594Z [DEBUG] waypoint.runner.agent: handled our one job in ODR mode, exiting
2021-11-05T19:36:59.594Z [INFO] waypoint.runner.agent: quit request received, gracefully stopping runner
2021-11-05T19:36:59.594Z [WARN] waypoint.runner.agent.runner.watch_config: exiting due to context ended
2021-11-05T19:36:59.595Z [WARN] waypoint: context cancelled, stopping interrupt listener loop
2021-11-05T19:37:19.457Z [INFO] waypoint: waypoint version: full_string="v0.6.2 (99350730+CHANGES)" version=v0.6.2 prerelease="" metadata="" revision=99350730+CHANGES
2021-11-05T19:37:19.458Z [DEBUG] waypoint: home configuration directory: path=/root/.config/waypoint
2021-11-05T19:37:19.458Z [INFO] waypoint.server: attempting to source credentials and connect
1 Like
Thanks for the extra info @paladin-devops - Feel free to add that to the issue you created for extra context. I think batch jobs should hopefully alleviate this issue.
1 Like
Thanks again for the help @briancain. I was actually just perusing the docs for info on the Nomad ODR configuration and I didn’t see it on this page. Should it be here or would it be in another location? (I dig the new versioned docs btw!)
