Waypoint install to nomad with NFS CSI mount

Waypoint
1

Hi there,

I am trying to install a waypoint server to my nomad cluster. After Roomba-ing about a bit, I’m getting the following error:

❌ Creating persistent volume
! Error installing server into nomad: Failed creating Nomad persistent volume: Unexpected response
code: 500 (rpc
  error: 1 error occurred:
  	* controller create volume: rpc error: controller create volume:
  CSI.ControllerCreateVolume: controller plugin returned an internal error, check
  the plugin allocation logs for more information: rpc error: code = Internal desc
  = failed to mount nfs server: rpc error: code = Internal desc = mount failed:
  exit status 32
  Mounting command: mount
  Mounting arguments: -t nfs -o noatime
  <nfs-server>:/<share-path> /tmp/waypoint-server
  Output: mount.nfs: rpc.statd is not running but is required for remote locking.
  mount.nfs: Either use '-o nolock' to keep locks local, or start statd.
  
  
  )

…and here is the command I’m running which returns the above error:

waypoint server install \
-platform="nomad" \
-nomad-service-provider="consul" \
-nomad-consul-service-hostname="waypoint.<my-domain>" \
-nomad-consul-service \
-nomad-consul-datacenter=<my-dc> \
-nomad-dc=<my-dc> \
-nomad-csi-volume-provider="nfs.csi.k8s.io" \
-nomad-csi-plugin-id="nfs" \
-nomad-csi-external-id="csi-nfs" \
-nomad-csi-fs="ext4" \
-nomad-csi-parameters="server=<my-nfs-server>" \
-nomad-csi-parameters="share=/<my-share-path>" \
-nomad-csi-parameters="mountPermissions=0" \
-nomad-runner-csi-volume="waypoint-server" \
-nomad-consul-service-ui-tags="controlplane, http, traefik.enable=true, traefik.http.routers.waypoint.rule=host(\`waypoint.<my-domain>\`), active, initialized" \
-accept-tos

Looking at the parameters I am able to pass in there does not appear to be a way to set the nolocks mountOption that would be required to get past this error.

Is there any under-documented ways to set mount options via parameters when instructing Waypoint on how to configure the volume? If I were to use Terraform to describe the volume in advance, I have access to a mount_options block. If I specify via the -nomad-runner-csi-volume flag a CSI volume that already exists, will it just try to use that volume?

I don’t recall the docs clarifying the designation of an existing volume vs creating a new one.

Thanks in advance,
Sam

2

Hey there @dehuszar,

It looks like we currently have the mount flags hard coded to noatime. We don’t have any other flags to override this currently. I’ve just opened up a PR (core/install: Add configurable CSI mount flags to install by demophoon · Pull Request #4387 · hashicorp/waypoint · GitHub) to add this flag in and we should have it in the next release of Waypoint.

Additionally you have a great point regarding the install reusing an existing volume instead of assuming that one doesn’t exist for Waypoint to create. I’ve opened an issue for this which can be followed on Github here: Nomad installations should allow an existing CSI volume to be used · Issue #4388 · hashicorp/waypoint · GitHub

1 Like