Volumes stuck: nomad.volumes_watcher: error releasing volume claims

Nomad
1

I’m experiencing problems with CSI: volumes are getting “stuck” after some client node failures - job re-scheduling fails withvolume <name> has exhausted its available writer claims. Master logs following errors:

nomad.volumes_watcher: error releasing volume claims: namespace=default volume_id=<volume_name1> error="1 error occurred:
         * Permission denied
"
nomad.volumes_watcher: error releasing volume claims: namespace=default volume_id=<volume_name2> error="3 errors occurred:
         * Permission denied
         * Permission denied
         * Permission denied
"

Any ideas what’s happening there? What kind of permissions is missing?

2

Hi @ivoronin :wave:

I’m not sure what the problem is…

Which CSI driver are you using? Could you share the job files you used to deploy them?