I’ve been racking my head trying to solve an issue with some code I have written which makes use of Nomad Exec and have run into an issue where Nomad seems to be blocking an ACL.WhoAmI RPC call witha permission denied…
I have ACLs enabled in Nomad & Consul and also have consul connect enabled (however there is no use of connect in this code right now) I also have the nomad job running in a non-default region and namespace but as far as I am aware my ACL policy exists within the correct region and targets the correct namespace
Quick log snippet from Nomad:
Mar 09 17:28:12 soc-se-07 nomad[1098]: 2023-03-09T17:28:12.583+0100 [INFO] client.alloc_runner.task_runner: Task event: alloc_id=99bc2a20-09b3-bf87-99dd-4d5dd6e02b17 task=executor type=Started msg="Task started by client" failed=false
Mar 09 17:28:14 soc-se-07 nomad[1098]: 2023-03-09T17:28:14.426+0100 [ERROR] client.rpc: error performing RPC to server: error="rpc error: Permission denied" rpc=ACL.WhoAmI server=10.0.10.12:4647
Mar 09 17:28:14 soc-se-07 nomad[1098]: 2023-03-09T17:28:14.426+0100 [ERROR] client.rpc: error performing RPC to server which is not safe to automatically retry: error="rpc error: Permission denied" rpc=ACL.WhoAmI server=10.0.10.12:4647
Mar 09 17:28:14 soc-se-07 nomad[1098]: 2023-03-09T17:28:14.427+0100 [INFO] client: task exec session starting: exec_id=00de9880-0fba-86e2-7875-b610b4e8d34f alloc_id=99bc2a20-09b3-bf87-99dd-4d5dd6e02b17 task=executor command=["/bin/bash"] tty=false
Mar 09 17:28:14 soc-se-07 nomad[1098]: 2023-03-09T17:28:14.427+0100 [INFO] client: task exec session ended with an error: error="rpc error: Permission denied" code=<nil>
Mar 09 17:28:14 soc-se-07 nomad[1098]: 2023-03-09T17:28:14.437+0100 [ERROR] http: request failed: method=GET path="/v1/client/allocation/99bc2a20-09b3-bf87-99dd-4d5dd6e02b17/exec?command=%5B%22%2Fbin%2Fbash%22%5D&namespace=cicd-gitlab®ion=se&task=executor&tty=false" error="rpc error: Permission denied" code=500
What ACL policy would I be missing to be able to to a self token lookup?
namespace "${namespace}" {
capabilities = [
"list-jobs",
"read-job",
"dispatch-job",
"read-logs",
"alloc-exec",
"alloc-lifecycle"
]
}
node {
policy = "read"
}
agent {
policy = "read"
}
Thanks!