Anonymous token lacks permission 'service:write' on vault

Could anyone shed some light on the meaning behind this error?
[ERROR] agent.client: RPC failed to server: method=Catalog.Register server=172.18.100.19:8300 error="rpc error making call: Permission denied: anonymous token lacks permission 'service:write' on \"vault\". The anonymous token is used implicitly when a request does not specify a token."

I can’t find any documentation of service:write permission I haven’t given.

Token policy

node_prefix "" {
  policy = "write"
}
service_prefix "" {
  policy = "write"
}
service "" {
  policy = "write"
  intentions = "write"
}

I’ve also attracted the global-management policy to the token for good measures but still no luck.

1 Like

Just to clarify, I’ve set the agent token on the client to allow ACL authentication.

Jan 04 00:40:09 vault consul[21683]: ==> Starting Consul agent...
Jan 04 00:40:09 vault consul[21683]:                Version: '1.20.1'
Jan 04 00:40:09 vault consul[21683]:             Build Date: '2024-10-29 19:04:05 +0000 UTC'
Jan 04 00:40:09 vault consul[21683]:                Node ID: 'c821a29c-8ceb-7ae2-0749-eb0ee94700cd'
Jan 04 00:40:09 vault consul[21683]:              Node name: 'ip-172-18-100-17'
Jan 04 00:40:09 vault consul[21683]:             Datacenter: 'dc1' (Segment: '')
Jan 04 00:40:09 vault consul[21683]:                 Server: false (Bootstrap: false)
Jan 04 00:40:09 vault consul[21683]:            Client Addr: [127.0.0.1] (HTTP: 8500, HTTPS: -1, gRPC: -1, gRPC-TLS: -1, DNS: 8600)
Jan 04 00:40:09 vault consul[21683]:           Cluster Addr: 172.18.100.17 (LAN: 8301, WAN: 8302)
Jan 04 00:40:09 vault consul[21683]:      Gossip Encryption: true
Jan 04 00:40:09 vault consul[21683]:       Auto-Encrypt-TLS: true
Jan 04 00:40:09 vault consul[21683]:            ACL Enabled: true
Jan 04 00:40:09 vault consul[21683]:     ACL Default Policy: allow
Jan 04 00:40:09 vault consul[21683]:              HTTPS TLS: Verify Incoming: true, Verify Outgoing: true, Min Version: TLSv1_2
Jan 04 00:40:09 vault consul[21683]:               gRPC TLS: Verify Incoming: true, Min Version: TLSv1_2
Jan 04 00:40:09 vault consul[21683]:       Internal RPC TLS: Verify Incoming: true, Verify Outgoing: true (Verify Hostname: true), Min Version: TLSv1_2
Jan 04 00:40:09 vault consul[21683]: ==> Log data will now stream in as it occurs:
Jan 04 00:40:09 vault consul[21683]: 2025-01-04T00:40:09.897Z [WARN]  agent: skipping file /etc/consul.d/consul.env, extension must be .hcl or .json, or config format must be set
Jan 04 00:40:09 vault consul[21683]: 2025-01-04T00:40:09.898Z [WARN]  agent: "agent" token present in both the configuration and persisted token store, using the persisted token
Jan 04 00:40:09 vault consul[21683]: 2025-01-04T00:40:09.908Z [WARN]  agent.auto_config: skipping file /etc/consul.d/consul.env, extension must be .hcl or .json, or config format must be set
Jan 04 00:40:09 vault consul[21683]: 2025-01-04T00:40:09.926Z [INFO]  agent.auto_config: automatically upgraded to TLS
Jan 04 00:40:09 vault consul[21683]: 2025-01-04T00:40:09.928Z [INFO]  agent.client.serf.lan: serf: EventMemberJoin: ip-172-18-100-17 172.18.100.17

Hi @praveenprem,

The error you shared means that, someone (in your case vault) is trying to register a service named vault into Consul. The request is coming without a valid ACL token, and as a result, Consul agent has fallen back to the anonymous token, which doesn’t have the required permission to fulfil the request.

The right step in this case is to create a token with the required permissions, and configure your vault as shown below:

service_registration "consul" {
  ...
  token = "<the created token here>"
  ...
}

The minimal required policy for the token in the above case would be:

service "vault" {
  policy = "write"
}

You can read more about various builtin tokens in Consul here, which will help you understand why having an agent token didn’t work in this case.

Ref: ACL Tokens | Consul | HashiCorp Developer

I hope this helps!