Can't read allocation logs — Path escapes the alloc directory

[ncg]

Hi all, I’m running a job with the Podman driver and I’m unable to view any allocation’s logs. The job runs without issue:

$ nomad alloc status c64bc458-b257-f0a2-2bae-5c11d302d0f3
ID                  = c64bc458-b257-f0a2-2bae-5c11d302d0f3
Eval ID             = eb408004
Name                = docs.example[0]
Node ID             = a5f5c480
Node Name           = nomad-client
Job ID              = docs
Job Version         = 0
Client Status       = running
Client Description  = Tasks are running
Desired Status      = run
Desired Description = <none>
Created             = 7h54m ago
Modified            = 7h53m ago
Deployment ID       = 1c6de703
Deployment Health   = healthy

Allocation Addresses:
Label  Dynamic  Address
*http  yes      10.10.20.4:5678

Task "server" is "running"
Task Resources:
CPU        Memory           Disk     Addresses
0/100 MHz  828 KiB/300 MiB  300 MiB  

Task Events:
Started At     = 2025-01-26T17:02:40Z
Finished At    = N/A
Total Restarts = 0
Last Restart   = N/A

Recent Events:
Time                       Type        Description
2025-01-26T12:02:40-05:00  Started     Task started by client
2025-01-26T12:02:29-05:00  Driver      Pulling image docker.io/hashicorp/http-echo:latest
2025-01-26T12:02:28-05:00  Task Setup  Building Task Directory
2025-01-26T12:02:28-05:00  Received    Task received by client

But trying to read the logs returns an error:

$ nomad alloc logs c64bc458-b257-f0a2-2bae-5c11d302d0f3  
Failed to read stdout file: error reading file: Unexpected response code: 500 (failed to list entries: Path escapes the alloc directory)

Within the client, I can see that the file /opt/nomad/alloc/c64bc458-b257-f0a2-2bae-5c11d302d0f3/alloc/logs/server.stdout.0 exists and contains the application’s logs.

Can anyone point me in the right direction for resolving this error?

[ncg]

I resolved this by changing my data directory from /opt/nomad to /var/opt/nomad. My agents are running on Fedora CoreOS, where /opt is a symlink to /var/opt. It seems like there’s some special handling of symlinks in the Nomad source code (nomad/helper/escapingfs/escapes.go at main · hashicorp/nomad · GitHub) which was where the error was coming from.