[Nomad] How to Clean /var/lib/docker/overlay2 or limit the allocation logs stream

Nomad
1

Hi Nomad team,
We have a case that we need to run job that have huge docker images… and we use the docker driver

and we found that if :

  • there is many allocation in particular node using the images…

its make /var/lib/docker/overlay2/ generate huge disk usage, and make the disk usage 100%,

and we found out that its because of the allocation logs, generate many log, but we need it, especially the latest logs one…
but when the disk got full… its make the logs stopped, do we have a way to rotate it automatically? if disk full and allocation is not in terminal state, just rotate the old logs, so we can still make the allocation continue running with latest logs?

2

Hi @petrukngantuk1,

Thanks again for using Nomad.

Does the logs stanza meet your use case? This seems like exactly what it was designed for.

  • Derek