I’ve been using nomad-vector-logger in production since sometime. It’s working well for my usecase that I described above.
The daemon runs alongside vector
(same group, different tasks so that they can share the same alloc_dir
). It periodically fetches a list of allocations on the current node with their filepath and templates out a vector
config. The user can provide their own extra vector
templates to configure extra transformation (JSON/logfmt parsing/splitting namespaces etc) and then finally route to sink providers (Clickhouse/Elastic/S3 etc).
When the allocs are stopped, this daemon also removes them from the templated file after a configurable delay period, to ensure that Vector has finished processing all the remaining logs.
If anyone is facing an issue with exec/raw_exec tasks logging, check this out Feedback welcome!