General recommendations for logging

I’ve been using nomad-vector-logger in production since sometime. It’s working well for my usecase that I described above.

The daemon runs alongside vector (same group, different tasks so that they can share the same alloc_dir). It periodically fetches a list of allocations on the current node with their filepath and templates out a vector config. The user can provide their own extra vector templates to configure extra transformation (JSON/logfmt parsing/splitting namespaces etc) and then finally route to sink providers (Clickhouse/Elastic/S3 etc).

When the allocs are stopped, this daemon also removes them from the templated file after a configurable delay period, to ensure that Vector has finished processing all the remaining logs.

If anyone is facing an issue with exec/raw_exec tasks logging, check this out :slight_smile: Feedback welcome!

1 Like