Vault - Monitoring best practises - how you do it?

Vault
1

Hello Everyone,

I hope you’re all doing well!

I have a general question about how you monitor your Vault infrastructure in your organization.

In our case, we’ve implemented two main features in Vault:

  • The classic Key-Value Store
  • The PKI

We’re curious to know what metrics you monitor for your Vault setup and which tools you use for that purpose.

For example, with our PKI, we’d like to monitor certificate expiration dates and set up proactive alerts. Unfortunately, this isn’t a built-in feature in Vault. How do you handle this in your environment?

Looking forward to hearing your thoughts and best practices!

2

Are you able to leverage Vault Agent?

Instead of being reactive to alerts, you can be proactive and have Vault Agent automatically regenerate your certificates.

For a PKI use case, Vault Agent would render your certificates, monitor the TTL, regenerate the certificates (before they expire), and reload your application.

Vault Agent Config (snippet):

...
template {
  source      = "/etc/vault.d/templates/pki.ctmpl"
  destination = "/opt/webapp/tls/cache"
  perms       = "600"

  exec = {
    command = "..." # your reload/restart command here
  }
}
...

Vault Agent Template:

{{- with pkiCert "pki/issue/demo" "common_name=webapp.example.com" "ttl=30d" -}}
{{ .Data.Key }}{{ .Data.Cert }}{{ .Data.CA }}
{{- .Key | writeToFile "/opt/webapp/tls/server.key" "vault" "www-data" "640" }}
{{- .Cert | writeToFile "/opt/webapp/tls/server.pem" "vault" "www-data" "644" "newline" }}
{{- .CA | writeToFile "/opt/webapp/tls/server.pem" "vault" "www-data" "644" "append" }}
{{- end -}}
3

To answer your original question regarding monitoring of Vault. You should capture the following items for true Vault observability. Ideally, forwarding/storing the logs within a centralized observability solution which will enable you to build custom dashboards, reports, and alerts.

  • Vault Operational Logs
  • Vault Audit Logs
  • Vault Telemetry
  • Host Metrics

Checkout these resources:

Example Diagram:

image
image1384×948 42.7 KB