Vault with native Mysql Replication

I have a two vault instances running on two separate servers each with its own mysql database(HA not enabled). Both mysql databases have master-master replication enabled on them. So changes on one vault(V1) instance for example adding a new secret is immediately reflected in the database(both V1 and V2 database have same number of rows after the change) of other vault instance(V2). But when I try using cli or api to get the secret on the other vault instance(V2) its not returned.
It only reflected after vault service is restarted on V2.

I need to know how I can change this behavior and make the secret immediately available on the second instance. restarting the service each time is not really an option.
Any help would be highly appreciated.

I think there is a mistake in your setup: how should the second Vault Instance know that the first Vault Instance has entered something? There is generally a kind of “trigger” missing (one of which seems to be a restart and “re-reading” the database entries).

So this trigger in case of Vault is restart? Is there any other way to trigger it to read entries from database or to atleast inform vault that database entries has changed. I tried reading the docs but couldn’t find anything related.

Disabling cache(disable_cache= true) in the config helped. Now every time it reads from the database.
Didn’t find this option earlier.

1 Like

I hope you’re doing this in a test/lab and not in production. Aside from being completely unsupported it’s also likely that it’ll bite you hard at the worst possible time. MySQL’s replications schemes are contradictory to how Vault does it’s locking and updates. It’ll work fine until you hit a (I imagine) mid-size level of load and it’ll go out of sync and your nodes may either seal or throw all sorts of weird errors – which may even lead to data loss if you have that sort of write load going on.

1 Like