Vault cluster initialization with integrated raft storage failing

I am trying to initialize vault cluster with one leader and 2 followers. Using retry_join stanza to provide the leaders in each of the configs, when I execute vault operator init command it only initializes leader when trying to unseal vault on the followers get error saying ‘vault is not initialized’