Now pki is in some weird state existing but I can’t list or add anything.
vault secrets enable pki
Error enabling: Error making API request.
URL: POST http://localhost:8400/v1/sys/mounts/pki
Code: 400. Errors:
* path is already in use at pki/
Creating a new root ca also fails:
vault write pki/root/generate/internal common_name="domain.net" issuer_name="root-2023" ttl=87600h > root_2023_ca.crt
Error writing data to pki/root/generate/internal: Error making API request.
URL: PUT http://localhost:8400/v1/pki/root/generate/internal
Code: 404. Errors:
* no handler for route "pki/root/generate/internal". route entry is tainted.
Browsing through web UI somewhat works and if I click configuration under PKI secrets it works but going to configure I get error:
Error
Attempted to handle event `loadingData` on <secret-engine:pki> while in state root.deleted.uncommitted. Called with { _id: 10999, _label: "Fetching secret-engine' with id: pki", _state: undefined, _result: undefined, _subscribers: [ ] }.
Any help would be greatly appreciated. Also I need to explicitly say this is production environment … so I need to fix this without losing any data mainly secrets store or any other integrations.
Vault PKI secrets engines can store a lot of data in the Vault storage backend.
It looks to me like your original vault secrets disable pki timed out and was aborted part way through, leaving broken remnants behind at pki/ in Vault.
I would first attempt to retry the vault secrets disable pki and see if it makes further progress, and eventually succeeds.
You’re going to need to get the broken secrets engine removed from pki/ before you can successfully recreate a fresh one at the same path.
I would gladly recreate pki secrets under pki/ but whenever I try to run vault secrets disable pki returns with error: Error disabling secrets engine at pki/: context deadline exceeded.
It seems I can’t properly disable/delete PKI secrets. Any other way to force remove it?
I managed to create pki-new/ store but not sure how I can convince vault to use that store for certificate generation.
Ideally I would like to have vault certificates under pki/ store.
It is possible that it is making some amount of progress each time.
Which Vault storage type are you using? (That affects how easy or hard it would be to confirm whether that is the case.)
Meanwhile, one option would be to try just leaving vault secrets disable pki running in a loop automatically, and see if it eventually succeeds.
Technically yes, but it would involve shutting Vault down, putting it into recovery mode, and performing manual manipulation of internal JSON structures. You really don’t want to go there until all other options are exhausted.
That is entirely up to the clients making requests to Vault - they would have to call URLs containing pki-new instead of pki.
I am not sure if this will work with a partially broken secrets engine, but it’s worth a try, and if it does work, it would free you to get your new pki/ set up and working, and worry about fully cleaning up the old one later.
Hey vault secrets move worked. Now I can recreate PKI with pki/ path.
I guess I can live with a broken unused secrets store. We’re moving to new vault later on this year so leaving stalled/broken path until we migrate shouldn’t be an issue.
Thank you for a working suggestion. I’m just going to recreate the initial pki/ until we migrate to new deployment.