I have a Docker based app and have tried to get Docker Secrets, and then AWS Secrets Manager working with it. Neither seemed to work due to my container’s non-root configuration. Someone recommended looking into Vault. But before I do that I want to get some comfort that Vault can be used with Docker. Specifically my problem has been with getting Credentials. I think this is due to the non-root ownership in the container. Is Vault being used successfully with non-root Docker?
There shouldn’t be any issue with using containers where processes run as non-root for any of those three mechanisms. When using Docker swarm secrets you’d need to set the uid/gid/mode settings in your Docker Compose file (as the defaults only allow reading by the root user).
Both AWS Secrets Manager & Vault are API based, so the remote server knows nothing about the user you are running as. If you are using additional tools (e.g. vault agent) to fetch secrets to a mounted volume then as with Docker swarm secrets you’d need to configure that tool with the right permissions settings to allow any files to be read by your application.
thank you for your response. I have been obsessing over the lack of root user as causing the problem that I can’t seem to get the Credentials correctly so i can unlock Secrets Manager, when that was not the problem at all. It is actually this line that is failing:
$result = $client->getSecretValue([‘SecretId’ => $secretName,]); and I have set secretName to “mysql_password”. So the problem might be with my secrets_manager setup? I will play with it some more today before “throwing up my hands” and switching over to Vault.
I don’t know which client library you are using, but there should be ways to debug this. If it is throwing an error you should get some details about what it is, and normally you can enable debug logging to see the exact API calls and responses.
Are you sure you are passing the credentials to your application correctly (either access/secret keys or from attached roles) and have the secretsmanager:GetSecretValue (and possibly kms:Decrypt) permissions?
Switching from using SSM to Vault wouldn’t be intrinsically better (or worse) - you still need to call the various APIs via a client library and debug any coding/permission issues.
Are you sure you are passing the credentials to your application correctly
Well, I am sure that the call below is not giving an error code!
$client = new SecretsManagerClient([
'version' => 'latest',
'region' => 'us-east-1',
]);
$client is returning a structure and not an error code! So I am assuming that is not the problem. I did notice that $HOME is /home/richb201 and I don't
have a dir called that. I do have one called richb201-XPS-13-9370. I was wondering if I should change $HOME to be /home/richb201-XPS-13-9370 and then
stick .aws/credentials there? I am not sure of the Global implications of changing $HOME? But the fact that the SecretsManagerClient is not
returning an error code makes me think that $client is getting the credentials and that is not the issue.
>>secretsmanager:GetSecretValue
I was not aware that there are permissions needed for this call. I just call it like this with the $client returned from SecretsManagerClient().
Creating an object like that with a client library generally doesn’t perform any API calls (as the library hasn’t yet been asked to do anything). Therefore you’d be unlikely to see most permission errors until you actually try to do an action (get/set a secret for example).
All AWS APIs require the correct permissions attached to the role or user you are authenticated with (using a specific access/secret key or via credentials stored in the .aws/credentials file or attached to the EC2 instance or pod the application is running on.
If you don’t have the correct permissions I would expect the call to fetch a secret to fail. Normally some sort of error would be returned or exception thrown.
As I mentioned previously I would suggest enabling debug logging which should show the actual API calls being sent with the responses from AWS. That should give an indication of what is happening as well as where the failure is occurring.
enabling debug logging
any idea how to do this? I realized today that I don’t have aws cli in the container so configure wouldn’t work. Moreover the $HOME is not set in the container, and I read that $HOME is very important to reading credentials.
Thanks for your help. I managed to get past the “no error code” problem by running docker run --rm -it -v ~/.aws:/root/.aws amazon/aws-cli command which is now able to almost find the credentials.
thanks again…