How to share sensitive files between tasks but hide from UI

krundru · November 8, 2022, 3:06pm

Hi Team,

We have a TaskGroup with two or more tasks. One is responsible for generating certificates using an internal Certificate-gen client and storing them in the alloc/data directory. My other task (say Nginx) consumes these certs for TLS Termination. This setup works well, but given the sensitive nature of certs, we don’t want to display the certs/contents in UI.

So, are there any options for sharing secrets like files between tasks in a group

thanks

seth.hoenig · November 9, 2022, 6:54pm

Hi @krundru, with ACLs enabled you should be able to craft ACL tokens for users without the permissions necessary for reading task directories / running commands in allocations.

krundru · November 11, 2022, 6:06am

Thanks, @seth.hoenig, for the response, but I think using ACL, we can restrict viewing all alloc files but not a specific file/folder, correct?

I definitely don’t want to hide all alloc files but only specific files like certs.

krundru · November 15, 2022, 5:11am

It looks like there is no option in nomad to share sensitive files between tasks, and the secret directory is tightly coupled with Hashicorp tools
We need to have a direct secret directory on the parent alloc, and it should be hidden from UI/API like task level secret folder.

ACL can only hide the entire FS but not a specific file, and the hiding file system can result in poor developer experience.

github.com

hashicorp/nomad/blob/main/nomad/client_fs_endpoint.go#L127-L134


      
          	// Check namespace filesystem read permissions
          	allowNsOp := acl.NamespaceValidator(acl.NamespaceCapabilityReadFS)
          	aclObj, err := f.srv.ResolveToken(args.AuthToken)
          	if err != nil {
          		return err
          	} else if !allowNsOp(aclObj, alloc.Namespace) {
          		return structs.ErrPermissionDenied
          	}