Differentiating between creating and updating

Vault
1

From what I understand, creating and updating entities go through the same API. However, the API requires the token to have update capabilities for the path. This means that when updating a non-existent entity, the request will succeed.

This is where my problem starts. I want to be able to update entities, but prevent creating non-existent entities. There doesn’t seem to be a way to do this since the API doesn’t differentiate the two operations similar to how the create/update userpass user API works.

A simple existence check before performing the update is not sufficient since another client could delete the entity after I am able to update the entity, which is essentially a race condition.

2

I think my assumptions were wrong. It seems like there is an API that only updates existing entities which is what I should be using. However, this doesn’t change the issue with updating a non-existent entity using the original API mentioned.

3

It seems like if the user is created, content is returned in the response. However, if the user is updated, no content is returned in the response. Since the API documentation doesn’t indicate this, I am skeptical of making this assumption; however, it seems to be the best chance of determining if the API updated an existing user.

4

Hi, just curious as to why it’s important for you to make that distinction. Could you describe your use case in better detail?

5

Imagine a client can create an entity providing a name and email. Essentially an entity should be created with the name of name and email in the metadata. The request will look like the following:

curl \
    --header "X-Vault-Token: $vault_token" \
    --request POST \
    --data "{ \"name\": \"$name\", \"metadata\": { \"email\": \"$email\" }  }" \
    http://127.0.0.1:8200/v1/identity/entity

where vault_token is some token with appropriate policies.

Now, imagine multiple clients performing the same request but with different emails. Because the API doesn’t return an error if the entity already exists, the emails will be overwritten.

6

Ok, perhaps we should take a step back.

What exactly do you consider to be clients and why are they creating entities?

Are you developing a new authentication plug-in for Vault?

According to the Identity store documentation

“When a client authenticates via any credential backend (except the Token backend), Vault creates a new entity.”

“Vault Entity is used to count the number of Vault clients.”

And you can see that in fact the entity name must be of type string: entity-<UUID> to avoid collision across multiple clients. Edit: entity-<UUID> is just he default value but can actually be any string provided by the user.

What kind of authentication backend will your principals use to authenticate? Have you considered using one of the existing authentication backends?

7

I am essentially creating a simplified version of Vault’s API. This simplified version essentially performs one or more Vault requests to accomplish the goal of the operation. In this simplified API, one of the operations allows to easily create a user which creates an entity, a userpass user, and an alias to link the entity and userpass user. This allows users to login using userpass to get access to secrets in Vault.

Anyone who uses this simplified API is what I call clients. It has nothing to do with Vault. I have no say in how clients use the simplified API, so it must be as robust as possible.

When multiples of this simplified create user operation are performed, there is a concurrency issue because of how the creating entities API allows updating existing entities. This issue may never arise because of how rarely this operation would happen, but it is a possible issue nevertheless.

8

So according to this, you don’t need to manage entities via your facade API.
When a userpass user authenticates, Vault takes care of creating a new entity for them, if one doesn’t already exist.

9

Manually creating an entity allows putting it in a group, giving it aliases, and giving it policies all before the user can login.

Letting Vault create entities for me will prevent doing any of this since the entity doesn’t exist until someone logs in using an authentication method—and logging in as the userpass user is not a good solution for this since userpass is possibly one of multiple authentication methods that users can use.

10

So I guess you’ll have to rely on the following not to be set

  • id (string: <optional>) - ID of the entity. If set, updates the corresponding existing entity.

I’d assume, if a client POSTs to /identity/entity providing a name that hasn’t been used yet, the API returns the id of a new entity. The client then stores the id and any following requests containing that same id would be considered an update rather than a create operation.

If any other client tries to create a new entity with the same name without providing the id I would expect the API to throw an error, though I haven’t tested that.

11

Using the create an entity API but using an existing name without providing an id does not throw an error. Instead, it is updated.

Running the following script will easily reproduce this (jq is used for nice output):

#!/bin/bash

vault_token='<insert_token>'

echo "Create an entity that is unique"
curl -s \
    --header "X-Vault-Token: $vault_token" \
    --request POST \
    --data '{ "name": "some_name", "metadata": {"meta": "data"} }' \
    http://127.0.0.1:8200/v1/identity/entity | jq
echo -e "Done\n"

echo "Read the created entity"
curl -s \
    --header "X-Vault-Token: $vault_token" \
    http://127.0.0.1:8200/v1/identity/entity/name/some_name | jq
echo -e "Done\n"

echo "Create an entity but using an existing name"
curl -s \
    --header "X-Vault-Token: $vault_token" \
    --request POST \
    --data '{ "name": "some_name", "metadata": {"uh": "oh"} }' \
    http://127.0.0.1:8200/v1/identity/entity | jq
echo -e "Done\n"

echo "Read the entity to check if it was updated"
curl -s \
    --header "X-Vault-Token: $vault_token" \
    http://127.0.0.1:8200/v1/identity/entity/name/some_name | jq
echo -e "Done\n"

echo "Cleanup"
curl \
    --header "X-Vault-Token: $vault_token" \
    --request DELETE \
    http://127.0.0.1:8200/v1/identity/entity/name/some_name
echo "Done"

This script results in following output (data may vary):

Create an entity that is unique
{
  "request_id": "77ec0f9a-5d43-adc4-32a3-35562742ff48",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "aliases": null,
    "id": "2cc21b11-bcf7-f5a2-89c5-a92358e7e313",
    "name": "some_name"
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}
Done

Read the created entity
{
  "request_id": "4364fe86-d309-8aba-ef38-6b0ceed9df84",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "aliases": [],
    "creation_time": "2023-03-22T19:13:38.155405615Z",
    "direct_group_ids": [],
    "disabled": false,
    "group_ids": [],
    "id": "2cc21b11-bcf7-f5a2-89c5-a92358e7e313",
    "inherited_group_ids": [],
    "last_update_time": "2023-03-22T19:13:38.155405615Z",
    "merged_entity_ids": null,
    "metadata": {
      "meta": "data"
    },
    "name": "some_name",
    "namespace_id": "root",
    "policies": []
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}
Done

Create an entity but using an existing name
Done

Read the entity to check if it was updated
{
  "request_id": "9d505bb4-88b4-52e1-9852-3ea43f869da6",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "aliases": [],
    "creation_time": "2023-03-22T19:13:38.155405615Z",
    "direct_group_ids": [],
    "disabled": false,
    "group_ids": [],
    "id": "2cc21b11-bcf7-f5a2-89c5-a92358e7e313",
    "inherited_group_ids": [],
    "last_update_time": "2023-03-22T19:13:38.219235824Z",
    "merged_entity_ids": null,
    "metadata": {
      "uh": "oh"
    },
    "name": "some_name",
    "namespace_id": "root",
    "policies": []
  },
  "wrap_info": null,
  "warnings": null,
  "auth": null
}
Done

Cleanup
Done

Note how no content was returned in the update request and the metadata field was updated from {"meta": "data"} to {"uh": "oh"}.

12

:thinking:

In that case, I’d probably create an enhancement request or even submit a PR with this change of behavior.

I’m still not quite sure if that’s a bug or a feature.

I can imagine that being initially designed as a way to abstract users from the underlying auth methods, Vault Entities rely on the fact that clients (in the Vault sense) are trusted because they have to be authenticated first.

I can see how in your case, that’s not applicable though.

13

This settles the question (bug or feature):

And this confirms your suspicion: