Expected blocking query behavior for non-extant KV

Consul
1

Does anyone know what the expected behavior is for blocking queries against non-extant keys? I see that when an GET request is made for a key that doesn’t exist the result is a 404, but an X-CONSUL-INDEX header is returned.

Intuitively I would expect that if I make a blocking query with the returned index the request would block until either the key existed or the wait time expired (presumably returning another 404 with the same index). However, what I observe is when I issue a request with the ?index parameter for a non-extant key the 404 is returned immediately with a new index header.

I suppose it makes sense since a missing key has no “state” in which an index can be stored. In whatever case, is there any way to make blocking queries against keys that don’t exist? I’m trying to avoid spamming requests. My current solution is to sleep on a 404, but that means that if the key is created I’m stuck waiting the full sleep before detecting it.

2

Hi @jscaltreto,

Welcome!

I just tested this with both Consul 1.6.2 & 1.7.2, and am seeing the expected behavior.

$ curl --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
        --include $CONSUL_HTTP_ADDR/v1/kv/test-key
HTTP/1.1 404 Not Found
Vary: Accept-Encoding
X-Consul-Index: 6430285
X-Consul-Knownleader: true
X-Consul-Lastcontact: 0
Date: Tue, 31 Mar 2020 05:25:30 GMT
Content-Length: 0

The response hangs when I issue a subsequent query using the index returned in the initial request.

curl --header "X-Consul-Token: $CONSUL_HTTP_TOKEN" \
     --include $CONSUL_HTTP_ADDR/v1/kv/test-key\?index=6430285

The query only returns a response after I set the KV with consul kv put.

Would you mind sharing the version of Consul you are testing with, and a bit more detail regarding how you’re performing the blocking query?

Thanks.

3

Thanks @blake,

I think you’ve confirmed what I suspected, and that’s that I’m in desperate need of an upgrade! I’m on v1.4.0. The behavior I’m seeing is somewhat different:

$ time curl --include 'http://localhost:8500/v1/kv/foo-bar'
HTTP/1.1 404 Not Found
Vary: Accept-Encoding
X-Consul-Index: 91559726
X-Consul-Knownleader: true
X-Consul-Lastcontact: 0
Date: Tue, 31 Mar 2020 11:41:24 GMT
Content-Length: 0


real    0m0.013s
user    0m0.002s
sys     0m0.006s
$ time curl --include 'http://localhost:8500/v1/kv/foo-bar?index=91559726'
HTTP/1.1 404 Not Found
Vary: Accept-Encoding
X-Consul-Index: 91559765
X-Consul-Knownleader: true
X-Consul-Lastcontact: 0
Date: Tue, 31 Mar 2020 11:41:36 GMT
Content-Length: 0


real    0m0.013s
user    0m0.004s
sys     0m0.005s

Thanks for the confirming how this should be working; now I just need a window to effect an upgrade.

1 Like
4

Hi,

I just downloaded Consul 1.4.0, ran the agent in dev mode, and verified that blocking queries are working correctly in my environment.

I’m curious as to why you’re seeing different behavior. Could you run Consul in debug mode and verify that the HTTP request being received by the agent is not somehow being malformed?