"Not Capable" errors after getting "3.0 version"

[hongchaodeng]

When doing

curl http://${ETCD_HOST}:${ETCD_PORT}/version

It gives output:

{"etcdserver":"3.0.3","etcdcluster":"3.0.0"}

Nonetheless, client still gets "not capable" error. The client is fine after retrying, which means etcd cluster has negotiated 3.0 version thereafter.

This "/version" endpoint is useful for bootstrapping servers after etcd.

[hongchaodeng]

@heyitsanthony

[heyitsanthony]

A brief analysis of what's going wrong: the version http handler is returning the most recent semver but the enabled map is only updated on a 500ms periodic basis. Since the grpc interceptor does the capability check on the enabled map, there is up to a 500ms window where etcd reports 3.0.0 capabilities but will reject 3.0.0 requests. In other words, the value exposed in the version endpoint should only be updated when/after the enabled capability map is updated, not before.