Add SNI override for k8s 1.18 config file

[klizhentas]

kubectl added support for SNI overrides in 1.18: kubernetes/kubernetes#88769

we should update tsh to set the tls-server-name field in kubeconfig which should, theoretically, solve the problem of 600+ clusters solved in #3870 for tsh

probably requires updating kube proxy GetTLSConfigForClient to match logic in middleware

verify that solution solves the problem

[awly]

@klizhentas should we backport this to 4.4/4.3/4.2?

[awly]

@russjones @andrejtokarcik let's try to fix this soon-ish (before 5.1 is fully ready)
Let me know if you'd like to sync to get more details.

[deusxanima]

Will this fix apply only to tsh client and be supported across releases (so for example, a client running tsh client 4.2 w/ 5.1.2 root cluster)?

[awly]

Yes, the fix will apply to tsh only.
It will work for any client version we support (at this point, probably 4.3/5/6) and server versions as old as 4.2 if not older.

[awly]

so for example, a client running tsh client 4.2 w/ 5.1.2 root cluster

Users should not be using 4.2 clients against 5.1 servers. See https://goteleport.com/teleport/docs/admin-guide/#component-compatibility

[travelton]

Just to be clear, we are back porting this to 4.2, correct? We have a customer needing this in 4.2.

[awly]

4.2 is no longer a supported version of Teleport (see https://goteleport.com/docs/faq/#which-version-of-teleport-is-supported)
4.4 is the oldest supported version for now, until v7 comes out.

[awly]

Ugh, there's a wrinkle in implementing this: the k8s endpoint serving cert doesn't have the necessary SAN with a cluster name. So simply setting the same SNI value as we do when talking to the Auth API does not work (without disabling client-side TLS verification)

The "correct" way to solve this is to update the SANs we generate for k8s (and all other TLS endpoints) to contain the encoded cluster name. However, that's a bit of a migration. The clients can't set the SNI values until all servers get updated and rotate their serving certs. So backporting it will be a no-go.

Need to think through some other possibilities:

• use a more clever TLS validation that doesn't rely on SNI to figure out the cert issuer
• some hacky usage of an existing SAN to signal "i'm a user, only load the CA of the current cluster"
• check proxy version via the Ping endpoint and set SNI if it's a version containing the fix
• if >N CAs are loaded (like >100), only use the current cluster CA; this is most likely the root cluster so the calling user must have a cert from the root cluster

[awly]

The "fix" is here #6519
Assuming this is approved, backporting all the way to 4.2 should be easy

[awly]

Backport PRs:

• v6 Backport v6: kube: handle large number of trusted clusters in mTLS handshake #6666
• v5 Backport v5: kube: handle large number of trusted clusters in mTLS handshake #6667
• v4.4 Backport v4.4: kube: handle large number of trusted clusters in mTLS handshake #6668
• v4.3 Backport 4.3: kube: handle large number of trusted clusters in mTLS handshake #6669
• v4.2 Backport 4.2: kube: handle large number of trusted clusters in mTLS handshake #6670

[awly]

All backports merged except 4.3 (pending some unrelated test failures).