no programmatic way to get discovery token ca hash

[kapilt]

if your creating a token, via kubeadm token create .. or any other mechanism, there is no way to get the token ca certificate hash outside of trying to parse the human readable output (and thus non stable) of kubeadm init. There should a machine readable way of creating tokens with ca hashes, else one is forced to use --discovery-token-unsafe-skip-ca-verification even if there is desire to do the right thing (tm). Atm the alternatives are either parsing the likely to be changed human read output of kubeadm init or reimplementing the hash calculation of kubeadm over the ca. the later wouldn't be so bad if it didn't require reading the source as its not a simple as ie none of the outputs of cd /etc/kubernetes/pki && sha256sum * show the same sha as that output by kubeadm init.

[kad]

Have you tried recent version of kubeadm ?

# kubeadm token create --print-join-command
kubeadm join --token 5d2dc8.3e93f8449167639b 10.0.2.66:6443 --discovery-token-ca-cert-hash sha256:44a68d4a2c2a86e05cc0d4ee8c9c6b64352c54e450021331c483561e45b34388

[ConorNevin]

its still not great - anyone trying to this use this programmatically will have to parse the command in order to extract both the token and discovery token hash. It would be better if we could specify an output form like json so we can parse it safely

[kapilt]

thanks, i didn't realize that there was a separate/new command for this. I agree, it would be nicer to have this in json. my previous workflow for 1.6 and 1.7, was kubeadm token generate, and then reuse that value when init'ing and join'ing, the kubeadm token generate was machine readable single value line delimited, but its now effectively a dead command since it lacks the cert hash (and not per se extensible due to that format). re why machine readable, i have additional parameters to pass for the join and some that i need to replace like dns for the ip.

[thecatwasnot]

Yup, I had to dump the join-command to a file, and then use sed to add an ignore-preflight-errors flag, not awful but not particularly friendly either.

[timothysc]

So there is an ongoing effort to trim up phases to make it more programmatic and de-dupe the flags, this is only one user story. @fabriziopandini do you have a parent issue for this somewhere?

[fabriziopandini]

@timothysc
The issue for graduating phases is #454, but it is kind of outdated and might be it makes sense to start with a new one as soon as the upcoming KEP is approved.

However, with regards to this issue, IMO we should open a dedicated umbrella issue to tackle the demand for machine readable output in a consistent way across all commands, because the request spans from kubeadm init/phases to kubeadm token and also kubeadm upgrade (see #494).

WDYT? if it is ok for you, I will open the new umbrella issue ...

[davemac30]

I just hit this issue myself. I was able to calculate the sha256sum of the ca cert's public key using the following:

$ openssl x509 -in /etc/kubernetes/pki/ca.crt -noout -pubkey | openssl rsa -pubin -outform DER 2>/dev/null | sha256sum | cut -d' ' -f1

Not pretty...

[Pinaute]

I use the regular expressions. I tested with Ansible.

- hosts: localhost
  tasks:
    - shell: kubeadm token create --print-join-command
      register: results
    - debug:
        var: results.stdout
    - set_fact:
        token: "{{ results.stdout | regex_search(regexp, '\\2') | first }}"
      vars:
        regexp: '([^\s]+\s){4}([^\s]+)'
    - debug:
        var: token
    - set_fact:
        hash: "{{ results.stdout | regex_search(regexp, '\\2') | first }}"
      vars:
        regexp: '([^\s]+\s){6}([^\s]+)'
    - debug:
        var: hash

Result:

TASK [debug] *******************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "results.stdout": "kubeadm join 192.168.1.2:6443 --token 3a0fje.octau87o6x30dz8i --discovery-token-ca-cert-hash sha256:1fd18093fb89b364879d5667b7ec84fd24171c30de0070deb6a3801b54a0f85c"
}

TASK [set_fact] ****************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [debug] *******************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "token": "3a0fje.octau87o6x30dz8i"
}

TASK [set_fact] ****************************************************************************************************************************************************************************************************
ok: [localhost]

TASK [debug] *******************************************************************************************************************************************************************************************************
ok: [localhost] => {
    "hash": "sha256:1fd18093fb89b364879d5667b7ec84fd24171c30de0070deb6a3801b54a0f85c"
}

[timothysc]

Closing as there are several workarounds.

[graykode]

This is not recommend but, In https://github.com/cablespaghetti/kubeadm-aws/blob/master/worker.sh#L45 --discovery-token-unsafe-skip-ca-verification also work

[SeryioGonzalez]

on Azure, using tags for worker and master resources:
rg is a variable with the RG name

masterPrivateIp=$(az network nic list -g $rg --query "[?tags.module == 'k8smasters'].ipConfigurations[0].privateIpAddress" -o tsv)
tokenId=$(ssh $masterIp "kubeadm token list | grep -v TOKEN | cut -d' ' -f1")
tokenSHA=$(ssh $masterIp "openssl x509 -in /etc/kubernetes/pki/ca.crt -noout -pubkey | openssl rsa -pubin -outform DER 2>/dev/null | sha256sum | cut -d' ' -f1")

joinCommand="kubeadm join $masterPrivateIp:6443 --token $tokenId --discovery-token-ca-cert-hash sha256:$tokenSHA"

for ip in $(az network public-ip list -g $rg --query [?tags.module == 'k8sworkers'].ipAddress -o tsv)
do
ssh $ip $joinCommand
done

[bmitzkus]

just in case somebody stumbles upon this, in ansible you can simplify @davemac30's approach significantly:

- name: print certificate info
  community.crypto.x509_certificate_info:
    path: /etc/kubernetes/pki/ca.crt
  register: cert
- name: debug
  debug:
    msg: "sha256:{{ cert.public_key_fingerprints.sha256 | replace(':', '') }}"

[brianthelion]

Note that /etc/kubernetes/pki/ca.crt is also stored as a secret in the kube-system namespace:

NAME=$(kubectl get secrets --no-headers -o custom-columns=":metadata.name" -n kube-system | fgrep root-ca-cert-publisher-token-)
kubectl get secrets/${NAME} -n kube-system -o json | jq -r '.data["ca.crt"] | @base64d'