Importing public keys

[Ovomaltine85]

Hi!

Thank you for this great library and the effort you invested. I tested a lot and it really works well.

I've one question regarding the derivation of a shared secret when using EC. How can public keys be imported, which are part of a x509 certificate? I read in the documentation something about a function called import_key(), but I did not found it. Is it already implemented?

Unfortunately the examples and test file always generate new keys, but I need to get the public key from an existing x509.

Thank you!

[danni]

Oh yes, the functions are in pkcs11.util.x509 and pkcs11.util.rsa. I'd love a patch to update the documentation.

X.509 handling is a bit weak at the moment, it will decode the certificate into an object, but there's no function to decode the public key object and set all of its properties based on key type (e.g. RSA, DSA, ECDSA). This would be pretty simple to do based on what you see in pkcs11.util, it just needs writing.

[danni]

Realised the docs for util functions weren't generating (fixed it), here's the certificate import: http://python-pkcs11.readthedocs.io/en/latest/api.html#module-pkcs11.util.x509

If you felt able to submit a PR for importing the key from a certificate object, here's how we decode the ASN.1 object for an X.509 certificate: https://github.com/danni/python-pkcs11/blob/master/pkcs11/util/x509.py#L29 -- you want to pull out the subjectPublicKeyInfo field, which contains the algorithm (as an OID + params) and the key itself, which is separately ASN.1 encoded as a bit string (hooray).

You'd then need a mapping from algorithm OIDs (e.g. RSA) to the existing decode utils within python-pkcs11 which you could pass subjectPublicKey to.

[danni]

For ECDH, the you just need to decode the the certificate and pass the subjectPublicKey as Attribute.EC_POINT. PKCS#11 expects this valid X9.62 DER encoded (I think they just gave up on adding parameters when they got to EC). See http://python-pkcs11.readthedocs.io/en/latest/applied.html#elliptic-curve.

When deriving shared keys, the other user's EC_POINT (still X9.62 DER encoded) is passed as a mechanism parameter. See http://python-pkcs11.readthedocs.io/en/latest/applied.html#ec-diffie-hellman

[Ovomaltine85]

Thank you for your reply. I finally succeeded with using the packages of cryptography and pyasn1. It might appear a bit awkward, but it works for me. This is what I did so far:

Importing the needed modules:
from cryptography import x509 from cryptography.hazmat.backends import default_backend from pyasn1.codec.der.decoder import decode as der_decoder from pyasn1.type.univ import BitString

Reading the x509 certificate:
cert = x509.load_pem_x509_certificate(open(my_cert, 'rb').read(), default_backend())

Retrieving the public key from certificate:
pubdata = cert.public_key().public_bytes(cryptography.hazmat.primitives.serialization.Encoding.DER,cryptography.hazmat.primitives.serialization.PublicFormat.SubjectPublicKeyInfo)

To parse the ASN.1 information I used the following code from
https://stackoverflow.com/questions/21954512/parse-ec-public-key-with-pyasn1
With that I'm able to get the binary code of the EC-Point in string representation:

class curve(pyasn1.type.univ.Sequence): componentType = pyasn1.type.namedtype.NamedTypes( pyasn1.type.namedtype.NamedType('type',pyasn1.type.univ.ObjectIdentifier()), pyasn1.type.namedtype.NamedType('name',pyasn1.type.univ.ObjectIdentifier()) )

class EcPublicKey(pyasn1.type.univ.Sequence): componentType = pyasn1.type.namedtype.NamedTypes( pyasn1.type.namedtype.NamedType('curve', curve()), pyasn1.type.namedtype.NamedType('publickey', pyasn1.type.univ.BitString()) )

der_code,rest = der_decoder(pubdata, asn1Spec = EcPublicKey())

Finally I had to prepend an identifier for the regarding uncompressed elliptic curve (0x0461):
str_code = '10001100001%s' % (str(der_code['publickey']))

As str_code contains now the correct EC point, I converted it to byte code and used it as other_point parameter for python-pkcs11: derive_key().
Although this solution only fits to a certain curve, it works so far for me.

[danni]

Do you have an example key/code, I can clean this up and include utilities/documentation for you.

[Ovomaltine85]

What exactly do you mean by an example key/code?

[danni]

I meant an example certificate.

…

On Wed, 28 Jun 2017 at 16:46, Ovomaltine85 ***@***.***> wrote: What exactly do you mean by an example key/code? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAH5wGGWQhxazcaCz7l7B8vUhCSzCfu2ks5sIfa0gaJpZM4OEW0Z> .

[Ovomaltine85]

Sorry for late reply. Unfortunately I cannot post here the certificate I used for testing.

But the workaround I suggested should work with any X.509 certificate which is PEM-encoded and is based on a secp384r1 Elliptic Curve.

[danni]

I've added a new function to master called pkcs11.util.x509.decode_x509_public_key which has tests for importing keys from RSA, DSA and ECDSA certificates. Let me know if this works for you.

[danni]

http://python-pkcs11.readthedocs.io/en/latest/applied.html#importing-exporting-keys

[Ovomaltine85]

I tested your fix and got the following result.

At first I downloaded the current master from github, because the enhancement is not part of the current release. I installed the master with python3 setup.py build and got the following error:
`Traceback (most recent call last):
File "setup.py", line 65, in
test_suite='tests',
File "/usr/lib/python3.4/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/usr/local/lib/python3.4/dist-packages/setuptools/dist.py", line 323, in init
_Distribution.init(self, attrs)
File "/usr/lib/python3.4/distutils/dist.py", line 280, in init
self.finalize_options()
File "/usr/local/lib/python3.4/dist-packages/setuptools/dist.py", line 392, in finalize_options
ep.load()(self, ep.name, value)
File "/home/thomas/Downloads/python-pkcs11-master/.eggs/setuptools_scm-1.15.6-py3.4.egg/setuptools_scm/integration.py", line 22, in version_keyword
File "/home/thomas/Downloads/python-pkcs11-master/.eggs/setuptools_scm-1.15.6-py3.4.egg/setuptools_scm/init.py", line 119, in get_version
File "/home/thomas/Downloads/python-pkcs11-master/.eggs/setuptools_scm-1.15.6-py3.4.egg/setuptools_scm/init.py", line 97, in _do_parse
LookupError: setuptools-scm was unable to detect version for '/home/thomas/Downloads/python-pkcs11-master'.

Make sure you're either building from a fully intact git repository or PyPI tarballs. Most other sources (such as GitHub's tarballs, a git checkout without the .git folder) don't contain the necessary metadata and will not work.

For example, if you're using pip, instead of https://github.com/user/proj/archive/master.zip use git+https://github.com/user/proj.git#egg=proj `

So I switched back to the last stable release (0.2.2) and installed it again with pip3. As the new function decode_x509_public_key is not part of 0.2.2, I manually copied the whole function to x509.py in the util folder.

I tested the new function and got the following error:
Traceback (most recent call last): File "derive_A.py", line 96, in <module> other_key = pkcs11.util.x509.decode_x509_public_key(pubdata) File "/usr/local/lib/python3.4/dist-packages/pkcs11/util/x509.py", line 37, in decode_x509_public_key x509, _ = der_decoder.decode(der, asn1Spec=Certificate()) File "/usr/local/lib/python3.4/dist-packages/pyasn1/codec/ber/decoder.py", line 908, in __call__ stGetValueDecoder, self, substrateFun File "/usr/local/lib/python3.4/dist-packages/pyasn1/codec/ber/decoder.py", line 358, in valueDecoder component, head = decodeFun(head, asn1Spec) File "/usr/local/lib/python3.4/dist-packages/pyasn1/codec/ber/decoder.py", line 908, in __call__ stGetValueDecoder, self, substrateFun File "/usr/local/lib/python3.4/dist-packages/pyasn1/codec/ber/decoder.py", line 358, in valueDecoder component, head = decodeFun(head, asn1Spec) File "/usr/local/lib/python3.4/dist-packages/pyasn1/codec/ber/decoder.py", line 915, in __call__ '%s not in asn1Spec: %s' % (tagSet, asn1Spec) pyasn1.error.PyAsn1Error: [0:0:6] not in asn1Spec: TagMap: posMap: [0:0:2] -> CertificateSerialNumber, [0:0:2]+[128:32:0] -> Version,

The variable pubdata has been initialisied as I wrote some posts above by the cryptography package:
cert = x509.load_pem_x509_certificate(open(my_cert, 'rb').read(), default_backend()) pubdata = cert.public_key().public_bytes(Encoding.DER,PublicFormat.SubjectPublicKeyInfo)

pubdata should therefore contain the DER encoded value of the x509 certificate which can be found by my_cert in my example.

How do you produce the parameter (pubdata) for decode_x509_public_key if you only got a X.509 certificate that is PEM encoded by default?

[danni]

Ah, so you're passing just the SubjectPublicKeyInfo, not the whole certificate?

decode_x509_public_key takes an entire X.509 certificate in DER-encoding (PEM is base64'ed DER, with headers). If you've already extracted the SubjectPublicKeyInfo you can pass this to pkcs11.util.ec.decode_ec_public_key.

[Ovomaltine85]

Yeah, that works better if I pass the certificate directly in DER format to the function. So far it works as it gives me now the EC_POINT attribute. But I cannot pass this value directly to the derive_key() as other_point parameter, as the curve information, e.g. x0461 (for secp384r1) is not prepended to the EC_POINT, so it still has to be added manually. Is there the possibility to automate it?

Thank you!

[danni]

It should give you the EC_POINT and EC_PARAMS. As far as I'm aware, the first couple of bytes of the point indicate whether it's compressed or uncompressed, but the point does not include the domain parameters (which is why they're encoded in a separate structure).

What HSM are you running against? If it's SoftHSM v2.2 it had a mistake in it's encoding/decoding of EC_POINT on some Linux, that SoftHSM v2.3 appears to fix (not entirely sure on what was going on there, but I was getting substrate underrun issues).

Also make sure you have the latest python-pkcs11, because I cleaned up some of my encoding handling to make sure I was in the right form, with the right number of DER wrappers (urgh).

I'll write a test of doing ECDH against an external crypto platform to make sure there are no more secret interchange issues.

[danni]

It turns out the standard actually explains the disparity:

The encoding in V2.20 was not specified and resulted in different implementations choosing different encodings. Applications relying only on a V2.20 encoding (e.g. the DER variant) other than the one specified now (raw) may not work with all V2.30 compliant tokens.

I'm adding a flag to decode_ec_public_key to allow you to use a raw or DER-encoded EC_POINT as required.

[danni]

Here's what I've added to the tests:

@requires(Mechanism.ECDH1_DERIVE)
def test_ecdh(self):

    # Retrieve our keypair, with our public key encoded for interchange
    alice_priv = self.session.get_key(key_type=KeyType.EC,
                                      object_class=ObjectClass.PRIVATE_KEY)
    alice_pub = self.session.get_key(key_type=KeyType.EC,
                                     object_class=ObjectClass.PUBLIC_KEY)
    alice_pub = encode_ec_public_key(alice_pub)

    from cryptography.hazmat.backends import default_backend
    from cryptography.hazmat.primitives.asymmetric import ec
    from cryptography.hazmat.primitives.serialization import \
        Encoding, PublicFormat, load_der_public_key

    # Bob generates a keypair, with their public key encoded for
    # interchange
    bob_priv = ec.generate_private_key(ec.SECP256R1,
                                       default_backend())
    bob_pub = bob_priv.public_key().public_bytes(
        Encoding.DER,
        PublicFormat.SubjectPublicKeyInfo,
    )

    # Bob converts Alice's key to internal format and generates their
    # shared key
    bob_shared_key = bob_priv.exchange(
        ec.ECDH(),
        load_der_public_key(alice_pub, default_backend()),
    )

    key = alice_priv.derive_key(
        KeyType.GENERIC_SECRET, 256,
        mechanism_param=(
            KDF.NULL, None,
            # N.B. it seems like SoftHSMv2 requires an EC_POINT to be
            # DER-encoded, which is not what the spec says
            decode_ec_public_key(bob_pub, encode_ec_point=Is.softhsm2)
            [Attribute.EC_POINT],
        ),
        template={
            Attribute.SENSITIVE: False,
            Attribute.EXTRACTABLE: True,
        },
    )
    alice_shared_key = key[Attribute.VALUE]