Add new check for weak RSA keys and padding modes

[presidentbeef]

Fixes #1736

Warns about these patterns:

• OpenSSL::PKey::RSA.new or OpenSSL::PKey::RSA.generate with key size < 2048 bits
• OpenSSL::PKey::RSA#public_encrypt, OpenSSL::PKey::RSA#public_decrypt, OpenSSL::PKey::RSA#private_encrypt, OpenSSL::PKey::RSA#private_decrypt with no padding mode specified (in which case it's PKCS1), with OpenSSL::PKey::RSA::PKCS1, or OpenSSL::PKey::RSA::NO_PADDING

[bdewater]

Thank you for working on this! Some feedback:

https://cwe.mitre.org/data/definitions/780.html is quite strong in calling anything other than OAEP padding secure, should we follow that recommendation? I hope nobody picks for example SSLV23_PADDING on purpose, but then again who knows if people don't know why PKCS1_PADDING is insecure.

The PR currently does not seem to detect calls using the generic PKey interface introduced in OpenSSL gem 3.0:

weak_rsa = OpenSSL::PKey.generate_key("RSA", rsa_keygen_bits: 1024)
weak_encrypted = weak_rsa.encrypt("data", rsa_padding_mode: "pkcs1")
weak_signature_digest = weak_rsa.sign("SHA256", "data", rsa_padding_mode: "pkcs1")
weak_signature_raw = weak_rsa.sign_raw(nil, "data", rsa_padding_mode: "pkcs1")

Edit: the new interface takes both Symbol and String-keyed hashes btw.

[presidentbeef]

Makes sense, I can add those in as well.

[presidentbeef]

Now warns on key size for OpenSSL::PKey.generate_key.

Warns on any padding mode other than OAEP.

Added padding checks for encrypt, decrypt, sign, verify, sign_raw, verify_raw.