ext/openssl/pkcs7 signing fails with EC keys. This happens because the EC keys do not respond to key.private? method which is used to check that the key is a private key. Aliasing the instance method OpenSSL::PKey::EC.private_key? as OpenSSL::PKey::EC.private? fixes this problem and allows EC keys to be used for EC key signing.
thanks for your analysis! You are right, there are several incosistencies when comparing EC's implementation to the PKey interface as it is implemented in RSA or DSA. I actually opened #6567 some time ago, collecting several of these inconsistencies. Fixing them completely will take some refactoring, but I'll certainly consider your pull request for that - thanks a lot for your work!
Please feel free to skim through the issues I opened myself, I'd appreciate your input!
It's only a one liner aliasing public_key? to public? but I did it partly to check that nothing disastrous happened if that change were made.
RE: the PKey interface I agree that the present situation does seem problematic. Looking at the issues you have collected there it definitely seems like there are a lot of inconsistancies One particular area I've experienced is the public key interface several methods assume that a public key will have the same class as private keys. I particularly encountered this when writing a unit test for pkcs7 signing with EC keys after applying the patch above.
The current pkcs7 unit tests dynamically construct x509 certificates from precomputed keys feeding the keys and desired attributes to a create_cert utility function in test/openssl/utils.rb. However substituting in an EC key for an RSA/DSA key does not work because the function expects the response to key.public_key to conform to the PKey specification. Particularly it expects that the output of key.public_key can be used as a valid argument to key.public_key= which is not true in the case of EC keys. In order to create a working test the most expedient solution was to add an explicit test to the certificate issuing function.
This had the effect of changing this simple statement:
cert.public_key = key.public_key