SecKeyCreateFromData fails on 10.9 with -2147415792

Question

I have a code that constructs an RSA public key on MacOS with the help of the security framework. This works fine on 10.11/10.12/10.13, but today I found that this fails on 10.9. Below is the constructor of the class that wraps the key:

CRSAPublicKey(const unsigned char* pExponent, const std::size_t nExponentSize, const unsigned char* pModulus, const std::size_t nModulusSize)
{
    static const SecAsn1Template kRsaPublicKeyTemplate[] = {
        { SEC_ASN1_SEQUENCE, 0, NULL, sizeof(ASN1_RSA_PUBLIC_KEY) },
        { SEC_ASN1_INTEGER, offsetof(ASN1_RSA_PUBLIC_KEY, m_Modulus), 0, 0 },
        { SEC_ASN1_INTEGER, offsetof(ASN1_RSA_PUBLIC_KEY, m_Exponent), 0, 0 },
        { 0, 0, 0, 0 },
    };
    ASN1_RSA_PUBLIC_KEY Asn1Key;
    Asn1Key.m_Modulus.Data = const_cast<unsigned char*>(pModulus);
    Asn1Key.m_Modulus.Length = nModulusSize;
    Asn1Key.m_Exponent.Data = const_cast<unsigned char*>(pExponent);
    Asn1Key.m_Exponent.Length = nExponentSize;
    MacOS::CAsn1CoderReference pAsn1Coder;
    OSStatus nStatus = SecAsn1CoderCreate(&pAsn1Coder);
    // Check nStatus
    SecAsn1Item DerKey;
    nStatus = SecAsn1EncodeItem(pAsn1Coder, &Asn1Key, kRsaPublicKeyTemplate, &DerKey);
    // Check nStatus
    const void* pKeys[] = { kSecAttrKeyType, kSecAttrKeyClass };
    const void* pValues[] = { kSecAttrKeyTypeRSA, kSecAttrKeyClassPublic };
    MacOS::CReference<CFDictionaryRef> pParameters(CFDictionaryCreate(kCFAllocatorDefault, pKeys, pValues, 2, nullptr, nullptr));
    // Check pParameters
    MacOS::CReference<CFDataRef> pKeyData(CFDataCreate(kCFAllocatorDefault, DerKey.Data, static_cast<CFIndex>(DerKey.Length)));
    // Check pKeyData
    MacOS::CReference<CFErrorRef> pError;
    m_PublicKey = SecKeyCreateFromData(pParameters, pKeyData, &pError);
    // Check m_PublicKey - this fails with "The operation couldn’t be completed. (OSStatus error -2147415792.)"
}

I removed some check macros etc, but this should illustrate the call sequence. On 10.9 I get a null pointer from SecKeyCreateFromData with an error code -2147415792. I tried adding the kSecAttrKeySizeInBits but that did not help. Meanwhile SecKeyGeneratePair with the same pParameters works fine, so I assume the issue is with the actual data. Is the ASN.1 coding supported only from 10.10 or something like that?

UPDATE

I got messed up in my tests, this actually does not work on 10.11 as well, which seems to corelate with addition SecKeyCreateWithData.

UPDATE 2

Looking at cssmerr.h this error code seems to be CSSMERR_CSP_INVALID_KEY.

Answer 1

Since I did not find the answer anywhere I had to solve this myself so I hope this helps others.

I did not find a direct explanation for the behavior changes of SecKeyCreateFromData, but since the documentation states:

Constructs a SecKeyRef object for a symmetric key.

10.12 added a SecKeyCreateWithData which does not mention anything about symmetric keys, maybe they added some additional functionality into SecKeyCreateFromData as well and that could be why it was working for me on 10.12 and up.

Nevertheless, to obtain 10.9 compatibility I used SecItemImport with the same ASN.1 encoded sequence:

static SecKeyRef ImportPkcs1PublicKey(const unsigned char* pExponent, const std::size_t nExponentSize, const unsigned char* pModulus, const std::size_t nModulusSize)
{
    static const SecAsn1Template kRsaPublicKeyTemplate[] = {
        { SEC_ASN1_SEQUENCE, 0, NULL, sizeof(ASN1_RSA_PUBLIC_KEY) },
        { SEC_ASN1_INTEGER, offsetof(ASN1_RSA_PUBLIC_KEY, m_Modulus), 0, 0 },
        { SEC_ASN1_INTEGER, offsetof(ASN1_RSA_PUBLIC_KEY, m_Exponent), 0, 0 },
        { 0, 0, 0, 0 },
    };
    ASN1_RSA_PUBLIC_KEY Asn1Key;
    Asn1Key.m_Modulus.Data = const_cast<unsigned char*>(pModulus);
    Asn1Key.m_Modulus.Length = nModulusSize;
    Asn1Key.m_Exponent.Data = const_cast<unsigned char*>(pExponent);
    Asn1Key.m_Exponent.Length = nExponentSize;
    MacOS::CAsn1CoderReference pAsn1Coder;
    OSStatus nStatus = SecAsn1CoderCreate(&pAsn1Coder);
    // Check nStatus and pAsn1Coder
    SecAsn1Item DerKey;
    nStatus = SecAsn1EncodeItem(pAsn1Coder, &Asn1Key, kRsaPublicKeyTemplate, &DerKey);
    // Check nStatus
    MacOS::CReference<CFDataRef> pKeyData(CFDataCreate(kCFAllocatorDefault, DerKey.Data, static_cast<CFIndex>(DerKey.Length)));
    // Check pKeyData
    SecExternalFormat nFormat = kSecFormatBSAFE;
    SecExternalItemType nType = kSecItemTypePublicKey;
    MacOS::CReference<CFArrayRef> pItems;
    nStatus = SecItemImport(pKeyData, NULL, &nFormat, &nType, 0, NULL, NULL, &pItems);
    // Check nStatus
    SecKeyRef pKey = reinterpret_cast<SecKeyRef>(const_cast<void*>(CFArrayGetValueAtIndex(pItems, 0)));
    // Check pKey
    CFRetain(pKey);
    return pKey;
}

The trick here was to guess nFormat and nType, but luckily I found a table with the mapping in Apple open source sources and [CocoaCryptoMac] in Github. I already had the PKCS1 version for my key as seen in the snippet in the question the remaining job was just to set the format accordingly. This answer has a lot of valuable info on PKCS1/PKCS8. Also, initially I was not sure if pKeyData is the string or binary form of the key, since there are lots of examples that use the string form with the SecKeyCreateFromData function. I just tried out all the options until the binary version worked.