How to get the type of a serialized ciphertext message?

Question

I'm using libsignal-protocol-java according to Signal's Documentation to implement end-to-end encryption in a messaging app.

Let's say Alice sends Bob a (pairwise) serialized ciphertext message. How does Bob know how to deserialize it? Doesn't he first need to know the type of the ciphertext message? My understanding is that a pairwise ciphertext message could either be a signal message (WHISPER_TYPE) or a pre key signal message (PREKEY_TYPE). So how does Bob know which type it is?

Is Alice supposed to also send Bob the type (as plaintext) of the ciphertext message?

Or is there another way that Bob could detect the type? For example, is Bob supposed to try to deserialize it as if it's a signal message, and if that fails, then try to deserialize it as if it's a pre key signal message?

Answer 1

After reviewing the source code of Signal Android, I think Alice is supposed to also send Bob the type (as plaintext) of the ciphertext message.

In the class SignalServiceCipher:

• The instance method encrypt(SignalProtocolAddress, Optional<UnidentifiedAccess>, byte[]) returns an instance of OutgoingPushMessage with its type property set to that of the ciphertext message and its body property set to the Base64 encoded serialized ciphertext message.

• The instance method decrypt(SignalServiceEnvelope, byte[]) gets the type of the ciphertext message from its first argument, which is an instance of SignalServiceEnvelope.

Answer 2

1 - Alice generate identityKeyPair(Long Term), signedPreKey(Medium Term) & Ephemeral PreKeys & save those keys into storage in base64. e.g

public static String generateIdentityKeyPair() {
    IdentityKeyPair identityKeyPair = KeyHelper.generateIdentityKeyPair();
    return encodeToBase64(identityKeyPair.serialize());
}

2 - Send

• List of PreKey Id & Public key to server in Serialized format
• Signed PreKey Id, signedPreKeyPublicKey, signedPreKeyRecordSignature
• Public Key of IdentityKeyPair
• Registration Id

For Encryption and Decryption you first have to make to encrypted session

private void initSessionFromPreKey() throws UntrustedIdentityException, InvalidKeyException {
        InMemorySignalProtocolStore protocolStore = new InMemorySignalProtocolStore(localUser.getIdentityKeyPair(), localUser.getRegistrationId());
        protocolStore.storePreKey(localUser.getPreKeys().get(0).getId(), localUser.getPreKeys().get(0));
        protocolStore.storeSignedPreKey(localUser.getSignedPreKey().getId(), localUser.getSignedPreKey());
        this.protocolStore = protocolStore;


        //Session
        SessionBuilder sessionBuilder = new SessionBuilder(protocolStore, remoteUser.getSignalProtocolAddress());
        PreKeyBundle preKeyBundle = new PreKeyBundle(
                remoteUser.getRegistrationId(),
                remoteUser.getSignalProtocolAddress().getDeviceId(),
                remoteUser.getPreKeyId(),
                remoteUser.getPreKeyPublicKey(),
                remoteUser.getSignedPreKeyId(),
                remoteUser.getSignedPreKeyPublicKey(),
                remoteUser.getSignedPreKeySignature(),
                remoteUser.getIdentityKeyPairPublicKey()
        );

        sessionBuilder.process(preKeyBundle);
        mSessionCipher = new SessionCipher(protocolStore, protocolAddress);
    }

Encryption & Decryption

public String encrypt(String message) throws InvalidVersionException, InvalidMessageException, UntrustedIdentityException, InvalidKeyException {
        createSession(Operation.ENCRYPT);
        CiphertextMessage ciphertextMessage = mSessionCipher.encrypt(message.getBytes());
        PreKeySignalMessage preKeySignalMessage = new PreKeySignalMessage(ciphertextMessage.serialize());
        return KeyUtils.encodeToBase64(preKeySignalMessage.serialize());
    }

    public String decrypt(String message) throws InvalidVersionException, InvalidMessageException, InvalidKeyException, DuplicateMessageException, InvalidKeyIdException, UntrustedIdentityException, LegacyMessageException {
        createSession(Operation.DECRYPT);
        byte[] bytes = KeyUtils.decodeToByteArray(message);
        byte[] decryptedMessage = mSessionCipher.decrypt(new PreKeySignalMessage(bytes));
        return new String(decryptedMessage, StandardCharsets.UTF_8);
    }

You can also look other source code which are available on github https://github.com/lvijay/DemoSignal

https://github.com/signalapp/libsignal-protocol-java/pull/21/commits/3496ed996359f6d3d8ee52dcecb8f8b0d45b3cbc (The Library author is using is wrapper of signal protocol you can change wrapper library to signal protocol)