Issue with message reconstruction using 16 QAM mapping in Python code

Question

I am working on a Python code that involves message transmission and reception using 16 QAM mapping. However, I am encountering an issue where I am not getting the original message as expected.

code:

import numpy as np
import string
import random
from difflib import SequenceMatcher

def transmitter(message):
    binary_message = ''.join(format(ord(char), '08b') for char in message)
    binary_chunks = [binary_message[i:i + 4] for i in range(0, len(binary_message), 4)]

    symbol_map = {'0000': complex(1, 1),
                  '0001': complex(1, 3),
                  '0010': complex(3, 1),
                  '0011': complex(3, 3),
                  '0100': complex(-1, 1),
                  '0101': complex(-1, 3),
                  '0110': complex(-3, 1),
                  '0111': complex(-3, 3),
                  '1000': complex(1, -1),
                  '1001': complex(1, -3),
                  '1010': complex(3, -1),
                  '1011': complex(3, -3),
                  '1100': complex(-1, -1),
                  '1101': complex(-1, -3),
                  '1110': complex(-3, -1),
                  '1111': complex(-3, -3)}

    qam_signal = [symbol_map[chunk] for chunk in binary_chunks]
    signal_parts = [(sample.real, sample.imag) for sample in qam_signal]
    flat_signal = [part for sample in signal_parts for part in sample]

    return flat_signal


def channel(sent_signal, noise_factor=1):
    sent_signal = np.array(sent_signal)
    assert np.size(sent_signal) <= 400, "n must be <= 200"
    n = np.size(sent_signal) // 2
    x = sent_signal[0:2*n]
    s = np.sum(x**2) / np.size(x)
    sigma = 1
    if s > 1:
        sigma = np.sqrt(s)
    Z = np.random.normal(0, sigma*noise_factor, size=(2*n,))
    A = np.array([[11, 10], [10, 11]])
    B = np.kron(np.eye(n), A)
    Y = B.dot(x) + Z.T
    return Y


def receiver(received_signal):
    def find_closest_point(point, symbol_map):
        # Find the constellation point closest to the received point
        distances = [np.abs(point - constellation_point) for constellation_point in symbol_map.keys()]
        closest_point = min(distances)
        closest_index = distances.index(closest_point)
        closest_complex = list(symbol_map.keys())[closest_index]
        closest_binary = symbol_map[closest_complex]
        return closest_binary

    received_signal = received_signal.flatten()

    qam_signal = [complex(received_signal[i], received_signal[i + 1]) for i in range(0, len(received_signal), 2)]

    # 16-QAM demodulation
    symbol_map = {complex(1, 1): '0000',
                  complex(1, 3): '0001',
                  complex(3, 1): '0010',
                  complex(3, 3): '0011',
                  complex(-1, 1): '0100',
                  complex(-1, 3): '0101',
                  complex(-3, 1): '0110',
                  complex(-3, 3): '0111',
                  complex(1, -1): '1000',
                  complex(1, -3): '1001',
                  complex(3, -1): '1010',
                  complex(3, -3): '1011',
                  complex(-1, -1): '1100',
                  complex(-1, -3): '1101',
                  complex(-3, -1): '1110',
                  complex(-3, -3): '1111'}

    demodulated_signal = [find_closest_point(point, symbol_map) for point in qam_signal]

    binary_message = ''.join(demodulated_signal)
    text_message = bytes([int(binary_message[i:i + 8], 2) for i in range(0, len(binary_message), 8)]).decode('latin-1')

    return text_message



def generate_random_string(length):
    # All ASCII characters
    ascii_characters = string.ascii_letters + string.digits + string.punctuation
    # Generate the random string
    random_string = ''.join(random.choice(ascii_characters) for _ in range(length))
    return random_string


# Example usage:
message = generate_random_string(50)
X = transmitter(message)  # Encode our message
Y = channel(X, noise_factor=0.5)  # Simulate the treatment done by the channel
reconstructed_message = receiver(Y)  # Decode the message received by the channel


print("Original message:", message)
print("Reconstructed message:", reconstructed_message)



def check_similarity(original_message, reconstructed_message):
    # Create a SequenceMatcher object
    matcher = SequenceMatcher(None, original_message, reconstructed_message)
    # Calculate the similarity ratio
    similarity_ratio = matcher.ratio()
    return similarity_ratio

# Similarity check
similarity_ratio = check_similarity(message, reconstructed_message)
print(f"Similarity ratio: {similarity_ratio:.2f}")

output:

Original message: ]?XQ52jc?>$K{~=[kC;'QveIM^c5Yzg=u6I*0A~;Tj8IXM_m)F
Reconstructed message: ??8333óó??4K{?;ûC;73óOO?ó3?s÷?s?O33C;4ó8O8O?ÿ?O
Similarity ratio: 0.16

Description

I have implemented a code that uses 16 QAM mapping to transmit and receive messages. The code consists of the following components:

• transmitter: Converts the message to binary, pads it, and maps each symbol to a complex value using a predefined constellation.

• receiver: Demodulates the received symbols, checks if they are valid ASCII characters, and reconstructs the message.

• channel: Simulates the channel and introduces noise to the transmitted signal. generate_random_string: Generates a random message for testing.

Issue:

The problem I am facing is that when I run the code, the reconstructed message is not the same as the original message.

Expected Behavior:

I expect the reconstructed message to be identical to the original message.

Question:

• What could be causing the discrepancy between the original and reconstructed messages in my code?

• Are there any possible errors or improvements that I may have overlooked?

• How can I modify the code to ensure the accurate reconstruction of the original message? Any guidance, suggestions, or explanations would be greatly appreciated.

Answer 1

First of all, thank your for providing a fully functional code. This helps a lot.

Unneeded padding

padded_binary_message = binary_message + '0' * (4 - len(binary_message) % 4)

Did you notice that resulting array was of size 101? It's because (4 - len(binary_message) % 4) equals 4, not 0. You should do padding only if modulo is NOT equals 4. Moreover, your 8-bits representation of your string will alway be a multiple of 4. You should just remove padding.

def transmitter(message):
    binary_message = ''.join(format(ord(c), '08b') for c in message)
    symbols = [binary_message[i:i+4] for i in range(0, len(binary_message), 4)]
    constellation = {
        '0000': complex(-6, 6), '0001': complex(-6, 2), '0010': complex(-6, -6),
        '0011': complex(-6, -2), '0100': complex(-2, 6), '0101': complex(-2, 2),
        '0110': complex(-2, -6), '0111': complex(-2, -2),'1000': complex(6, 6),
        '1001': complex(6, 2), '1010': complex(6, -6), '1011': complex(6, -2),
        '1100': complex(2, 6), '1101': complex(2, 2), '1110': complex(2, -6),
        '1111': complex(2, -2)
    }

    signal = [constellation[symbol] for symbol in symbols]
    return signal

Symbol demodulation

for constellation_point in constellation.keys():
    distance = np.abs(symbol[0] + symbol[1]*1j - constellation_point)
    if distance < min_distance:
        min_distance = distance
        closest_constellation_point = constellation_point

This code block tries to compute the closest 4-bits constellation point to a 8-bits symbols. You should instead demodulate each 4-bit independently then concatenate results.

def demodulate_4bits_signal(signal):
    constellation = {
        complex(-6, 6): 0, complex(-6, 2): 1, complex(-6, -6): 2,
        complex(-6, -2): 3, complex(-2, 6): 4, complex(-2, 2): 5,
        complex(-2, -6): 6, complex(-2, -2): 7, complex(6, 6): 8,
        complex(6, 2): 9, complex(6, -6): 10, complex(6, -2): 11,
        complex(2, 6): 12, complex(2, 2): 13, complex(2, -6): 14,
        complex(2, -2): 15
    }
    min_distance = float('inf')

    for constellation_point in constellation.keys():
        distance = np.abs(signal - constellation_point)
        if distance < min_distance:
            min_distance = distance
            closest_constellation_point = constellation_point

    return constellation[closest_constellation_point]


def demodulate_symbol(symbol):
    head, tail = symbol
    return (demodulate_4bits_signal(head) << 4) + demodulate_4bits_signal(tail)

As pointed in the comments, we should first test without noise :

random_message = generate_random_string(50)
sent_signal = transmitter(random_message)
reconstructed_message = receiver(sent_signal)

print("Original message:", random_message)
print("Reconstructed message:", reconstructed_message)

Success?

Original message: UZWE}jeD-aw2<(7w"JZ:+J2_H$LQdN6JYO5PfX<aAIRdSi_T>W
Reconstructed message: UZWE}jeD-aw2<(7w"JZ:+J2_H$LQdN6JYO5PfX<aAIRdSi_T>W

Noise

As for the reconstruction of the noisy signal, I am afraid that you are generating too much noise for your input.

print(received_signal - sent_signal)
[-4.00022976e+01 -40.j -3.99901691e+01 -40.j ... ]

The culprit is A = np.array([[11, 10], [10, 11]]) this amplify your signal by a factors of at least 10.

Is this code even needed? your Z variable already contains some noise, so let's try to only use that noise:

def channel(sent_signal, noise_factor=1):
    # ...
    Z = np.random.normal(0, sigma*noise_factor, size=(2*n,))
    Y = x + Z.T
    return Y

Original message: ...
Reconstructed message: ...
Similarity ratio: 0.26

a bit bad, however lets try with a lesser noise ratio:

Y = channel(X, noise_factor=0.25)

Original message: ...
Reconstructed message: ... 
Similarity ratio: 0.82

that's better. with noise_ratio=0.15 your algorithm capable of fully reconstructing your signal:

Y = channel(X, noise_factor=0.15)

Original message: ...
Reconstructed message: ... 
Similarity ratio: 1.0

Other improvements

• Define your constellation only once and create your reverse constellation from your first definition. Avoiding discrepancies between your two entities.
• use string.encode to directly manipulate bytes instead of transforming your ascii string into a string of bits, each of theses bytes takes a whole character. this is very heavy and error prone since you want to manipulate them as bits, not char.
• Numpy allows you more algorithm linearization, your demodulation could be a lot faster using numpy.argmin
• A function should do one and only one thing. Refactoring your code to ensure that every function is correctly named should easily provides you guidance on what is wrong.
• Your variables should have better naming.

Please note that my code snippets are designed to match yours. They should not be taken as example of clean coding.