OverflowError: Python int too large to convert to C int when using confluent_kafka Avro deserializer

47 Views Asked by At

I'm encountering an OverflowError when attempting to deserialize messages using the **confluent_kafka **Avro deserializer in Python. Here's a simplified version of my code:

class ConsumerKafka:
    def __init__(self, deserializer):
        self.deserializer = deserializer
        self.consumer = Consumer()

    def decode(self, msg_value):      
            deserialized_data = self.deserializer(msg_value, None)

I want to underline that sel. serilizer is AvroDeserializer object from onfluent_kafka.schema_registry.avro

when i call the self.deserializer(msg_value, None)


Traceback (most recent call last):
  File "c:\Users\048115571\Documents\python\TTA\read_from_topic\modules\consumer.py", line 30, in decode
    deserialized_data = self.deserializer(msg_value, None)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\Users\048115571\AppData\Local\Programs\Python\Python311\Lib\site-packages\confluent_kafka\schema_registry\avro.py", line 429, in __call__
    obj_dict = schemaless_reader(payload,
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "fastavro\\_read.pyx", line 1142, in fastavro._read.schemaless_reader
  File "fastavro\\_read.pyx", line 1169, in fastavro._read.schemaless_reader
  File "fastavro\\_read.pyx", line 748, in fastavro._read._read_data
  File "fastavro\\_read.pyx", line 621, in fastavro._read.read_record
  File "fastavro\\_read.pyx", line 740, in fastavro._read._read_data
  File "fastavro\\_read.pyx", line 558, in fastavro._read.read_union
  File "fastavro\\_read.pyx", line 724, in fastavro._read._read_data
  File "fastavro\\_read.pyx", line 393, in fastavro._read.read_array
  File "fastavro\\_read.pyx", line 748, in fastavro._read._read_data
  File "fastavro\\_read.pyx", line 621, in fastavro._read.read_record
  File "fastavro\\_read.pyx", line 740, in fastavro._read._read_data
  File "fastavro\\_read.pyx", line 558, in fastavro._read.read_union
  File "fastavro\\_read.pyx", line 770, in fastavro._read._read_data
  File "fastavro\\_logical_readers.pyx", line 22, in fastavro._logical_readers.read_timestamp_millis
  File "fastavro\\_logical_readers.pyx", line 24, in fastavro._logical_readers.read_timestamp_millis
OverflowError: Python int too large to convert to C int

I try to update all libraries but I can't sorting out. Could someone please provide insights into what might be causing this error and how to address it? Thank you.

0

There are 0 best solutions below