python open function trunctating output

Question

I am very new to python and coding in general, so any help or point in the right direction is greatly appreciated.

I am trying to print an output to a file using python's open function. when I print the 'jstat_output' variable to the terminal it appears fine, but when I try to read the textfile it has been truncated.

the code:

with open(path_to_file, 'w', newline='') as o:
for item in jstat_output:
    o.write(f'{item}\n')

terminal output:

[Container(PayloadLength=81, Payload=u'  S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT     GCT   '), Container(PayloadLength=81, Payload=u'  S0     S1     E      O      M     CCS    YGC     YGCT    FGC    FGCT     GCT   ')]

notepad viewer:

Container: PayloadLength = 81 Payload = u' S0 S1 E O M '... (truncated, total 81) Container: PayloadLength = 81 Payload = u' S0 S1 E O M '... (truncated, total 81)

if anyone knows why this happens, or perhaps a different method to write to a file it would be a massive help; thanks.

    for pid in extracted_numbers:
        exec_resp = ecs.execute_command(
            cluster=f'{clusterArn[0]}',
            container=f'{containerNames[ttype]}',
            task=f'{tArns[tarn]}',
            command=f'jstat -gcutil {pid} 250',
            interactive=True
    )
    session = exec_resp['session']
    connection = websocket.create_connection(session['streamUrl'])
    try:
        init_payload = {
            "MessageSchemaVersion": "1.0",
            "RequestId": str(uuid.uuid4()),
            "TokenValue": session['tokenValue']
        }
        connection.send(json.dumps(init_payload))

        AgentMessageHeader = c.Struct(
            'HeaderLength' / c.Int32ub,
            'MessageType' / c.PaddedString(32, 'ascii'),
        )

        AgentMessagePayload = c.Struct(
            'PayloadLength' / c.Int32ub,
            'Payload' / c.PaddedString(c.this.PayloadLength, 'ascii')
        )

        while True:
            response = connection.recv()

            message = AgentMessageHeader.parse(response)

            if 'channel_closed' in message.MessageType:
                raise Exception('Channel closed before command output was received')

            if 'output_stream_data' in message.MessageType:
                break
    finally:
        connection.close()

    payload_message = AgentMessagePayload.parse(response[message.HeaderLength:])
    jstat_output.append(payload_message)

print(jstat_output)
with open(path_to_file, openvar, newline='') as o:
    print(f'{jstat_output}', file=o)

this is the portion of the code that is related to the problem.

Answer 1

The reason that your print() prints the non-truncated version is that it's printing the repr() of each item instead of the str() of it.

I believe this will do what you want:

with open(path_to_file, 'w') as o:
    for item in jstat_output:
        o.write(repr(item) + "\n")

The clue is that jstat_output is a list and when you printed it with:

print(jstat_output)

you got the desired output.

When a list is rendered as a string, it renders the repr() of each of its individual items. If the items are formatted individually (e.g. by doing f"{item}" or print(item)), you're instead getting their str() representation. In this case, the str() representation is truncated when the item is large, whereas repr() includes the whole thing. Explicitly calling repr(item) instead of using an f-string should therefore get you the output you're looking for. You could also still use an f-string as long as you put the repr() inside the {}, i.e. f"{repr(item)}\n".

A better solution might be to write your own function to output a Container in the desired format rather than relying on its __repr__ implementation, but that would require being able to see the definition of the Container class.