How to use Popen to capture output from a console window (Windows OS)?

19 Views Asked by At

I am at wits end trying to find a solution to this, I've spent several hours trying to solve this and come up with nothing. The problem is fairly simple. I want to monitor stdout in real time from a console window (so subprocess.run and subprocess.communicate(), or anything that blocks until the procedure is finished) are not options. I am using Windows systems so anything relying on Linux can't be used. Currently using Python 3.10

First Attempt:

    proc = subprocess.Popen(
        ['launch', 'some', 'exe', 'file'],
        cwd="some/directory",
        bufsize=0,
        universal_newlines=True,
        stdout=subprocess.PIPE,
        text=True,
        stdin=subprocess.PIPE,
        creationflags=CREATE_NEW_CONSOLE
    )

    while True:
        output = proc.stdout.readline()

        # Check if the output contains information about hitting a breakpoint
        if 'Some Text' in output:
            foo(var)
            break
        else:
            print(output)

Now, this works in the sense that the command runs, and I read the output in real time, and once 'Some Text' is found, foo runs. The problem is, the new console is blank, and the output is printed to the Python window, not the new console. This is because stdout = subprocess.PIPE 'steals' the output from the console Window. This does not work for my use case because the exe file is an interactive terminal that I need the user to interact with. I do not care about having them interact with the subprocess through Python, I need to capture the output so I know when to run foo.

Other Attempts:
Some of these will be more in depth than others. I will describe what I tried and why it hasn't worked.

Echo Terminal

What I tried: Launching a thread that runs proc and launching proc2 on another thread. The idea was to echo proc.stdout (proc2.stdin = proc.stdout), and then accept user input and write it to proc.stdin.

Why it doesn't work: Writing to stdin blocks and I was unable to even get a simple echo statement to run. I didn't pursue this super in depth because my research indicated blocking would be a severe problem I'm fighting against the whole time.

Writng to sty.stdout

proc = subprocess.Popen(
        ['launch', 'some', 'exe', 'file'],
        cwd="some/directory",
        bufsize=0,
        universal_newlines=True,
        stdout=sys.stdout,
        text=True,
        creationflags=CREATE_NEW_CONSOLE
    )

What I tried: When setting stdout to sys.stdout, the process behaves as expected (outputs to the right window). My idea here was to replace sys.stdout with a "listener" that has a write() method that writes to both the original sys.stdout and a new file stream.

Why it doesn't work: When Popen writes to sys.stdout, it doesn't call sys.stdout.write. Instead, it dumps it directly to the python.stdout file stream

Pipes

What I tried: I tried a lot here. I tried directly accessing python sys.stdout and adding a pipe to the end of it. I've tried adding a Pipe to the start of sys.stdout and then passing it along. I've tried piping subprocess.stdout back in to the subprocess.stdin

Why it doesn't work: Honestly I've tried too much here to really go into detail, but I never even got close with any of them

Microsoft Python Libraries What I tried: Using Microsofts Python libraries to get the pid of the launched console and then directly write to it.

Why it doesn't work: To be honest I'm not sure, but I never made any type of progress with this method

GUI

What I tried: This is probably the closest I've gotten to what I'm looking for. I created a super simple GUI that is supposed to mock a terminal.

Why it doesn't work: It's not that this doesn't work, per say, but that it's a lot of effort. For example, when the user presses control + c (when the exe is not launched in Python), the program pauses (not terminate). Propagating ctr+c from the GUI to the subprocess is proving to be a problem. Further, this is just one functionality, and continuing down this path seems like a waste of effort for what (seems like) should be a fairly simple task.

    proc = subprocess.Popen(
        ['launch', 'some', 'exe', 'file'],
        cwd="some/directory",
        bufsize=0,
        universal_newlines=True,
        stdout=subprocess.PIPE,
        stdin=subprocess.PIPE,
        text=True,
    )

    while True:
        output = proc.stdout.readline()

        write_to_gui(output)

Final Thoughts I'm not sure if missing something, or if what I'm trying to do is impossible, but I feel like I should be able to do it. I'm open to any other suggestions or implementations for how to do this, but I really am at the end of my rope and haven't been able to find any resources to help with this. I'm including a rough diagram of the situation in case I wasn't clear somewhere. Thanks! https://i.stack.imgur.com/D3ClE.jpg

0

There are 0 best solutions below