Python subprocess.Popen not working

Question

I've been reading up on a lot of documentations but am still not sure what I'm doing wrong.

So I have a separate shell script that fires up a separate server then the one I'm working on. Once the server is connected, I want to run ls and that's it. However, for some reason stdin=subprocess.PIPE is preventing the Popen command from terminating so that the next line could execute. For example because the code is stuck I'll Ctrl+C but I'll get an error saying that wait() got a keyboard interrupt. Here's an example code:

import subprocess
from time import sleep

p1 = subprocess.Popen("run_server", 
                      stdout = subprocess.PIPE, 
                      stdin = subprocess.PIPE)
#sleep(1)
p1.wait()
p1.communicate(input = "ls")[0]"

If I replace p1.wait() with sleep(1), the communicate command does run and displays ls, but the script that runs the server detects eof on tty and terminates it self. I must have some kind of wait between Popen and communicate because the server script will terminate for the same reason.

Answer 1

p.wait() does not return until the child process is dead. While the parent script is stuck on p.wait() call; your child process expects input at the same time -- deadlock. Then you press Ctrl+C in the shell; it sends SIGINT signal to all processes in the foreground process group that kills both your parent Python script and run_server subprocess.

You should drop the .wait() call:

#!/usr/bin/env python
from subprocess import Popen, PIPE

p = Popen(["run_server"], stdout=PIPE, stdin=PIPE)
output = p.communicate(b"ls")[0]

Or in Python 3.4+:

#!/usr/bin/env python3
from subprocess import check_output

output = check_output(["run_server"], input=b"ls")

If you want to run several commands then pass them all at once:

input = "\n".join(["ls", "cmd2", "etc"]) # with universal_newlines=True

As you know from reading the subprocess docs, p.communicate() waits for the child process to exit and therefore it should be called at most once. As well as with .wait(), the child process is dead after .communicate() has returned.

Answer 2

The fact that when you Ctrl+C and your traceback says you were stuck in wait() means the next line is executing, the next line is wait(). wait() won't return until your p1 process returns. However, it seems your p1 process won't return until you send it a command, 'ls' in your case. Try sending the command then calling wait().:

import subprocess
from time import sleep

p1 = subprocess.Popen("run_server", 
                      stdout = subprocess.PIPE, 
                      stdin = subprocess.PIPE)
#sleep(1)
p1.communicate(input = "ls")[0]"
p1.wait()

Otherwise, make sure your "run_server" script terminates so your script can advance past p1.wait()