How can I read stdout/stderr contents from a Process which is still running?

Question

I have some problems using the Process and ProcessBuilder classes in Java. Here's my existing code, which works (this is code being executed in a JUnit test btw):

ProcessBuilder processBuilder = new ProcessBuilder()
        .command( commandLine )
        .directory( new File( directory ) )
        .redirectErrorStream( true );

Process process = processBuilder.start();

assertThat( process.exitValue() )
        .as( "Process exited with non-zero exit code. Stdout and stderr content:\n\n%s", readAllAsString( process.getInputStream() ) )
        .isZero();

The readAllAsString() method looks like this:

private String readAllAsString( InputStream inputStream ) throws IOException {
    return CharStreams.toString( new InputStreamReader(
            inputStream, StandardCharsets.UTF_8
    ) );
}

This works fine, but it doesn't handle all scenarios. What if the process doesn't exit normally, but sits and wait for input on its standard input? So, I'm trying to add something like this:

if ( !process.waitFor( 10, TimeUnit.SECONDS ) ) {
    assertThat( false )
            .as( "Timed out waiting for process, aborting. Stdout and stderr content:\n\n%s", readAllAsString( process.getInputStream() ) )
            .isTrue();
}

...but this is where the problem starts. The test sits and waits forever; it never reaches the end of the InputStream. The readAllAsString() method probably reaches the end of the stream and then blocks, waiting on the process to write any more data to it. (Note: this is just my assumption, I haven't verified this in the debugger)

If I try to workaround this by adding a destroy() call like this:

if ( !process.waitFor( 10, TimeUnit.SECONDS ) ) {
    process.destroy();

    assertThat( false )
            .as( "Timed out waiting for process, aborting. Stdout and stderr content:\n\n%s", readAllAsString( process.getInputStream() ) )
            .isTrue();
}

...I get an exception when trying to read the data from the stream:

java.io.IOException: Stream closed
    at java.base/java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:168)
    at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:334)
    at java.base/sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:270)
    at java.base/sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:313)
    at java.base/sun.nio.cs.StreamDecoder.read(StreamDecoder.java:188)
    at java.base/java.io.InputStreamReader.read(InputStreamReader.java:177)
    at java.base/java.io.Reader.read(Reader.java:250)
    at com.google.common.io.CharStreams.copyReaderToBuilder(CharStreams.java:121)
    at com.google.common.io.CharStreams.toStringBuilder(CharStreams.java:179)
    at com.google.common.io.CharStreams.toString(CharStreams.java:165)
    at fi.hibox.test.scripts.CentreConsoleTest.readAllAsString(CentreConsoleTest.java:80)
    at fi.hibox.test.scripts.CentreConsoleTest.centre_console_can_call_getVersion_successfully(CentreConsoleTest.java:70)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

How can I workaround this? What's the proper way to read all the data from this InputStream, without blocking at the end of the available data? Or are there any other, better ways to fix this?

Answer 1

The problem with using CharStreams.toString() for this is that it expects the stream to have reached EOF. In cases where the process is still running, this is not the case. If you rewrite the readAllAsString() method like this it works, with a few caveats:

• All input must be currently available. This method is non-blocking; it just reads the data from the InputStream which is currently available. Any output written by the process after the readAllAsString() method is called will not be included.
• This presumes that the buffer for the pipe doesn't get full. On my current Linux system the limit seems to be 1 MiB (cat /proc/sys/fs/pipe-max-size).

Java 7+ implementation

private String readAllAsString( InputStream inputStream ) throws IOException {
    if ( inputStream.available() == 0 ) {
        return "";
    }

    // Note: cannot use Guava's CharStreams.toString() in this case, since the stream
    // has not necessarily reached EOF yet (if we are called when the process is
    // still running)
    byte[] inputData = new byte[inputStream.available()];
    int bytesRead = inputStream.read( inputData, 0, inputStream.available() );
    ByteBuffer byteBuffer = ByteBuffer.wrap( inputData, 0, bytesRead );

    return StandardCharsets.UTF_8.decode( byteBuffer ).toString();
}