Standard I/O stream -- fgets() buffering type

Question

The book "advanced programming in unix environment" discussed pipe in chapter 15, it shows that we should pay attention to the buffering type when deal with standard I/O functions.

The buffering types for different opened standard I/O streams are (discussed in chapter 5 of the book):

• standard error is unbuffered
• streams connected to terminal devices is line-buffered
• all other streams are fully-buffered

When parent/child connect to a pipe, the end (which should be a FILE * type object, according to the interface) that they used to communicate should be fully-buffered according to the rule list above (since it's a stream connected to pipe). But the behavior of sample code from that chapter seems to be something NOT fully-buffered.

Here is the sample code:

myuclc.c:

1   #include "apue.h"
2   #include <ctype.h>

3   int
4   main(void)
5   {
6       int     c;

7       while ((c = getchar()) != EOF) {
8           if (isupper(c))
9               c = tolower(c);
10          if (putchar(c) == EOF)
11              err_sys("output error");
12          if (c == '\n')
13              fflush(stdout);
14      }
15      exit(0);
16  }

popen1.c:

1   #include "apue.h"
2   #include <sys/wait.h>

3   int
4   main(void)
5   {
6       char    line[MAXLINE];
7       FILE    *fpin;
8
9       if ((fpin = popen("myuclc", "r")) == NULL)  // "myuclc" is executable file compile-link by "myuclc.c"
10          err_sys("popen error");
11      for ( ; ; ) {
12          fputs("prompt> ", stdout);
13          fflush(stdout);
14
15          if (fgets(line, MAXLINE, fpin) == NULL) /* read from pipe */
16              break;
17          if (fputs(line, stdout) == EOF)
18              err_sys("fputs error to pipe");
19      }
20      if (pclose(fpin) == -1)
21          err_sys("pclose error");
22      putchar('\n');
23      exit(0);
24  }

So my question is : fgets() in line 15 of popen1.c should be fully-buffered according to the buffering rules, why does it act like line-buffered or unbuffered:

Furthermore, I also tried to setvbuf() before fgets() to specifically set buffering type to _IOFBF (fully-buffered) of fpin, still not work.

prompt> abc
abc
prompt> ABC
abc
prompt> efg
efg
prompt> EFG
efg
prompt>

Answer 1

In myuclc.c you perform an explicit flush on every newline:

12          if (c == '\n')
13              fflush(stdout);

This causes the stream to be manually line-buffered. Whenever you flush the pipe, the process on the other end will be unblocked and will read whatever was in the buffer at the time.

The "buffering rules" talk about when this flushing happens automatically. Unbuffered streams are automatically flushed after every write command (fprintf, fputc, etc.). Line-buffered streams are automatically flushed whenever a newline character is written to the stream.

And all streams are flushed when the buffer fills up, when the stream is closed, or when the writer performs an explicit flush

Answer 2

Your code doesn't match your description. You're talking about the pipe system call, but the code uses popen. popen is a function not in ISO C but in POSIX, and is subject to its own set of requirements in POSIX. POSIX doesn't say what is the buffering mode of popen-ed streams, unfortunately. It has this curious wording, though: "Buffered reading before opening an input filter may leave the standard input of that filter mispositioned. Similar problems with an output filter may be prevented by careful buffer flushing; for example, with fflush." I cannot make sense of the first sentence: how can reading take place before opening? The second sentence seems to imply that popen streams may be fully buffered, and so explicit fflush may be necessary to be sure data is passed on to an output pipe. Of course, if that process itself is reading input with full buffering, it might not help!

If you create a pipe with the pipe system call, obtaining a pair of file descriptors, you can then create FILE * streams over these descriptors with fdopen. That, again, is not an ISO C function. Therefore it is not bound by the requirement which ISO C gives for fopen, namely: "When opened, a stream is fully buffered if and only if it can be determined not to refer to an interactive device. The error and end-of-file indicators for the stream are cleared." To see whether that is true of fdopen, we have to look into POSIX. Unfortunately, POSIX is silent about this; it doesn't say anything about buffering. It also doesn't say that fdopen inherits any special requirements from fopen. It does say that the meaning of the mode flags is "exactly as specified in fopen(), except that modes beginning with w shall not cause truncation of the file."

POSIX has a description of fopen, and that description mirrors the above quoted ISO C text about buffering, verbatim. Since POSIX's description of fdopen doesn't have any such text and doesn't have any requirement that fdopen must follow requirements from fopen (other than with regard to the meaning of the mode flags), the buffering set up by fdopen is up in the air. A conforming fdopen could set up full buffering even if the file descriptor is a TTY.

Thus, if you're using fdopen or popen, and the choice of buffering matters in your situation, you should arrange it yourself with setvbuf.

Regarding:

I also tried to setvbuf() before fgets() ...

Buffering affects output. The stdio input functions do not delay the delivery of buffered input data to the application. The fact that you're able to read individual lines from the process connected to the pipe means that that process is flushing its own output buffer for each line. That line is then transmitted through the pipe and available to your own process. The stdio library is not going to delay your fgets operation until more lines accumulate, even under full buffering. That's not how it works; full buffering means output is accumulated until a buffer fills up or fflush is called.