Why setrlimit(RLIMIT_NPROC) doesn't work when run as root but works fine when run as a normal user?

Question

I wrote the following C program to limit the maximum processes this program can create (on Linux). This program uses setrlimit(), and it is expected that this program can create at most 4 processes.

// nproc.c
#include <stdio.h>
#include <unistd.h>
#include <sys/resource.h>

int main(void)
{
    struct rlimit rlim;
    rlim.rlim_cur = rlim.rlim_max = 4;
    setrlimit(RLIMIT_NPROC, &rlim);
    for (int i = 0; i < 3; ++i) printf("%d\n", fork());
    sleep(1);
    return 0;
}

When I compiled and ran this program as a normal user, it gave the following output:

$ ./nproc
-1
-1
-1

-1 indicates that fork() failed and rlimit was working properly to limit maximum processes that a program can create. But when I ran this program as root, it gave the following output:

$ sudo ./nproc
25926
25927
25928
0
0
25929
0
25930
25931
0
0
0
25932
0

We can see that all the fork() succeeded and rlimit was not working properly. Where is the problem?

Answer 1

the following proposed code:

1. cleanly compiles
2. fails to perform the desired functionality (?why?)
3. incorporates all the needed header files
4. only the 'parent' tries to create child processes
5. note: the OPs and the proposed program both exit without waiting for the child processes to finish. I.E. The main program should be calling wait() or wait_pid() for each child process started.
6. Note: the call to sleep(1) keeps the output nice and organized. However, during that sleep the child complete and exits, so there is actually only 1 child process running any at any one time, so even if the call to setrlimit() had been successful, that 'fork()` loop could have run forever.

and now, the proposed code:

#include <stdio.h>
#include <stdlib.h>

#include <sys/time.h>
#include <sys/resource.h>

#include <sys/types.h>
#include <unistd.h>

int main( void )
{
    struct rlimit rlim;
    rlim.rlim_cur = rlim.rlim_max = 4;

    if( getrlimit(RLIMIT_NPROC, &rlim) == -1 )
    {
        perror( "getrlimit failed" );
        exit( EXIT_FAILURE );
    }

    if( setrlimit(RLIMIT_NPROC, &rlim) == -1 )
    {
        perror( "setrlimit failed" );
        exit( EXIT_FAILURE );
    }

    for (int i = 0; i < 4; ++i) 
    {
        pid_t pid = fork();
        switch( pid )
        {
            case -1:
            perror( "fork failed" );
            exit( EXIT_FAILURE );
            break;

            case 0:
            printf( "child pid: %d\n", getpid() );
            exit( EXIT_SUCCESS );
            break;

            default:
            printf( "parent pid: %d\n", getpid() );
            break;
        }
        sleep(1);
    }
    return 0;
}

a run of the program results in:

fork failed: Resource temporarily unavailable

which indicates a problem with the call to setrlimit()

from the MAN page:

RLIMIT_NPROC
          This is a limit on the number of extant process (or,  more  pre‐
          cisely  on  Linux,  threads) for the real user ID of the calling
          process.  So long as the current number of  processes  belonging
          to  this process's real user ID is greater than or equal to this
          limit, fork(2) fails with the error EAGAIN.

          The RLIMIT_NPROC limit is not enforced for processes  that  have
          either the CAP_SYS_ADMIN or the CAP_SYS_RESOURCE capability.

so, the call to setrlimit() is limiting the number of threads, not the number of child processes

However, if we add a couple of print statements immediately after the call to getrlimit() and again after the call to setrlimit() the result is:

    if( getrlimit(RLIMIT_NPROC, &rlim) == -1 )
    {
        perror( "getrlimit failed" );
        exit( EXIT_FAILURE );
    }

    printf( "soft limit: %d\n", (int)rlim.rlim_cur );
    printf( "hard limit: %d\n\n", (int)rlim.rlim_max );

    if( setrlimit(RLIMIT_NPROC, &rlim) == -1 )
    {
        perror( "setrlimit failed" );
        exit( EXIT_FAILURE );
    }


    if( getrlimit(RLIMIT_NPROC, &rlim) == -1 )
    {
        perror( "getrlimit failed" );
        exit( EXIT_FAILURE );
    }

    printf( "soft limit: %d\n", (int)rlim.rlim_cur );
    printf( "hard limit: %d\n\n", (int)rlim.rlim_max );

then the result is:

soft limit: 27393
hard limit: 27393

soft limit: 27393
hard limit: 27393

parent pid: 5516
child pid: 5517
parent pid: 5516
child pid: 5518
parent pid: 5516
child pid: 5519
parent pid: 5516
child pid: 5520

which indicates that call to: setrlimit() did not actually change the limits for child processes

Note: I'm running ubuntu linux 18.04