Why Git gives the error "Resource temporarily unavailable (code: 11)" when simultaneously executing multiple instances?

Question

I have an automated test that tries to clone a bunch of specific revisions from the local Git repository by using the following asynchronous Nim procedure.

proc cloneSpecificRevision(downloadMethod: DownloadMethod,
                           url, downloadDir: string,
                           vcsRevision: Sha1Hash) {.async.} =
  assert vcsRevision != notSetSha1Hash
  display("Cloning", "revision: " & $vcsRevision, priority = MediumPriority)
  case downloadMethod
  of DownloadMethod.git:
    let downloadDir = downloadDir.quoteShell
    createDir(downloadDir)
    discard await tryDoCmdExAsync("git", @["-C", downloadDir, "init"])
    discard await tryDoCmdExAsync("git",
      @["-C", downloadDir, "remote", "add", "origin", url])
    discard await tryDoCmdExAsync("git",
      @["-C", downloadDir, "fetch", "--depth", "1", "origin", $vcsRevision])
    discard await tryDoCmdExAsync("git",
      @["-C", downloadDir, "reset", "--hard", "FETCH_HEAD"])
  of DownloadMethod.hg:
    discard await tryDoCmdExAsync("hg", @["clone", url, "-r", $vcsRevision])

The result is

Downloading /tmp/tlockfile/origins/dep1 using git
    Cloning revision: 8b6ce61df05f4e21ad954f9ddb487eda8fb64f41
  Executing git -C /tmp/nimble_94295/_tmptlockfileoriginsdep1_0.1.0_8b6ce61df05f4e21ad954f9ddb487eda8fb64f41 init
Downloading /tmp/tlockfile/origins/dep2 using git
    Cloning revision: d1ba2f37a1647ca1ed804dfa0c8c73a7de98201e
  Executing git -C /tmp/nimble_94295/_tmptlockfileoriginsdep2_0.1.0_d1ba2f37a1647ca1ed804dfa0c8c73a7de98201e init
    Error:  Resource temporarily unavailable (code: 11)

The used Git version is:

> git --version
git version 2.31.1

The problem happens only on Linux and my tests are passing normally on Windows and macOS systems. What can be the reason for this problem?

Answer 1

Error 11 on Linux is EAGAIN, Resource temporarily unavailable. The Linux fork system call is probably producing this error because, well, there are not enough resources at the moment.

What's a resource? There are a lot of them, but one of them is memory and another is processes. Both can be "temporarily" used up since programs (processes) run for a while, then exit; when they exit, they release the resources they were using. See also Fork fails with "resource temporarily unavailable". Which resource?, OSError: [Errno 11] Resource temporarily unavailable. What causes this?, and fork() failing with Out of memory error (note that the last one shows errno 12, ENOMEM, but you can get EAGAIN on other Unix-like systems here so I've included it even though you mentioned Linux specifically).

Note that this has nothing to do with Git itself: if you've hit a process limit with fork, it won't matter what you intend to run after the fork returns in the child process.