Write-read conflicts in Linux C++ IO concurrency

Question

Considering the writing of an online evaluation system, each JudgeTask has an independent input file path, output file path, expected file path, pending program path, and corresponding evaluation results. For each JudgeTask, a thread is started with std::async, and each thread completes:

1. run: Use fork to create a child process, redirect the input with a pipe in the process, and redirect the output to the specified file
2. test: Read the expected file and output the file comparison
3. The result is returned.

The problem occurs in the test phase: test is on the same thread as run, that is, it waits for run to write before it is read. test opens the file, feof, ferror are okay, but when you call fgetc to read, the corresponding file stream still has data but returns EOF: this happens completely randomly. Even if you use fsync/flock, it won't help.

I used the following code to add threads： (NOTE I've removed a lot of the error handling code, and you can assume that all the pointers point to legitimate data)

std::future<std::unique_ptr<JudgeResult>> makeJudgeTask(
    JudgeTask *task, const BasicCodeRunner *runner, const std::function<DiffResult(FILE *, FILE *)> &streamComparer) {
    return std::async(std::launch::async, [task, runner, streamComparer]() {
        runJudgeTask(task, runner);
        testJudgeTask(task, streamComparer);
        return std::move(task->_result);
    });
}

And the following code runs the program and completes the redirecting input and output:

void runJudgeTask(JudgeTask *task, const BasicCodeRunner *runner) {
    if (task->executed()) {
        return;
    }
    auto runResult 
        = runner->runOnFile(task->_testProgramName, task->_inputFilePath, task->_outputFilePath);
    task->_hasExecuted = true;
    if (runResult.exitCode != 0) {
        task->_result = /* make error result */
        return;
    }
    task->_result = std::make_unique<JudgeResult>(
        task->_inputFilePath.stem(), ProgramExecutionStatus::RUNNING, runResult.resourceUsage);
}

RunCodeCaseResult LinuxCodeRunner::runOnFile(const fs::path &programPath,
                                             const fs::path &inputPath,
                                             const fs::path &outputPath) const {
    int inputFd = open(inputPath.c_str(), O_RDONLY);
    int outputFd = open(outputPath.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0644);
    if (inputFd == -1 || outputFd == -1) {
         // error
    }
    auto [status, runtime, memory] = _runFork(inputFd, outputFd, programPath.c_str());
    close(inputFd);
    fsync(outputFd);
    close(outputFd);
    if (status == -1) {
        std::exit(errno);
    } else if (status != 0) {
        MyLogger.debug("program {} exited with status {}", programPath.u8string(), status);
    }
    return {inputPath.stem().u8string(), outputPath, status, runtime, memory};
}

std::tuple<int, long, long> 
LinuxCodeRunner::_runFork(int inputFd, int outputFd, const char *programPath) noexcept {
    using namespace std::literals;
    int pid = fork();
    if (pid == -1) {
        return std::make_tuple(-1, 0, 0);
    } else if (pid == 0) {
        dup2(inputFd, STDIN_FILENO);
        dup2(outputFd, STDOUT_FILENO);
        dup2(outputFd, STDERR_FILENO);
        execl(programPath, programPath, nullptr);
        exit(0);
    } else {
        int status;
        rusage usage {};
        wait3(&status, 0, &usage);
        // return info
    }
}

test and run are under the same thread. And test is simply waiting for the run to finish and then using fgetc to compare the data character by character.

void testJudgeTask(JudgeTask *task, const std::function<DiffResult(FILE *, FILE *)> &streamComparer) {
    while (!task->_hasExecuted) std::this_thread::yield();
    if (task->_result->status != ProgramExecutionStatus::RUNNING) {
        return;
    }
    FILE *expectedFileStream = fopen(task->_expectedFilePath.c_str(), "r");
    FILE *outputFileStream = fopen(task->_outputFilePath.c_str(), "r+");
    // error handled.
    auto [expected, output, tokenCount] = streamComparer(outputFileStream, expectedFileStream);
    if (expected.empty() && output.empty()) {
        task->_result->status = ProgramExecutionStatus::ACCEPTED;
    } else {
        task->_result->status = ProgramExecutionStatus::WRONG_ANSWER;
        task->_result->detail = DiffResult {expected, output, tokenCount};

    }
}

streamComparer is just a simple loop, checked character by character via fgetc, all the way to EOF.

Even though there is data in the file stream, fgetc returns an EOF as soon as it is called, why does this happen?