preempt RT conflict with docker in yocto linux

Question

I'm having a strange problem with my project, I'll describe the problem: I patched Preempt RT on yocto linux and it proved in real time that it can run real time scheduling tasks like FIFO and RR, then I configured docker on top of that, strange thing happened, when I started docker, my real time scheduling tasks like FIFO and RR can't run. Does anyone have any thoughts or can you point out where the error might be occurring? (the FIFO and RR testing is in the host, not in any container)

before install docker

root@intel-corei7-64:~# uname -ar
Linux intel-corei7-64 5.15.137-rt71-intel-pk-preempt-rt #1 SMP PREEMPT_RT Fri Feb 2 16:01:33 UTC 2024 x86_64 GNU/Linux
root@intel-corei7-64:~# sudo /userdata/test_thread_rr_fifo fifo
My thread FIFO is running 1 seconds...
My thread FIFO is running 2 seconds...
My thread FIFO is running 3 seconds...
My thread FIFO is running 4 seconds...

after install docker

root@intel-corei7-64:~# tar -xvf docker-24.0.7.tgz
root@intel-corei7-64:~# cp docker/* /usr/bin/
root@intel-corei7-64:~# systemctl enable docker
create symlink /etc/systemd/multi-user.target.wants/docker.service -> /etc/systemd/system/docker.service.
root@intel-corei7-64:~# systemctl start docker
root@intel-corei7-64:~# sudo /userdata/test_thread_rr_fifo fifo
Please use sudo to test. Or pthread_created Failed: This kernel not support FIFO
root@intel-corei7-64:~# docker ps
CONTAINER ID    IMGAE    COMMAND    CREATED    STATUS    PORTS    NAMES
root@intel-corei7-64:~# docker version
Client:
  Version:       24.0.7
  API version:   1.43
  Go version:    go1.20.10
-----------------------------------------------------------

root@intel-corei7-64:~# systemctl disable docker
root@intel-corei7-64:~# systemctl stop docker
root@intel-corei7-64:~# sudo /userdata/test_thread_rr_fifo fifo
My thread FIFO is running 1 seconds...
My thread FIFO is running 2 seconds...
My thread FIFO is running 3 seconds...
My thread FIFO is running 4 seconds...

the FIFO and RR test code

#include <pthread.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <errno.h>
#include <iostream>

void* my_thread_body(void* arg)
{
    long cnt = 0;
    while (1) {
        std::cout << "My thread FIFO is running " << ++cnt << " seconds..." << std::endl;
        sleep(1);
    }
}

void* my_thread_body2(void* arg)
{
    long cnt = 0;
    while (1) {
        std::cout << "My second thread RR is running " << ++cnt << " seconds..." << std::endl;
        sleep(1);
    }
}

int main(int argc, char* argv[])
{
    pthread_t my_thread, my_thread2;
    struct sched_param param;
    param.sched_priority = 50;
    pthread_attr_t attr;
    int ret = 0;

    if (argc < 2) {
        std::cout << "Usage: sudo " << argv[0] << " [scheduling policy]\n\nUsable policy: fifo, rr \n\neg.  sudo " <<argv[0]<< " fifo\n" << std::endl;
        return -1;
    }

    if ((strcmp(argv[1], "fifo") == 0) || (strcmp(argv[1], "rr") == 0)) {
        if ((ret = pthread_attr_init(&attr)) != 0) {
            perror("pthread_attr_init()");
            std::cout << "ret code: " << ret << std::endl;
            return -1;
        }
        if (strcmp(argv[1], "rr") == 0) {
            if ((ret = pthread_attr_setschedpolicy(&attr, SCHED_RR)) != 0) {
                std::cout << "pthread_attr_setschedpolicy() Failed" << std::endl;
                return -1;
            }
        }
        if (strcmp(argv[1], "fifo") == 0) {
            if ((ret = pthread_attr_setschedpolicy(&attr, SCHED_FIFO)) != 0) {
                std::cout << "pthread_attr_setschedpolicy() Failed" << std::endl;
                return -1;
            }
        }

        if ((ret = pthread_attr_setschedparam(&attr, &param)) != 0) {
            std::cout << "pthread_attr_setschedparam Failed" << std::endl;
            return -1;
        }
        if ((ret = pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED)) != 0) {
            std::cout << "pthread_attr_setinheritsched() Failed" << ret << std::endl;
            return -1;
        }

        if (strcmp(argv[1], "fifo") == 0) {
            if ((ret = pthread_create(&my_thread, &attr, my_thread_body, NULL)) != 0) {
                std::cout << "Please use sudo to test. Or pthread_create Failed: This kernel not support FIFO" << std::endl;
                return -1;
            }
            pthread_join(my_thread, NULL);
        }
        else if (strcmp(argv[1], "rr") == 0) {
            if ((ret = pthread_create(&my_thread2, &attr, my_thread_body2, NULL)) != 0) {
                std::cout << "Please use sudo to test. Or pthread_create Failed: " << strerror(ret)  << std::endl;
                return -1;
            }
            pthread_join(my_thread2, NULL);
        }
    }
    else {
        std::cout << "error param" << std::endl;
    }
    std::cout << "Bye!" << std::endl;
    return 0;
}

tried:

1. I changed the docker version from 1.18 to 2.24, the issues still happen
2. I check the docker config from kernel using the offical Check-config.sh, all things is good
3. the CONFIG_RT_GOURP_SCHED is enbaled in the kernel
4. I compared the kernel setting this x86 yocto linux with arm64 yocto linux (this one doesn't have the isusse), no obivous differences.
5. When I use systemctl disable docker and systemctl stop docker, the FIFO and RR task can work again.

expecting:

find out what occurs this isssue?

Answer 1

Few weeks go ahead, and I found the reason why Docker conflict with RT(intel RT-patch kernel) is that the CONFIG_RT_GROUP_SCHED is enabled in the kernel config. After I disable it, the docker won't be conflicted with RT-Patch, and then the tag in docker-compose.yml need to be commented like this:

#mem_limit : 300m
#memswap_limit : 0 
#mem_reservation : 250m