linux mbind call fails on HUGETLB memory

Question

I'm trying to use mbind to bind my memory to a specific NUMA node. I noticed it fails if my memory is allocated with mmap using MAP_HUGETLB, but mbind succeeds if I dont try to MAP_HUGETLB.

I definitely want both HUGE pages and to map the memory to a specific NUMA node. AmI doing something wrong, or is this not supported?

Running on CentOS7

//mbind fails on this
void * buf = mmap( NULL, sz, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE | MAP_HUGETLB, 0, 0 );
//works on this
void * buf2 = mmap( NULL, sz, PROT_READ | PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, 0, 0 );

int numaNode = 1;
unsigned long nodemask = 0;
nodemask |= 1 << numaNode;
if( mbind( buf, sz, MPOL_BIND, &nodemask, sizeof(nodemask) * 8, 0 ) < 0 )
    printf( "mbind buf failed: %s\n", strerror(errno) );

if( mbind( buf2, sz, MPOL_BIND, &nodemask, sizeof(nodemask) * 8, 0 ) < 0 )
    printf( "mbind buf2 failed: %s\n", strerror(errno) );

Edit: to be clear the mmap works fine and my machine is configured with huge pages. I didn’t check the mmap return code here for brevity

Answer 1

You don't check mmap return value for error. It fails to allocate huge pages unless you reserve them first.

Reserve huge pages with hugeadm, for example:

sudo hugeadm --pool-pages-min 2MB:4 --pool-pages-max 2MB:8 # 4-8 2MB pages
sudo hugeadm --pool-pages-min 1GB:2 --pool-pages-max 1GB:4 # 2-4 1GB pages

---

Another option is to use transparent huge pages without having to pass any extra flags to mmap. See Transparent Hugepage Support for full details.

Answer 2

I think this artilcle arlready solved under, please check for it.

link