ld finds library when run stand along but cant find when building executable

Question

I am trying to build a program that requires the linking of libndctl. However, ld fails to find the library, and the build errors out at the linking stage stating (this used to build without error some time before):

Command:

g++ tatp_db.cc tatp_nvm.cc ../include/txopt.cc -lpmem -lpthread  -o tatp_nvm -std=c++11 -static

Error:

  /usr/bin/ld: /usr/local/lib/libpmem.a(libpmem_all.o): in function `pmem2_device_dax_alignment':
memset_t_avx512f.c:(.text+0xb527): undefined reference to `ndctl_new'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xb59e): undefined reference to `ndctl_namespace_get_dax'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xb5b5): undefined reference to `ndctl_dax_get_align'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xb5ce): undefined reference to `ndctl_unref'
/usr/bin/ld: /usr/local/lib/libpmem.a(libpmem_all.o): in function `pmem2_device_dax_size':
memset_t_avx512f.c:(.text+0xb62f): undefined reference to `ndctl_new'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xb6a9): undefined reference to `ndctl_namespace_get_dax'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xb6c0): undefined reference to `ndctl_dax_get_size'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xb700): undefined reference to `ndctl_unref'
/usr/bin/ld: /usr/local/lib/libpmem.a(libpmem_all.o): in function `pmem2_region_namespace':
memset_t_avx512f.c:(.text+0xc3ab): undefined reference to `ndctl_bus_get_first'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc3c0): undefined reference to `ndctl_region_get_first'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc3d5): undefined reference to `ndctl_namespace_get_first'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc3f2): undefined reference to `ndctl_namespace_get_dax'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc41e): undefined reference to `ndctl_dax_get_daxctl_region'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc463): undefined reference to `daxctl_dev_get_first'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc475): undefined reference to `daxctl_dev_get_devname'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc4e1): undefined reference to `daxctl_dev_get_next'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc512): undefined reference to `ndctl_namespace_get_btt'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc529): undefined reference to `ndctl_btt_get_block_device'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc53b): undefined reference to `ndctl_namespace_get_pfn'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc552): undefined reference to `ndctl_pfn_get_block_device'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc564): undefined reference to `ndctl_namespace_get_block_device'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc5d1): undefined reference to `ndctl_namespace_get_next'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc5ec): undefined reference to `ndctl_region_get_next'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc607): undefined reference to `ndctl_bus_get_next'
/usr/bin/ld: /usr/local/lib/libpmem.a(libpmem_all.o): in function `pmem2_get_region_id':
memset_t_avx512f.c:(.text+0xc64f): undefined reference to `ndctl_new'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc6f8): undefined reference to `ndctl_region_get_id'
/usr/bin/ld: memset_t_avx512f.c:(.text+0xc70a): undefined reference to `ndctl_unref'
collect2: error: ld returned 1 exit status
make: *** [Makefile:9: all] Error 1

I tried fixing this by manually adding -lndctl to the build command

g++ tatp_db.cc tatp_nvm.cc ../include/txopt.cc -lpmem -lpthread -lndctl -o tatp_nvm -std=c++11 -static

Error:

/usr/bin/ld: cannot find -lndctl
collect2: error: ld returned 1 exit status
make: *** [Makefile:9: all] Error 1

I see that the library exists in /usr/lib/ and when running ld stand-alone (/usr/bin/ld -lndctl --verbose) it finds the library. Does anyone know what causes this to happen and a fix?

Answer 1

Command: g++ tatp_db.cc tatp_nvm.cc ../include/txopt.cc -lpmem -lpthread -o tatp_nvm -std=c++11 -static

That command is wrong: you said yourself that this binary requires libndtcl, yet you aren't listing that library on the command line, so of course its symbols end up being unresolved.

I tried fixing this by manually adding -lndctl to the build command

That is the correct fix.

/usr/bin/ld: cannot find -lndctl
I see that the library exists in /usr/lib/

There are a few likely root causes:

1. If you are linking a 64-bit executable (which seems likely), /usr/lib/libndctl* may exist, but is a 32-bit binary. You need a 64-bit version (usually found in /usr/lib64) and that version is not present.
2. You have only libndctl.so*, but not libndtcl.a (the latter is required for -static link). To fix, you would need to install libndctl-dev or similar package.