I am trying to investigate the behavior of memory allocators in different versions of glibc. I am using x86-64 Ubuntu 22.04 LTS in WSL.
Building glibc-2.31
I built glibc-2.31 from source with this configure command from a build directory:
$ /home/<username>/glibc-src/glibc-2.31/configure --prefix=/home/<username>/glibc-2.31 --disable-werror --enable-kernel=3.2 --host=x86_64-linux-gnu --build=x86_64-linux-gnu CC=gcc CXX=g++
Both the gcc and g++ version is 11.4.0. I'm not sure if this is important, but glibc-2.35 was already installed at the default location.
I also modified Makefile and support/Makefile in the source to solve a linker error while building, but I doubt this is causing the segfaults.
Makefile
577 #ifeq (,$(CXX))
578 #LINKS_DSO_PROGRAM = links-dso-program-c
579 #else
580 #LINKS_DSO_PROGRAM = links-dso-program
581 #endif
582 LINKS_DSO_PROGRAM = links-dso-program-c
support/Makefile
191 #ifeq (,$(CXX))
192 #LINKS_DSO_PROGRAM = links-dso-program-c
193 #else
194 #LINKS_DSO_PROGRAM = links-dso-program
195 #LDLIBS-links-dso-program = -lstdc++ -lgcc -lgcc_s $(libunwind)
196 #endif
197 LINKS_DSO_PROGRAM = links-dso-program-c
Running make results in a successful build, and I ran make install afterwards to install the binaries at /home/<username>/glibc-2.31.
On Ubuntu 20.04: It may be worth noting that keeping the original Makefile and support/Makefile in the glibc source, and not making the modifications shown above, did not produce a linker error. I tried this in a Ubuntu 20.04 Docker container.
Segfaults from using custom glibc
Compiling and running C programs with the default system glibc on Ubuntu 22.04 works fine. However, linking a C program against this custom glibc built in Ubuntu 22.04 and using the dynamic linker from the build results in a segfault.
// test.c
int main()
{
return 0;
}
$ gcc -o test test.c -L/home/<username>/glibc-2.31/lib -Wl,-rpath=/home/<username>/glibc-2.31/lib -Wl,-dynamic-linker=/home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
$ ldd test
linux-vdso.so.1 (0x00007ffede365000)
libc.so.6 => /home/<username>/glibc-2.31/lib/libc.so.6 (0x00007f906d23b000)
/home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007f906d403000)
$ ./test
Segmentation fault
GDB shows that the segfault happens inside the .text segment of the ld-linux-x86-64.so.2 linker:
gef➤ context
[ Legend: Modified register | Code | Heap | Stack | String ]
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── registers ────
$rax : 0x00007fffffffdec0 → 0x0000002000000000
$rbx : 0x00007fffffffdee0 → 0x00007ffff7ff413a → "LINUX_2.6"
$rcx : 0x00007ffff7ffeae8 → 0x00007ffff7ffea10 → 0x00007ffff7ffe778 → 0x00007ffff7ffe750 → 0x00007ffff7fcf000 → 0x00010102464c457f
$rdx : 0x00007fffffffdeb0 → 0x00007fffffffdec0 → 0x0000002000000000
$rsp : 0x00007fffffffdde8 → 0x00007fffffffdee0 → 0x00007ffff7ff413a → "LINUX_2.6"
$rbp : 0x00007fffffffdeb0 → 0x00007fffffffdec0 → 0x0000002000000000
$rsi : 0x00007ffff7ffe750 → 0x00007ffff7fcf000 → 0x00010102464c457f
$rdi : 0x00007ffff7ff4144 → "__vdso_clock_gettime"
$rip : 0x00007ffff7fdc204 → <_dl_lookup_symbol_x+20> mov %fs:0x10, %rax
$r8 : 0x00007fffffffdee0 → 0x00007ffff7ff413a → "LINUX_2.6"
$r9 : 0x0
$r10 : 0x6ffffe35
$r11 : 0x70000022
$r12 : 0x00007ffff7ff4144 → "__vdso_clock_gettime"
$r13 : 0x00007ffff7ffe750 → 0x00007ffff7fcf000 → 0x00010102464c457f
$r14 : 0x0
$r15 : 0x00007ffff7ffe1a0 → 0x0000555555554000 → jg 0x555555554047
$eflags: [ZERO carry PARITY adjust sign trap INTERRUPT direction overflow RESUME virtualx86 identification]
$cs: 0x33 $ss: 0x2b $ds: 0x00 $es: 0x00 $fs: 0x00 $gs: 0x00
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── stack ────
0x00007fffffffdde8│+0x0000: 0x00007fffffffdee0 → 0x00007ffff7ff413a → "LINUX_2.6" ← $rsp
0x00007fffffffddf0│+0x0008: 0x00007fffffffe020 → 0x00007ffff7fd34c0 → <dl_main+0> endbr64
0x00007fffffffddf8│+0x0010: 0x00007fffffffdeb0 → 0x00007fffffffdec0 → 0x0000002000000000
0x00007fffffffde00│+0x0018: 0x00007ffff7ffe750 → 0x00007ffff7fcf000 → 0x00010102464c457f
0x00007fffffffde08│+0x0020: 0x0000000000000000
0x00007fffffffde10│+0x0028: 0x00007ffff7ffe1a0 → 0x0000555555554000 → jg 0x555555554047
0x00007fffffffde18│+0x0030: 0x00007ffff7fd452e → <dl_main+4206> mov -0x170(%rbp), %rdx
0x00007fffffffde20│+0x0038: 0x0000000000000000
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── code:x86:64 ────
0x7ffff7fdc1ff <_dl_lookup_symbol_x+15> push %rbp
0x7ffff7fdc200 <_dl_lookup_symbol_x+16> mov %rdx, %rbp
0x7ffff7fdc203 <_dl_lookup_symbol_x+19> push %rbx
→ 0x7ffff7fdc204 <_dl_lookup_symbol_x+20> mov %fs:0x10, %rax
0x7ffff7fdc20d <_dl_lookup_symbol_x+29> sub $0xa8, %rsp
0x7ffff7fdc214 <_dl_lookup_symbol_x+36> mov %rsi, 0x10(%rsp)
0x7ffff7fdc219 <_dl_lookup_symbol_x+41> mov %rcx, 0x20(%rsp)
0x7ffff7fdc21e <_dl_lookup_symbol_x+46> mov %r8, 0x8(%rsp)
0x7ffff7fdc223 <_dl_lookup_symbol_x+51> mov %r9d, 0x1c(%rsp)
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── source:dl-lookup.c+646 ────
641 /* Make sure nobody can unload the object while we are at it. */
642 if (__glibc_unlikely (flags & DL_LOOKUP_GSCOPE_LOCK))
643 {
644 /* We can't just call __rtld_lock_lock_recursive (GL(dl_load_lock))
645 here, that can result in ABBA deadlock. */
→ 646 THREAD_GSCOPE_RESET_FLAG ();
647 __rtld_lock_lock_recursive (GL(dl_load_lock));
648 /* While MAP value won't change, after THREAD_GSCOPE_RESET_FLAG ()
649 it can e.g. point to unallocated memory. So avoid the optimizer
650 treating the above read from MAP->l_serial as ensurance it
651 can safely dereference it. */
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── threads ────
[#0] Id 1, Name: "test", stopped 0x7ffff7fdc204 in _dl_lookup_symbol_x (), reason: SIGSEGV
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── trace ────
[#0] 0x7ffff7fdc204 → _dl_lookup_symbol_x(undef_name=0x7ffff7ff4144 "__vdso_clock_gettime", undef_map=0x7ffff7ffe750, ref=0x7fffffffdeb0, symbol_scope=0x7ffff7ffeae8, version=0x7fffffffdee0, type_class=0x0, flags=0x0, skip_map=0x0)
[#1] 0x7ffff7fd452e → dl_vdso_vsym(name=0x7ffff7ff4144 "__vdso_clock_gettime")
[#2] 0x7ffff7fd452e → setup_vdso_pointers()
[#3] 0x7ffff7fd452e → dl_main(phdr=<optimized out>, phnum=<optimized out>, user_entry=<optimized out>, auxv=<optimized out>)
[#4] 0x7ffff7febbab → _dl_sysdep_start(start_argptr=0x7fffffffe170, dl_main=0x7ffff7fd34c0 <dl_main>)
[#5] 0x7ffff7fd2fe8 → _dl_start_final(arg=0x7fffffffe170)
[#6] 0x7ffff7fd2fe8 → _dl_start(arg=0x7fffffffe170)
[#7] 0x7ffff7fd2108 → _start()
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
gef➤
gef➤ i file
Symbols from "/home/<username>/test/test".
Native process:
Using the running image of child process 11178.
While running this, GDB does not access memory from...
Local exec file:
`/home/<username>/test/test', file type elf64-x86-64.
Entry point: 0x555555555040
0x0000555555554318 - 0x0000555555554349 is .interp
0x0000555555554350 - 0x0000555555554380 is .note.gnu.property
0x0000555555554380 - 0x00005555555543a4 is .note.gnu.build-id
0x00005555555543a4 - 0x00005555555543c4 is .note.ABI-tag
0x00005555555543c8 - 0x00005555555543ec is .gnu.hash
0x00005555555543f0 - 0x0000555555554480 is .dynsym
0x0000555555554480 - 0x0000555555554519 is .dynstr
0x000055555555451a - 0x0000555555554526 is .gnu.version
0x0000555555554528 - 0x0000555555554548 is .gnu.version_r
0x0000555555554548 - 0x0000555555554608 is .rela.dyn
0x0000555555555000 - 0x000055555555501b is .init
0x0000555555555020 - 0x0000555555555030 is .plt
0x0000555555555030 - 0x0000555555555040 is .plt.got
0x0000555555555040 - 0x0000555555555138 is .text
0x0000555555555138 - 0x0000555555555145 is .fini
0x0000555555556000 - 0x0000555555556004 is .rodata
0x0000555555556004 - 0x0000555555556030 is .eh_frame_hdr
0x0000555555556030 - 0x00005555555560c4 is .eh_frame
0x0000555555557de0 - 0x0000555555557de8 is .init_array
0x0000555555557de8 - 0x0000555555557df0 is .fini_array
0x0000555555557df0 - 0x0000555555557fc0 is .dynamic
0x0000555555557fc0 - 0x0000555555558000 is .got
0x0000555555558000 - 0x0000555555558010 is .data
0x0000555555558010 - 0x0000555555558018 is .bss
0x00007ffff7fd12a8 - 0x00007ffff7fd12c8 is .note.gnu.property in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd12c8 - 0x00007ffff7fd12ec is .note.gnu.build-id in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd12f0 - 0x00007ffff7fd13c4 is .hash in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd13c8 - 0x00007ffff7fd14c0 is .gnu.hash in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd14c0 - 0x00007ffff7fd17f0 is .dynsym in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd17f0 - 0x00007ffff7fd1a14 is .dynstr in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd1a14 - 0x00007ffff7fd1a58 is .gnu.version in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd1a58 - 0x00007ffff7fd1afc is .gnu.version_d in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd1b00 - 0x00007ffff7fd1f20 is .rela.dyn in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd1f20 - 0x00007ffff7fd1fc8 is .rela.plt in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd2000 - 0x00007ffff7fd2080 is .plt in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd2080 - 0x00007ffff7fd2090 is .plt.got in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd2090 - 0x00007ffff7fd2100 is .plt.sec in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fd2100 - 0x00007ffff7ff2504 is .text in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7ff3000 - 0x00007ffff7ff7bfb is .rodata in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7ff7bfc - 0x00007ffff7ff8328 is .eh_frame_hdr in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7ff8328 - 0x00007ffff7ffabac is .eh_frame in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7ffc520 - 0x00007ffff7ffce64 is .data.rel.ro in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7ffce68 - 0x00007ffff7ffcfd8 is .dynamic in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7ffcfd8 - 0x00007ffff7ffcfe8 is .got in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7ffd000 - 0x00007ffff7ffd050 is .got.plt in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7ffd060 - 0x00007ffff7ffdff8 is .data in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7ffe000 - 0x00007ffff7ffe198 is .bss in /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
0x00007ffff7fcf120 - 0x00007ffff7fcf164 is .hash in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcf168 - 0x00007ffff7fcf1b8 is .gnu.hash in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcf1b8 - 0x00007ffff7fcf2d8 is .dynsym in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcf2d8 - 0x00007ffff7fcf34a is .dynstr in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcf34a - 0x00007ffff7fcf362 is .gnu.version in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcf368 - 0x00007ffff7fcf3a0 is .gnu.version_d in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcf3a0 - 0x00007ffff7fcf4b0 is .dynamic in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcf4b0 - 0x00007ffff7fcf504 is .note in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcf504 - 0x00007ffff7fcf540 is .eh_frame_hdr in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcf540 - 0x00007ffff7fcf61c is .eh_frame in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcf620 - 0x00007ffff7fcfc15 is .text in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcfc15 - 0x00007ffff7fcfcb1 is .altinstructions in system-supplied DSO at 0x7ffff7fcf000
0x00007ffff7fcfcb1 - 0x00007ffff7fcfce5 is .altinstr_replacement in system-supplied DSO at 0x7ffff7fcf000
gef➤
gef➤ i shared
From To Syms Read Shared Object Library
0x00007ffff7fd2100 0x00007ffff7ff2504 Yes /home/<username>/glibc-2.31/lib/ld-linux-x86-64.so.2
gef➤
On Ubuntu 22.04: I have also tried building glibc-2.32 and glibc-2.33 and linking programs against those versions of glibc. Version 2.32 also produces a segfault, but version 2.33 makes programs run fine.
On Ubuntu 20.04: In addition to not needing to modify the Makefiles mentioned above, building glibc-2.31 on Ubuntu 20.04 and linking C programs against the custom glibc-2.31 results in normal execution without segfaults. After moving the libc-2.31.so and ld-linux-x86-64.so.2 files from the custom glibc-2.31 built in Ubuntu 20.04 to the Ubuntu 22.04 environment, and linking C programs against those files in Ubuntu 22.04, programs also runs normally without immediate segfaults.
What is causing the segfaults when using glibc-2.31 built in Ubuntu 22.04? Did I incorrectly configure glibc before building? Maybe Ubuntu 22.04 has files that makes glibc-2.31 build incorrectly?
We don't know and can't tell: you had some error (which you didn't tell us about) which you fixed in some way (which you didn't tell us about). I suspect that your fix may in fact be incorrect, since everything works when GLIBC-2.31 is built without your fix (albeit on an older system, but that shouldn't matter).
Everything else you are doing appears to be correct, and works fine.
I suggest that you ask a different question: "I am trying to build GLIBC-2.31 on Ubuntu 22.04 and encounter the following error ... How do I fix that?"