Why does `-fno-omit-frame-pointer` interfere with ASAN?

Question

During a recent project, I've tested combinations of different compiler flags and sanitizers to evaluate the relevance for debugging my C-code. By testing the impact of these combinations, I stumbled across a behavior that I did not understand.

Reproducer

I use a small hello-world code-example that contains a memory leak to trigger the address sanitizer (ASAN):

#include<stdlib.h>
#include<stdio.h>

int main () {
  int * memleak = calloc(1, sizeof(int)); // no free -> leaked memory
  printf ("A memleaked memory: %d\n", *memleak);
  printf ("Hello World\n"); // Note: I found that if I comment out this function, ASAN will also report again
}

Observation

I worked with different combinations of compiler- and linker-flags and sometimes I observed that the address sanitizer reported the memleak, while on other occasions it did not report the memleak. I've eliminated all potential compiler-flags until I found a minimal set of flags that influenced ASAN to either report or ignore the memory leak:

ASAN will report the memory leak when compiled with the commands

cc -fsanitize=address -fno-omit-frame-pointer -Og -o main.c.o -c ./main.c      && cc -o hello main.c.o -fsanitize=address,undefined && ./hello # returns 1
cc -fsanitize=address,undefined -Og -o main.c.o -c ./main.c                     && cc -o hello main.c.o -fsanitize=address,undefined && ./hello # returns 1
cc -fsanitize=address,undefined -fno-omit-frame-pointer -o main.c.o -c ./main.c && cc -o hello main.c.o -fsanitize=address,undefined && ./hello # returns 1

ASAN will not report the memory leak when compiled with the command

cc -fsanitize=address,undefined -fno-omit-frame-pointer -Og -o main.c.o -c ./main.c && cc -o hello main.c.o -fsanitize=address,undefined && ./hello # returns 0

However

I observe the same behavior, independent of using GCC or clang. Therefore, I'm concerned that this is not a bug, caused by an unintended interference between the different sanitizers, optimization level and the flag -fno-omit-frame-pointer, but instead it is intended behavior that I just am not able to understand, because of my lack of knowledge what the impact of -fno-omit-frame-pointer is.

If anybody could summarize what -fno-omit-frame-pointer/-fomit-frame-pointer does and in which cases it works, or explain the impact of this flag on the given example, or point me to the place where to find this information, I would be grateful.

For completeness

I'm working on Arch-linux and have the following versions of software running:

• gcc 11.1.0-1
• clang 13.0.0-2
• glibc 2.33-5

However, I've just tested and verified that the example and observations will also work on the docker image gcc:bullseye for linux/amd64 from docker-hub.

Answer 1

This is a known issue with LeakSanitizer: see e.g. #1233, #937 or #699. The core reason is that Lsan is a much simpler tool than Asan and does not guarantee that leak is detected. Moreover it's ability to detect a particular leak depends on layout of stack frames which can vary due to unrelated factors (like addition of frame pointers in your case).

Unfortunately there is no reliable solution for this issue - just autotest your application on as many compilers (gcc, clang) and/or platforms (x86, ARM, Android, etc.) as you can and some of them will catch a leak with high probability.