I use the Spec 2006 benchmarks to measure the performance of a few private passes and the generated code. So far, all my work was based on LLVM 3.1.
Last week I wanted to move forward with LLVM, and I rebased all of my passes to LLVM 3.4. This required only adjusting a few #include lines. However, after that change I measured a huge performance regression. Next I disabled all of my custom passes, but still performance was down. Here are the numbers of a few Spec benchmarks compiled with LLVM 3.1 and 3.4
LLVM 3.1 LLVM 3.4
-------- --------
401.bzip2 7.50 17.5
429.mcf 4.72 8.10
456.hmmer 4.18 10.1
458.sjeng 4.85 8.86
433.milc 13.4 26.0
470.lbm 13.4 12.6
I also noticed that the number of functions called by e.g. 456.hmmer went from 6 to 45009; similar numbers show for most other benchmarks. Inlining failed?
I compile the Spec benchmarks with
CC=clang -g -std=c89 -D_GNU_SOURCE -c -emit-llvm
and then call opt with
opt -simplifycfg -mem2reg -break-constgeps <my-passes> benchmark.bc -o benchmark.schnufte.bc
Once opt is done, the bitcode is lowered with llc and linked into an executable.
The performance regression is very noticable, and I doubt that 3.4 would have been released with a regression as obvious as that. So my question is: did I miss something? Are optimizations now invoked more explicitly? I added -std-compile-opts to my opt call after my passes, but that didn't help at all.
EDIT As suggested, I posed the question to the LLVMdev mailing list, but the thread receives little attention.