java flight recorder(jfr) consumes 100% cpu when its supposed to have only 1-2% overhead

Question

the moment i start JFR from jcmd, cpu goes to 100%, and it takes minimum of 20mins to get back to normal after jfr has stopped. normally the java process consumes ~20% of cpu.
its a spring boot(2.7.5) based application with mix of kotlin(1.6.21) and java(17).

jvm version,

openjdk version "17.0.5" 2022-10-18
OpenJDK Runtime Environment Temurin-17.0.5+8 (build 17.0.5+8)
OpenJDK 64-Bit Server VM Temurin-17.0.5+8 (build 17.0.5+8, mixed mode, sharing)

linux version,

Linux production-vm 4.9.0-7-amd64 #1 SMP Debian 4.9.110-3+deb9u2 (2018-08-13) x86_64 GNU/Linux

Answer 1

Most likely, this is a manifestation of the JVM issue JDK-8324241.

When you start JDK Flight Recorder for the first time, it flushes (deoptimizes) all compiled methods. This can drastically affect performance: first, because all Java code starts executing in the interpreter; second, because JIT compiler threads start recompiling lots of methods at once.

Possible workarounds are:

• Start JFR from the very beginning, when there are not too many compiled methods yet.
• Add any JVM TI agent that acquires can_redefine_classes capability at JVM startup with -agentpath option.
• Use other low overhead profilers that do not suffer from the mentioned issue, e.g., async-profiler.

The bug has been fixed in JDK 23. The fix might be backported to JDK 21 and JDK 17 some time later, but nobody guarantees that so far.