I have a an object {} which holds a lot of sub-objects, it's basically like a large embedded key-value store of objects. They need to be in-memory for performance reasons (access latency to them is critical).
This object is loaded once on startup and remains static. The app is having large GC lags in the Major GC phase, I assume since it needs to also traverse this huge references tree. However, I know this is static, and I'd like to exclude this from GC somehow.
Maybe some v8 gc expert can also recommend how to trace whether this object is indeed being scanned by major GC? or how to speed up this scanning if it cannot be eliminated?
I tried using the obscure "Eternal" handle (native addon), making the parent object an Eternal, in hope the GC algorithm can then avoid scanning this huge tree, but while it does becomes eternal, it seems the Mark-Sweep has not gained any speed (or I'm not able to see the difference in a smaller test). I did make sure the object becomes Eternal and I did make sure this prevents it (and all children) from being collected but maybe not from being scanned.
Are there other advanced tricks I can use to "hide" this object from GC?
I guess another idea is to serialize this object to some sort of external memory (Buffer?) and deserialize on access, but as I said, I'm worried the latency penalty for access will be too much.
Update March 14th:
Adding an --trace-gc-nvp I collected of a heavy GC.
I'm looking at pause times and mark.main=1368.6 + mark.main=564.0 (which seems like time on the main thread?) that appeared together when I triggered a GC.
[18726:0x7fe875a16000] 549342 ms: pause=1421.3 mutator=480.8 gc=ms reduce_memory=1 time_to_safepoint=0.00 heap.prologue=0.00 heap.embedder_tracing_epilogue=0.00 heap.epilogue=0.00 heap.epilogue.reduce_new_space=0.04 heap.external.prologue=0.0 heap.external.epilogue=0.0 heap.external.weak_global_handles=1.2 clear=29 clear.dependent_code=0.0 clear.maps=0.6 clear.slots_buffer=0.0 clear.string_table=25.4 clear.weak_collections=0.6 clear.weak_lists=0.8 clear.weak_references=3.3 complete.sweep_array_buffers=0.0 epilogue=0.0 evacuate=11.9 evacuate.candidates=0.0 evacuate.clean_up=0.0 evacuate.copy=0.4 evacuate.prologue=0.0 evacuate.epilogue=0.0 evacuate.rebalance=0.0 evacuate.update_pointers=11.4 evacuate.update_pointers.to_new_roots=10.5 evacuate.update_pointers.slots.main=0.9 evacuate.update_pointers.weak=0.0 finish=0.5 finish.sweep_array_buffers=0.4 mark=1373.4 mark.finish_incremental=0.0 mark.roots=4.2 mark.main=1368.6 mark.weak_closure=0.5 mark.weak_closure.ephemeron=0.0 mark.weak_closure.ephemeron.marking=0.1 mark.weak_closure.ephemeron.linear=0.0 mark.weak_closure.weak_handles=0.2 mark.weak_closure.weak_roots=0.1 mark.weak_closure.harmony=0.0 mark.embedder_prologue=0.0 mark.embedder_tracing=0.0 prologue=0.3 sweep=1.2 sweep.code=0.0 sweep.map=0.0 sweep.old=0.8 incremental=0.0 incremental.finalize=0.0 incremental.finalize.body=0.0 incremental.finalize.external.prologue=0.0 incremental.finalize.external.epilogue=0.0 incremental.layout_change=0.0 incremental.sweep_array_buffers=0.0 incremental.sweeping=4.3 incremental.embedder_prologue=0.0 incremental.embedder_tracing=0.0 incremental_wrapper_tracing_longest_step=0.0 incremental_finalize_longest_step=0.0 incremental_finalize_steps_count=0 incremental_longest_step=0.0 incremental_steps_count=0 incremental_marking_throughput=454557 incremental_walltime_duration=0 background.mark=5465.6 background.sweep=0.0 background.evacuate.copy=0.0 background.evacuate.update_pointers=0.1 background.unmapper=0.0 unmapper=0.0 total_size_before=1220256112 total_size_after=1212858824 holes_size_before=11506904 holes_size_after=249456 allocated=3530448 promoted=9216 semi_space_copied=0 nodes_died_in_new=0 nodes_copied_in_new=0 nodes_promoted=0 promotion_ratio=0.3% average_survival_ratio=0.2% promotion_rate=74.4% semi_space_copy_rate=0.0% new_space_allocation_throughput=6877.3 unmapper_chunks=60 compaction_speed=27514
[18726:0x7fe875a16000] 550056 ms: pause=713.6 mutator=0.8 gc=ms reduce_memory=1 time_to_safepoint=0.00 heap.prologue=0.00 heap.embedder_tracing_epilogue=0.00 heap.epilogue=0.00 heap.epilogue.reduce_new_space=0.01 heap.external.prologue=0.0 heap.external.epilogue=0.0 heap.external.weak_global_handles=0.0 clear=9 clear.dependent_code=0.0 clear.maps=0.5 clear.slots_buffer=0.0 clear.string_table=6.7 clear.weak_collections=0.5 clear.weak_lists=0.1 clear.weak_references=2.2 complete.sweep_array_buffers=0.0 epilogue=0.0 evacuate=20.5 evacuate.candidates=0.0 evacuate.clean_up=0.0 evacuate.copy=2.2 evacuate.prologue=0.0 evacuate.epilogue=0.0 evacuate.rebalance=0.0 evacuate.update_pointers=18.3 evacuate.update_pointers.to_new_roots=15.3 evacuate.update_pointers.slots.main=3.0 evacuate.update_pointers.weak=0.0 finish=0.0 finish.sweep_array_buffers=0.0 mark=565.3 mark.finish_incremental=0.0 mark.roots=0.6 mark.main=564.0 mark.weak_closure=0.7 mark.weak_closure.ephemeron=0.0 mark.weak_closure.ephemeron.marking=0.0 mark.weak_closure.ephemeron.linear=0.0 mark.weak_closure.weak_handles=0.3 mark.weak_closure.weak_roots=0.1 mark.weak_closure.harmony=0.0 mark.embedder_prologue=0.0 mark.embedder_tracing=0.0 prologue=0.0 sweep=1.1 sweep.code=0.0 sweep.map=0.0 sweep.old=0.7 incremental=0.0 incremental.finalize=0.0 incremental.finalize.body=0.0 incremental.finalize.external.prologue=0.0 incremental.finalize.external.epilogue=0.0 incremental.layout_change=0.0 incremental.sweep_array_buffers=0.0 incremental.sweeping=4.3 incremental.embedder_prologue=0.0 incremental.embedder_tracing=0.0 incremental_wrapper_tracing_longest_step=0.0 incremental_finalize_longest_step=0.0 incremental_finalize_steps_count=0 incremental_longest_step=0.0 incremental_steps_count=0 incremental_marking_throughput=454557 incremental_walltime_duration=0 background.mark=2247.3 background.sweep=0.0 background.evacuate.copy=0.0 background.evacuate.update_pointers=4.5 background.unmapper=0.4 unmapper=0.0 total_size_before=1212842032 total_size_after=1212769512 holes_size_before=15545968 holes_size_after=276432 allocated=18446744073709534824 promoted=0 semi_space_copied=0 nodes_died_in_new=0 nodes_copied_in_new=0 nodes_promoted=0 promotion_ratio=0.3% average_survival_ratio=0.2% promotion_rate=74.4% semi_space_copy_rate=0.0% new_space_allocation_throughput=6871.3 unmapper_chunks=60 compaction_speed=196592
Also, I tried another little experiment, using a very interesting project called objectbuffer. IIUC it moves a JS object to a buffer representation, essentially taking it off the Old Space and into External Space (outside GC?), and indeed there is some improvement:
[74946:0x7f9ef0216000] 417345 ms: pause=416.4 mutator=161.3 gc=ms reduce_memory=1 time_to_safepoint=0.00 heap.prologue=0.00 heap.embedder_tracing_epilogue=0.00 heap.epilogue=0.00 heap.epilogue.reduce_new_space=0.01 heap.external.prologue=0.0 heap.external.epilogue=0.0 heap.external.weak_global_handles=0.0 clear=8 clear.dependent_code=0.0 clear.maps=0.4 clear.slots_buffer=0.0 clear.string_table=6.1 clear.weak_collections=0.5 clear.weak_lists=0.1 clear.weak_references=1.9 complete.sweep_array_buffers=0.0 epilogue=0.0 evacuate=35.2 evacuate.candidates=0.0 evacuate.clean_up=0.0 evacuate.copy=8.1 evacuate.prologue=0.0 evacuate.epilogue=0.1 evacuate.rebalance=0.0 evacuate.update_pointers=26.9 evacuate.update_pointers.to_new_roots=11.5 evacuate.update_pointers.slots.main=15.4 evacuate.update_pointers.weak=0.0 finish=4.2 finish.sweep_array_buffers=4.1 mark=364.7 mark.finish_incremental=0.0 mark.roots=0.7 mark.main=363.7 mark.weak_closure=0.4 mark.weak_closure.ephemeron=0.0 mark.weak_closure.ephemeron.marking=0.1 mark.weak_closure.ephemeron.linear=0.0 mark.weak_closure.weak_handles=0.2 mark.weak_closure.weak_roots=0.1 mark.weak_closure.harmony=0.0 mark.embedder_prologue=0.0 mark.embedder_tracing=0.0 prologue=0.1 sweep=1.2 sweep.code=0.0 sweep.map=0.0 sweep.old=0.7 incremental=0.0 incremental.finalize=0.0 incremental.finalize.body=0.0 incremental.finalize.external.prologue=0.0 incremental.finalize.external.epilogue=0.0 incremental.layout_change=0.0 incremental.sweep_array_buffers=0.0 incremental.sweeping=3.5 incremental.embedder_prologue=0.0 incremental.embedder_tracing=0.0 incremental_wrapper_tracing_longest_step=0.0 incremental_finalize_longest_step=0.0 incremental_finalize_steps_count=0 incremental_longest_step=0.0 incremental_steps_count=0 incremental_marking_throughput=600777 incremental_walltime_duration=0 background.mark=1447.7 background.sweep=0.0 background.evacuate.copy=30.7 background.evacuate.update_pointers=57.1 background.unmapper=15.9 unmapper=0.0 total_size_before=1187271352 total_size_after=1184864520 holes_size_before=127885080 holes_size_after=363824 allocated=1147072 promoted=4168 semi_space_copied=0 nodes_died_in_new=0 nodes_copied_in_new=0 nodes_promoted=0 promotion_ratio=0.3% average_survival_ratio=0.8% promotion_rate=34.7% semi_space_copy_rate=0.0% new_space_allocation_throughput=6552.1 unmapper_chunks=11 compaction_speed=260826
[74946:0x7f9ef0216000] 417902 ms: pause=557.0 mutator=0.5 gc=ms reduce_memory=1 time_to_safepoint=0.00 heap.prologue=0.00 heap.embedder_tracing_epilogue=0.00 heap.epilogue=0.00 heap.epilogue.reduce_new_space=0.01 heap.external.prologue=0.0 heap.external.epilogue=0.0 heap.external.weak_global_handles=0.0 clear=8 clear.dependent_code=0.0 clear.maps=0.4 clear.slots_buffer=0.0 clear.string_table=6.4 clear.weak_collections=0.5 clear.weak_lists=0.1 clear.weak_references=1.8 complete.sweep_array_buffers=0.0 epilogue=0.0 evacuate=42.1 evacuate.candidates=0.0 evacuate.clean_up=0.0 evacuate.copy=9.0 evacuate.prologue=0.0 evacuate.epilogue=0.1 evacuate.rebalance=0.0 evacuate.update_pointers=33.0 evacuate.update_pointers.to_new_roots=14.9 evacuate.update_pointers.slots.main=18.1 evacuate.update_pointers.weak=0.0 finish=0.0 finish.sweep_array_buffers=0.0 mark=413.5 mark.finish_incremental=0.0 mark.roots=0.6 mark.main=412.5 mark.weak_closure=0.4 mark.weak_closure.ephemeron=0.0 mark.weak_closure.ephemeron.marking=0.0 mark.weak_closure.ephemeron.linear=0.0 mark.weak_closure.weak_handles=0.2 mark.weak_closure.weak_roots=0.1 mark.weak_closure.harmony=0.0 mark.embedder_prologue=0.0 mark.embedder_tracing=0.0 prologue=0.0 sweep=1.2 sweep.code=0.0 sweep.map=0.0 sweep.old=0.7 incremental=0.0 incremental.finalize=0.0 incremental.finalize.body=0.0 incremental.finalize.external.prologue=0.0 incremental.finalize.external.epilogue=0.0 incremental.layout_change=0.0 incremental.sweep_array_buffers=0.0 incremental.sweeping=3.5 incremental.embedder_prologue=0.0 incremental.embedder_tracing=0.0 incremental_wrapper_tracing_longest_step=0.0 incremental_finalize_longest_step=0.0 incremental_finalize_steps_count=0 incremental_longest_step=0.0 incremental_steps_count=0 incremental_marking_throughput=600777 incremental_walltime_duration=0 background.mark=1642.3 background.sweep=0.0 background.evacuate.copy=34.2 background.evacuate.update_pointers=62.1 background.unmapper=16.2 unmapper=0.0 total_size_before=1184859816 total_size_after=1184749368 holes_size_before=96876512 holes_size_after=478768 allocated=18446744073709546912 promoted=0 semi_space_copied=0 nodes_died_in_new=0 nodes_copied_in_new=0 nodes_promoted=0 promotion_ratio=0.3% average_survival_ratio=0.8% promotion_rate=34.7% semi_space_copy_rate=0.0% new_space_allocation_throughput=6488.6 unmapper_chunks=11 compaction_speed=305301
The GC must visit all live objects to mark them, there is no way around that.
For numerical data, you could use TypedArrays (or Buffers), as their contents aren't visited by the GC (because there are no pointers there anyway). It sounds like you need to store objects though, and in that case I agree with the concern that deserializing on demand might have too much overhead.
Since Major GC cycles are incremental and partially concurrent, seeing long pauses there is unusual. I would suspect that the actual reason for these pauses is not directly related to your large object tree. Perhaps a performance profile could shed some light on what's happening.