As described in Python API Doc, there are some params in onnxruntime session options coressponding to memory configurations such as:
- enable_cpu_mem_arena
- enable_mem_usage
- enable_mem_pattern
There are some descriptions for them but I can not understaned their usage and the technical concepts behind them precisely.
Can anyone give me some clarity about these parameters?
enable_mem_pattern
Enable the memory pattern optimization.
The idea is if the input shapes are the same, we could trace the internal memory allocation and generate a memory pattern for future request. So next time we could just do one allocation with a big chunk for all the internal memory allocation.
Note Memory pattern optimization is only available when Sequential Execution mode is enabled (see OrtApi::SetSessionExecutionMode)
enable_cpu_mem_arena
Enable the memory arena on CPU.
Arena may pre-allocate memory for future usage.
Info based on the api doc: https://onnxruntime.ai/docs/api/c/struct_ort_api.html