profiling simple python script with VTune in ubuntu

Question

I am trying to profile with VTune a simple python script.

import numpy as np

def my_function():
    res = 0
    for i in range(100000000):
        res = res + i
    final_res = np.log(res)
    return final_res


def my_function2():
    res = 0
    for i in range(200000000):
        res = res + i
    final_res = np.log(res)
    return final_res

print(my_function())
print(my_function2())

My goal is to find out cpu/time wastefull functions. That's why I choose hotspots analysis.

However there are plenty of "cannot locate debugging information" warnings:

Cannot locate file `'.
Cannot locate file `'.
Cannot locate debugging information for file `/home/chris/venv/lib/python3.8/site-packages/numpy/random/mtrand.cpython-38-x86_64-linux-gnu.so'.
Cannot locate debugging information for file `/home/chris/venv/lib/python3.8/site-packages/numpy/random/bit_generator.cpython-38-x86_64-linux-gnu.so'.
Cannot locate debugging information for file `/home/chris/venv/lib/python3.8/site-packages/numpy/core/_multiarray_umath.cpython-38-x86_64-linux-gnu.so'.
Cannot locate debugging information for file `/home/chris/venv/lib/python3.8/site-packages/numpy.libs/libopenblas64_p-r0-15028c96.3.21.so'.
Cannot locate debugging information for file `/opt/intel/oneapi/vtune/2023.2.0/lib64/libtpsstool.so'.

and results are not valid and not stable every time I run program (on screenshot seems clearly that my_function2 and my function1 are not the only 2 functions with most of the time)false_results I am pretty new to Vtune. Can you help me fix these warnings and have better more accurate results? I am working with VM Box with Ubuntu 22.04, (default python 3.10) but I use python3.8.10.

Answer 1

VTune Profiler supports the Hotspots, Threading, and Memory Consumption analysis for Python* applications via the Launch Application and Attach to Process modes. For example, when your application does excessive numerical modeling, you need to know how effectively it uses available CPU resources. A good example of the effective CPU usage is when the calculating process spends most time executing native extension and not interpreting Python glue code.

For more info : https://www.intel.com/content/www/us/en/docs/vtune-profiler/user-guide/2023-0/python-code-analysis.html#GUID-83F05898-C62B-496A-A537-86FCA6A57AE8

Hotspots analysis in the user-mode sampling mode helps identify sections of your Python code that take a long time to execute (hotspots), along with their timing metrics and call stacks. It also displays the workload distribution over threads in the Timeline pane.

Use the blow link to get a detailed information https://www.intel.com/content/www/us/en/docs/vtune-profiler/user-guide/2023-0/python-code-analysis.html#GUID-149EB75A-9FC2-424D-9AD2-556343073A21

For more reference please use the below link: https://www.intel.com/content/www/us/en/docs/vtune-profiler/user-guide/2023-0/python-code-analysis.html

VTune Profiler supports Python code profiling with some limitations:

1. Only Python distribution 2.6 and later are supported.
2. If you use Python extensions that compile Python code to the native language (JIT, C/C++), the VTune Profiler may show incorrect analysis results. Consider using JIT Profiling API to solve this problem.
3. Python code profiling is supported for Windows and Linux target systems only.
4. In some cases, the VTune Profiler may not resolve full names of Python functions and modules on Windows OS. It displays correct source information, so you can view the source directly from the VTune Profiler's viewpoints.
5. Proper thread names are not always displayed in the Timeline pane.
6. If your application has very low stack depth, which includes called functions and imported modules, the VTune Profiler does not collect Python data. Consider using deeper calls to enable the profiling.
7. When collecting data remotely, the VTune Profiler may not resolve full function or method names, and display the source code of your Python script. To solve this problem for Linux targets, copy the source files to a directory on your host system with a path identical to the path on your target system before running the analysis.

https://www.intel.com/content/www/us/en/docs/vtune-profiler/user-guide/2023-0/python-code-analysis.html#GUID-E2E03683-E617-4257-AEFD-431CA18CCB2D

Regarding the plenty of "cannot locate debugging information" warnings:

When collecting data remotely, the VTune Profiler may not resolve full function or method names, and display the source code of your Python script. That means you have no debug information for your or system binaries (inside of binary or in separate file, for example pdb)

just look on this troubleshooting content

https://software.intel.com/content/www/us/en/develop/documentation/vtune-help/top/troubleshooting/error-cannot-locate-debug-info.html

To solve this problem for Linux targets, copy the source files to a directory on your host system with a path identical to the path on your target system before running the analysis.