I am writing a stack profiler for python using Rust. Here is the profiler code to collect stack trace:
use pyo3::prelude::*;
use pyo3::types::IntoPyDict;
use pyo3::types::{PyAny, PyDict};
use pyo3::wrap_pyfunction;
use std::thread;
use std::time::Duration;
use pyo3::types::PyString;
use pyo3::types::PyTuple;
use pyo3::types::PyList;
/// Function to start the profiler
#[pyfunction]
fn start_profiler(py: Python, interval: u64) -> PyResult<()> {
thread::spawn(move || -> PyResult<()> {
loop {
thread::sleep(Duration::from_millis(interval));
let gil_guard = Python::acquire_gil();
let py = gil_guard.python();
println!("Profiler thread running");
// Collect stack trace of the thread holding the GIL
if let Ok(sys) = py.import("sys") {
if let Ok(frames) = sys.call1("_current_frames", ()) {
if let Ok(frame_dict) = frames.downcast::<PyDict>() {
println!("Active Python thread idents:");
for (key, _value) in frame_dict.iter() {
println!("{:?}", key);
}
if let Ok(threading) = py.import("threading") {
if let Ok(current_thread) = threading.call_method0("current_thread") {
if let Ok(ident) = current_thread.getattr("ident").and_then(|i| i.extract::<u64>()) {
match frame_dict.get_item(ident) {
Some(frame) => {
if let Ok(traceback) = py.import("traceback") {
if let Ok(trace) = traceback.call_method1("format_stack", (frame,)) {
println!("Stack trace of Python thread {}: {:?}", ident, trace);
} else {
println!("Failed to format stack trace for thread ident {}", ident);
}
}
},
None => println!("No frame found for the given ident: {}", ident),
}
} else {
println!("Failed to extract ident");
}
} else {
println!("Failed to get current thread");
}
} else {
println!("Failed to import threading");
}
} else {
println!("Failed to downcast to PyDict");
}
} else {
println!("Failed to call _current_frames");
}
} else {
println!("Failed to import sys");
}
}
});
Ok(())
}
/// A Python module implemented in Rust.
#[pymodule]
fn rust_profiler(_py: Python, m: &PyModule) -> PyResult<()>{
m.add_function(wrap_pyfunction!(start_profiler, m)?)?;
Ok(())
}
The intension of the code is to create a background thread to monitor the python application threads. During sampling, I intend to collect the stack trace of the python thread that holds the GIL last time. I am using "current_thread" to find the thread. However, the code cannot find the current thread's identifier in the "_current_frame" dictionary. Following is the output of the profiler. The last line prints the thread id inside the python code.
...
Profiler thread running
Active Python thread idents: 140345858855744
No frame found for the given ident: 140345831622400
Current thread ID: 140345858855744
Can you explain why I cannot monitor the python thread that was holding the GIL? Which current thread has an identifier 140345858855744? Is it the Rust thread that I spawning? How can I fix the profiler?