Here is the pybind11 code, which defined a Stream class and exported it to python3.
#include <functional>
#include <iostream>
#include <memory>
#include <pybind11/detail/common.h>
#include <pybind11/functional.h>
#include <pybind11/gil.h>
#include <pybind11/pybind11.h>
#include <pybind11/stl.h>
#include <thread>
namespace py = pybind11;
class Stream {
public:
std::thread *thr;
Stream() {}
void addCallback(std::function<void()> func) { thr = new std::thread(func); }
~Stream() {
if (thr != nullptr && thr->joinable()) {
thr->join();
delete thr;
}
std::cout << "stream dtor" << std::endl;
}
};
PYBIND11_MODULE(example, m) {
py::class_<Stream>(m, "Stream")
.def(py::init<>())
.def("add_callback", &Stream::addCallback);
}
The python3 code to test it:
def func():
print("xxx ")
a = Stream()
a.add_callback(func)
When execute this python script using python3 test.py, the program exits with no print.
There are 3 scenarios that destructor worked:
- do not add callback
def func():
print("xxx ")
a = Stream()
#a.add_callback(func)
- create the stream in a function, and call the function
def func():
print("xxx ")
def call_func():
a = Stream()
a.add_callback(func)
call_func()
- create the Stream as normal but before exit, call time.sleep(1)
def func():
print("xxx ")
a = Stream()
a.add_callback(func)
time.sleep(1) # or some other value
I have done some research for it. Here is my findings:
- The first scenario does not make Stream related with other threads, so it will be collected when python run gc.
- The second scenario does trouble me, I gauss it's because the gc can make sure that Stream instance will not be used again, so it's also collected.
- The third scenario kept python's gil from exiting, so that the callback will be executed by gil.
Sometimes, the callback will execute part of it, see the following code:
def func():
for i in range(10):
time.sleep(0.1)
print(i/10)
a = Stream()
a.add_callback(func)
time.sleep(0.5)
it will print:
0.0
0.1
0.2
0.3
Which means the callback it executed until the main thread finish the last line.
Here are my questions:
- When python is about to exit, assume it will run gc, destruct all objects including Stream, why doesn't it call Stream's destructor? It seemed the destructor is omitted and gil is gone before the callback finished.
- What happened after the last line finished?