I'm trying to multiprocess a function whose parameters contain a numpy.busdaycalendar object. It gives me this error:
TypeError: cannot pickle 'numpy.busdaycalendar' object
Below is a simple code to replicate the issue:
from concurrent.futures import ProcessPoolExecutor
import numpy as np
def shift_one_day(x,constant):
return np.busday_offset(x,1,busdaycal=constant)
def main():
constant = np.busdaycalendar()
variables = [np.datetime64('2023-03-21'),np.datetime64('2023-03-22'),np.datetime64('2023-03-23')]
with ProcessPoolExecutor() as executor:
futures_list = []
for v in variables:
futures_list.append(executor.submit(shift_one_day,v,constant))
result_list = [future.result() for future in futures_list]
print(result_list)
if __name__ == '__main__':
main()
My question is:
- Is there a way to make the busdaycalendar object pickle-able while maintaining its functionality?
- If not, is there any workaround in this situation? Side information is that in all the function calls of
shift_one_day, theconstantobject are all the same and doesn't need changing.
I'm using windows 10 so it seems no "fork" option for multiprocessing is available. Environment: python=3.12.2 numpy=1.26.4. Packages are from conda-forge.