Scipy's differential_evolution not running in multiple cores despite setting the workers parameter

Question

I want to run in multiple cores scipy's differential_evolution function with the workers parameter.

I have followed the instructions of the documentation and developed the following script to check the running time of the function both with and without the use of the workers parameter:

import time

if __name__ == '__main__':
    from scipy.optimize import differential_evolution, rosen
    bounds = [( 0, 2 ), ( 0, 2 ), ( 0, 2 ), ( 0, 2 ), ( 0, 2 )]

    print( 'Beginning optimisation without parallelisation' )
    start = time.time()
    result = differential_evolution( rosen, bounds,
                                     workers=1,
                                     popsize=200,
                                     maxiter=1000,
                                     tol=0.001,
                                     seed=1 )
    end = time.time()
    print( 'Optimisation finished after {} seconds'.format( round( end - start, 2 ) ) )

    print( '\nBeginning optimisation with workers parameter set to 10' )
    start = time.time()
    result = differential_evolution( rosen, bounds,
                                     updating='deferred',
                                     workers=10,
                                     popsize=200,
                                     maxiter=1000,
                                     tol=0.001,
                                     seed=1 )
    end = time.time()
    print( 'Optimisation finished after {} seconds'.format( round( end - start, 2 ) ) )

After running the script, these are the running times returned for each call:

Beginning optimisation without parallelisation
Optimisation finished after 59.0 seconds

Beginning optimisation with workers parameter set to 10
Optimisation finished after 60.43 seconds

As you can see, the execution with the workers parameter set to 10 is even slower than the first one. Can you help me with this? Is there anything else that I need to do in order to implement the parallelisation in the differential_evolution function?

I am currently using Python 3.8 and my scipy version is 1.8.1.

Answer 1

@9769953 is correct. There is overhead introduced by multiprocessing: creation of processes, parceling out the computations to the subprocesses, etc. This overhead is only overcome if the computation is CPU intensive, which rosen isn't.

If you have scipy >1.9 and if your objective function can be vectorised, then vectorisation may offer a speed gain. If you can eliminate intermediate calculations in the vectorisation, that are common to the entire calculation, then the gains will be larger:

import time
import numpy as np
from scipy.optimize import differential_evolution, rosen

bounds = [( 0, 2 ), ( 0, 2 ), ( 0, 2 ), ( 0, 2 ), ( 0, 2 )]


print( 'Beginning optimisation without parallelisation' )
start = time.time()
result = differential_evolution( rosen, bounds,
                                 workers=1,
                                 popsize=200,
                                 updating='deferred',
                                 seed=1 )
end = time.time()
print( f'Optimisation finished after {end - start} seconds')
   
print( '\nBeginning optimisation with vectorisation' )
start = time.time()
result = differential_evolution( rosen, bounds,
                                 updating='deferred',
                                 popsize=200,
                                 seed=1,
                                 vectorized=True
                               )
end = time.time()
print( f'Optimisation finished after {end-start} seconds')

gives:

Beginning optimisation without parallelisation
Optimisation finished after 46.94838500022888 seconds

Beginning optimisation with vectorisation
Optimisation finished after 40.34918713569641 seconds