I'm trying to create pool of workers which will perform some HTTP communication and compute values using results.
My initial approach looks like this:
class Worker
include Celluloid
...
end
pool = Worker.pool(size: 5)
collection.map do
arguments = compute_arguments
pool.future.external_call(arguments)
end.inject(0) { |acc, el| acc + el.value }
It works more or less as expected — it seems like 5 concurrent processes are running external_call simultaneously.
The problem is, that for the code is not "streamlined", and precomputes all arguments (these computations are much faster than external_call) up-front. I'd like my code to wait in the first block, since pool is unable to provide future that will start it's work immediately.
future is not waiting for pool to release worker (nor is async). How can I make these calls wait for free worker in pool (i.e. perform something like get from pool which would return worker that is ready to use)?