I am trying to get idea how subprocesses work in the Mojolicious.
This is a code I am running:
use Mojo::Base -strict, -signatures;
use Mojo::IOLoop::Subprocess;
show "BEFORE";
for(0..5){
say "(START) This is $_ loop";
my $subprocess = Mojo::IOLoop::Subprocess->new;
$subprocess->run(
map {
sub {
my $sp = shift; # $subprocess
my $i = shift; # looking to get my param here
say "Hello, from $i!";
sleep 5;
say "Good bye, from $i!";
return 'Done', $i;
},
sub ($subprocess, $err, @results) {
say "Subprocess error: $err" and return if $err;
say "$results[0] SP: $results[1]!";
}
} $_ # assuming to send the $_ from the for() loop
);
$subprocess->ioloop->start unless $subprocess->ioloop->is_running;
say "(END) This is $_ loop";
}
show "AFTER";
This is the response:
======( "BEFORE" )===========[ 'Test.pm', line 168 ]======
"BEFORE"
(START) This is 0 loop
(END) This is 0 loop
(START) This is 1 loop
(END) This is 1 loop
(START) This is 2 loop
(END) This is 2 loop
(START) This is 3 loop
(END) This is 3 loop
(START) This is 4 loop
(END) This is 4 loop
(START) This is 5 loop
(END) This is 5 loop
======( "AFTER" )============[ 'Test.pm', line 197 ]======
"AFTER"
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 181.
Hello, from !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 181.
Hello, from !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 181.
Hello, from !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 181.
Hello, from !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 181.
Hello, from !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 181.
Hello, from !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 183.
Good bye, from !
Use of uninitialized value $results[1] in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 188.
Done SP: !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 183.
Good bye, from !
Use of uninitialized value $results[1] in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 188.
Done SP: !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 183.
Good bye, from !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 183.
Good bye, from !
Use of uninitialized value $results[1] in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 188.
Done SP: !
Use of uninitialized value $results[1] in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 188.
Done SP: !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 183.
Good bye, from !
Use of uninitialized value $i in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 183.
Good bye, from !
Use of uninitialized value $results[1] in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 188.
Done SP: !
Use of uninitialized value $results[1] in concatenation (.) or string at /home/sites/somesite/myapp/script/../lib/MyApp/Controller/Test.pm line 188.
Done SP: !
I have a few problems here:
- I cannot find the way how to run a process and hold on main sub until all of them are done (in parallel) (so, the
"AFTER"string must be run after all the subprocesses are finished). I am looking to run a number of subprocesses through the loop and need them to work in parallel and block the main program until all of them are done (the idea is like all() in the Promise, but accepting blocking code inside each subprocess, that won't block the other subprocesses to run in parallel). - I cannot pass a parameter into the subprocess. Trying to use map(), but it makes no sense - I always have only the default
$subprocess.
What is done:
I spent hours on Future async/await, Mojo async/await, Promise etc.., but it does not work if you have blocking code inside (like, for example, sleep() or perl built-in loops), which is predictable.
Subprocesses use different processes (it can be seen from $$ if I print it out from the subprocess, which gives always new value for each iteration). But I cannot find a way - how to get rid of described problems with it.
UPDATE 1
Thanking to @alexk comment I could defeat await for the processes to complete.
The code is below:
use Mojo::Base -strict, -signatures, -async_await;
use Mojo::IOLoop::Subprocess;
async sub testMethod{
show "BEFORE";
my @promises;
for(0..5){
say "(START) This is $_ loop";
my $subprocess = Mojo::IOLoop::Subprocess->new;
push @promises, $subprocess->run_p(
sub {
my $i = shift || 1;
#my $promise = Mojo::Promise->new;
say "Hello, from $i and $$!";
sleep 5;
say "Good bye, from $i and $$!";
#$promise->resolve("Done for $i");
#return $promise;
return "Done for $i";
}
)->then(sub ($_result) {
show $_result;
return $_result;
})->catch(sub {
my $err = shift;
say "Subprocess error: $err";
});
$subprocess->ioloop->start unless $subprocess->ioloop->is_running;
say "(END) This is $_ loop";
}
my @results = await Mojo::Promise->all_settled(@promises);
show @results;
show "AFTER";
}
What is left - to find out how to send parameter to this subprocess from the loop for()?
UPDATE 2
I have added parameter feed to the anon sub (with sub{}->(args)).
sub {
my $i = shift || 1;
my $promise = Mojo::Promise->new;
say "Hello, from $i and $$!";
sleep 2;
say "Good bye, from $i and $$!";
$promise->resolve("Done for $i");
return $promise;
#return "Done for $i";
}->($_)
But then calls became blocking (one by one). Also, it throws warnings. Here is an output:
======( "BEFORE" )===========[ 'Test.pm', line 189 ]======
"BEFORE"
(START) This is 0 loop
Hello, from 1 and 15396!
Good bye, from 1 and 15396!
(END) This is 0 loop
(START) This is 1 loop
Hello, from 1 and 15396!
Good bye, from 1 and 15396!
(END) This is 1 loop
(START) This is 2 loop
Hello, from 2 and 15396!
Good bye, from 2 and 15396!
(END) This is 2 loop
(START) This is 3 loop
Hello, from 3 and 15396!
Good bye, from 3 and 15396!
(END) This is 3 loop
(START) This is 4 loop
Hello, from 4 and 15396!
Good bye, from 4 and 15396!
(END) This is 4 loop
(START) This is 5 loop
Hello, from 5 and 15396!
Good bye, from 5 and 15396!
(END) This is 5 loop
Subprocess error: Can't locate object method "Promise=HASH(0xdc13e18)" via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
Subprocess error: Can't locate object method "Promise=HASH(0xdc0f008)" via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
Subprocess error: Can't locate object method "Promise=HASH(0xdc0a6c8)" via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
Subprocess error: Can't locate object method "Promise=HASH(0xdc03968)" via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
Subprocess error: Can't locate object method "Promise=HASH(0xdc134d0)" via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
Subprocess error: Can't locate object method "Promise=HASH(0xdc15988)" via package "Mojo" at /home/ubuntu/perl5/perlbrew/perls/perl-5.26.3/lib/site_perl/5.26.3/Mojo/IOLoop/Subprocess.pm line 53.
======( @results )===========[ 'Test.pm', line 224 ]======
[
{ status => "fulfilled", value => [1] },
{ status => "fulfilled", value => [1] },
{ status => "fulfilled", value => [1] },
{ status => "fulfilled", value => [1] },
{ status => "fulfilled", value => [1] },
{ status => "fulfilled", value => [1] },
]
======( "AFTER" )============[ 'Test.pm', line 233 ]======
"AFTER"
UPDATE 3 (SOLVED)
The code now works!
The problem was with use of for() loop.
I cannot use in the subprocess any var, created within the for() loop (for some reason). So, I could not use closure within this kind of loops.
Thus, I rewrote the code a little:
my @looper = (0..5);
foreach my $lp (@looper){
... subprocess preparation ...
sub {
my $i = $lp || 5;
return $i;
}
}
The only ting I would like to know - why for loops does not make me use vars, created within them..
The complete example:
use Mojo::Base -strict, -signatures, -async_await;
use Mojo::IOLoop::Subprocess;
use Mojo::Promise;
async sub testMehod{
$c->render_later;
my $start_timer1 = Time::HiRes::gettimeofday();
show "BEFORE";
my @promises;
my @looper = (0..5);
foreach my $lp (@looper){
say "(START) This is $lp loop";
my $subprocess = Mojo::IOLoop::Subprocess->new;
push @promises, $subprocess->run_p(
sub {
my $i = $lp || 5;
my $sp = shift; # $subprocess
#my $i = shift || 3;
#my $promise = Mojo::Promise->new;
say "Hello, from $i and $$!";
sleep 2;
say "Good bye, from $i and $$!";
#$promise->resolve("Done for $i");
#return $promise;
return "Done for $i";
#}->($_)
}
)->then(sub ($_result) {
show $_result;
return $_result;
})->catch(sub {
my $err = shift;
say "Subprocess error: $err";
});
$subprocess->ioloop->start unless $subprocess->ioloop->is_running;
say "(END) This is $lp loop";
}
my @results = await Mojo::Promise->all_settled(@promises);
show @results;
my $stop_timer1 = Time::HiRes::gettimeofday();
my $total1 = sprintf("%.5f\n", $stop_timer1 - $start_timer1);
return $c->render(text => "All done. Parallel: within $total1", status => 405);
}
Any help is appreciated!