I have an issue when trying to send multiple data to the root process from multiple processes.
Here's what I'm trying to do: Every process (rank > 0) sends one integer and one double in this order. The root process receives the integer and the double. It seems a very easy problem, but I can't figure this out.
I implemented in this way:
max = comm_size;
if(comm_size > ms->n_params){
max = ms->n_params;
}
for(r = 1; r < max; r++){
MPI_Recv(&i, 1, MPI_INT, r, 0, MPI_COMM_WORLD, NULL);
MPI_Recv(&f1, 1, MPI_DOUBLE, r, 0, MPI_COMM_WORLD, NULL);
...
...
}
...
printf("\n\033[1;92m[+]\033[1;0m Best hyperparameters found:\n");
}else{
MPI_Send(&best_hp, 1, MPI_INT, 0, 0, MPI_COMM_WORLD);
MPI_Send(&best_f1, 1, MPI_DOUBLE, 0, 0, MPI_COMM_WORLD);
}
It doesn't work as I expect. The printf gets executed before all processes have finished to work. It seems to work if I split the receiving loop in two loops. The first one waits for the integers and the second for the doubles, but I want to understand what's wrong with this implementation. Recv should be blocking, so I don't expect the root process to exit earlier. Send could be buffered as I send very little data, but in any case I don't expect this behavior.