How to run a code when a Laravel Job try is killed by timeout (Horizon)

Question

I created a Laravel Job with 3 tries and timeout after 10 minutes. I am using Horizon.

I can handle the failure after 3 tries using the method failed, but how can I handle the timeout event each 3 tries of this job ?

Used for logging and feedback, I want my user to be notified when the first or second try fails and it will be retried later.

class MyJob implements ShouldQueue
{
    public $tries = 3;
    public $timeout = 600;
    
    // [...]

    public function failed(Throwable $exception)
    {
        // The failure of the 3 tries.
    }

    // Any method for catching each timeouts ?
}

Answer 1

Ok I found the solution.

TLDR;

Put a pcntl_signal at the beginning of your your job handle() and then you can do something like call a onTimeout() method :

public function handle()
{
    pcntl_signal(SIGALRM, function () {
        $this->onTimeout();
        exit;
    });

    // [...]
}

public function onTimeout()
{
    // This method will be called each
}

The story behind :

The Queue documentation says : The pcntl PHP extension must be installed in order to specify job timeouts.

So, digging into the pcntl PHP documentation I found interesting pcntl_* functions. And a call of pcntl_signal into Illuminate/Queue/Worker.php l221.

It looks that if we register a method using pcntl_signal it replace the previous handler. I tried to load the Laravel one using pcntl_signal_get_handler but I can't manage to call it. So the workaroud is to call exit; so Laravel will consider the process as lost and mark it as timeout (?). There is the 3 tries, the retry_after is respected, and at the last try the job fails ... It may be cleaner to keep the original handler, but as it work well on my case so I will stop investigate.

Answer 2

You may define the $failOnTimeout property on the job class

/**
 * Indicate if the job should be marked as failed on timeout.
 *
 * @var bool
 */
public $failOnTimeout = true;

https://laravel.com/docs/9.x/queues#failing-on-timeout

Answer 3

I dont think there is a method for that,

But you can do something like catch the Error thrown if the job fails and verify that its from timeout exception which I believe would throw the exception handler Symfony\Component\Process\Exception\ProcessTimedOutException.

Something like;

public function handle() {
    try {
        // run job
    } catch (\Throwable $exception) {   

        // manually fail it if attempt is more than twice
        if ($this->attempts() > 2)             
            $this->fail($exception);

        // Check if the error it timeout related
        if ( $exception instanceof \Symfony\Component\Process\Exception\ProcessTimedOutException ) {
            // Whatever you want to do when it fails due to timeout
        }
        
        // release the job back to queue after 5 seconds
        $this->release(5);
        return;
    }
}

Just try running a job and make sure it fails because of timeout, to verify the actual timeout class exception