Three.js relative forces while having a negative force

Question

I'm making a small and simple car simulator. I have a class called GameController where the physics handler is described. In there the absolute forces are converted to relative forces and applied to the position of the object. But now when I am accelerating the object by adding a positive force (pressing W) and then slow down by a backwards force it stops and doesn't proceed to go backwards even though it gets a negative force.

I know this is caused by the orientation of the player. I set it to follow the velocity but when the velocity becomes negative it flips the object around it's up axis and now forwards = backwards and the negative force is now headed the other way. It looks like it stands still now.

This is the Physics Handler in the GameController Class:

physicsHandler_(o)
    {
        var vn = new THREE.Vector3().copy(o.velocity_).normalize();

        var up = new THREE.Vector3(0, 1, 0);
        var right = new THREE.Vector3();
        
        right.crossVectors(up, vn).normalize();
        up.crossVectors(vn, right).normalize();
        
        var matrix = new THREE.Matrix4();
        matrix.set
        (
            right.x, up.x, vn.x, 0,
            right.y, up.y, vn.y, 0,
            right.z, up.z, vn.z, 0,
            0, 0, 0, 1
        );

        o.force_.applyQuaternion(o.quaternion_);
        o.acceleration_ = new THREE.Vector3(o.force_.x / o.mass_, o.force_.y / o.mass_, o.force_.z / o.mass_);
        o.velocity_.addScaledVector(o.acceleration_, delta);
        o.position_.addScaledVector(o.velocity_, delta);

        o.quaternion_.setFromRotationMatrix(matrix);
        o.mesh_.position.copy(o.position_);
        o.mesh_.quaternion.copy(o.quaternion_);

        o.force_.set(0, 0, 0);
    }

This is the input part:

gc.addToLoop_(function()
{
    if(gc.input_.keys_['w'])
    {
        car.force_ = new THREE.Vector3(car.force_.x, car.force_.y, car.force_.z + car.forceMotor_);
    }
    if(gc.input_.keys_['s'])
    {
        car.force_ = new THREE.Vector3(car.force_.x, car.force_.y, car.force_.z - car.forceMotorBackwards_);
    }
    if(gc.input_.keys_['a'])
    {
        car.force_.applyQuaternion(new THREE.Quaternion().setFromAxisAngle(new THREE.Vector3(0, 1, 0), -5));
        
    }
    if(gc.input_.keys_['d'])
    {
        car.force_.applyQuaternion(new THREE.Quaternion().setFromAxisAngle(new THREE.Vector3(0, 1, 0), 5));
    }
});

I tried to use Chat-GPT. Didn't give any clear answers.

Also tried to check if the angle between the previous quaternion was the other way around but didn't get any results.