Three.js DecalGeometry looks different on perpendicular faces?

Question

I'm making an FPS shooter game, and I'm currently trying to add a bullet hole to the guns. It works on certain faces, but on the PERPENDICULAR faces, it looks completely messed up.

Working as expected:

Doesn't look correct on any perpendicular face:

This is super confusing to me... here is my code:

let raycaster = new THREE.Raycaster();
raycaster.setFromCamera(new THREE.Vector2(), camera);
let intersects = raycaster.intersectObjects(scene.children, true);

if (intersects.length > 0) {
    let intersectionPoint = intersects[0].point;

    let decalMaterial = new THREE.MeshBasicMaterial({
        map: textureLoader.load('../Assets/Images/bullet.png'),
        transparent: true,
        depthWrite: false,
        polygonOffset: true,
        polygonOffsetFactor: -4
    });

    let decalGeometry = new DecalGeometry(
        intersects[0].object,
        intersectionPoint,
        raycaster.ray.direction,
        new THREE.Vector3(0.5, 0.5, 0.5)
    );

    let decalMesh = new THREE.Mesh(decalGeometry, decalMaterial);
    scene.add(decalMesh);
}

There aren't any errors in the console. Any help is appreciated. Thanks