Swift changing UINavigationBar's shadow affects its UIBarButtonItem when appearance change

Question

I'm trying to update the UINavigationBar's shadow like this

func setCustomShadow() {
    self.layer.shadowColor = UIColor.red.cgColor
    self.layer.shadowOpacity = 0.1
    self.layer.shadowOffset = CGSize(width: 0, height: 5)
    self.layer.shadowRadius = 0.25
        
    self.layer.masksToBounds = false
}

it works fine when you scroll down like this

But the issue will occur when it get's back on top, the shadow will apply on the UIBarButtonItem like this

I want its current behavior, I just want to prevent the shadow on my UIBarButtonItem.

The question is, How can I prevent it from occurring on the UIBarButtonItem?

EDIT: I tried using UINavigationBarAppearance and update the shadowImage like this

let appearance1 = UINavigationBarAppearance()
appearance1.shadowImage = UIImage(named: "custom_shadow_image")

compactAppearance = nil
standardAppearance = appearance1
scrollEdgeAppearance = nil

with this method, I get a desired result on navigation bar's white appearance like this

but I'm still getting an issue on it's transparent appearance, it still displaying the shadowImage like this

I tried playing with compactAppearance, standardAppearance and scrollEdgeAppearance but still didn't get the desired output.

I don't want the shadowImage to be displayed on transparent appearance of the navigation bar, how can I do that?