Shadow moving across camera after setting activeFormat of AVCaptureDevice

Question

So I have a function that essentially walks through the formats of a device, finds the one I want with high resolution and frame rate, then sets that after the camera device inputs have been configured. For some reason though, setting the cameraDevice.activeFormat property causes the camera to have a shadow gradient move from right to left over the view and also looks like the camera is constantly attempted to focus (it 'jumps'). Not sure why. Any help is much appreciated. https://youtube.com/shorts/5h4ca1a-1EY?feature=share

func setUp() {

    // declaring variables

    let audioDevice = AVCaptureDevice.default(for: .audio)
    let cameraDevice = AVCaptureDevice.default(.builtInWideAngleCamera, for: .video, position: .back)
    var bestFormat: AVCaptureDevice.Format?
    var bestFrameRateRange: AVFrameRateRange?
    
    if let cameraDevice = cameraDevice, let audioDevice = audioDevice {
       
        // finding the format I want

        for format in cameraDevice.formats {
            let formatDesc = format.formatDescription
            let mediaSubType = formatDesc.mediaSubType
            let dimensions = formatDesc.dimensions
            
            for range in format.videoSupportedFrameRateRanges {
                if bestFormat == nil || ((dimensions.width >= bestFormat!.formatDescription.dimensions.width && dimensions.height >= bestFormat!.formatDescription.dimensions.height) && range.maxFrameRate >= bestFrameRateRange!.maxFrameRate) {
                    bestFormat = format
                    bestFrameRateRange = range
                }
            }
        }
        
        
        do {
            
            self.session.beginConfiguration()
            
            let videoInput = try AVCaptureDeviceInput(device: cameraDevice)
            let audioInput = try AVCaptureDeviceInput(device: audioDevice)
            
            if self.session.canAddInput(videoInput) && self.session.canAddInput(audioInput) {
                self.session.addInput(videoInput)
                self.session.addInput(audioInput)
            }
            if self.session.canAddOutput(self.output) {
                self.session.addOutput(output)
            }
            try cameraDevice.lockForConfiguration()
            

            // NOTE: The issue is setting the active format. If I don't set activeFormat, the shadow does NOT appear


            cameraDevice.activeFormat = bestFormat!
            var duration = bestFrameRateRange!.minFrameDuration
            if duration.seconds > 60 {
                duration = CMTime(seconds: 1, preferredTimescale: 60)
            }
            cameraDevice.activeVideoMinFrameDuration = duration
            cameraDevice.activeVideoMaxFrameDuration = duration
            
            cameraDevice.unlockForConfiguration()
            
            self.session.commitConfiguration()
            
        }
        catch {
            print(error.localizedDescription)
        }
    }
}

Answer 1

I'm not expert on that, but after little research: "The shadow gradient and jumping behavior might be occurring due to changes in camera settings triggered by switching to a new active format." You can try locking focus and exposure: After setting the activeFormat, lock the focus and exposure to prevent constant adjustments by the camera. You can do this using the lockForConfiguration() and unlockForConfiguration() methods of the camera device.

cameraDevice.lockForConfiguration()
if cameraDevice.isFocusModeSupported(.locked) {
    cameraDevice.focusMode = .locked
}
if cameraDevice.isExposureModeSupported(.locked) {
    cameraDevice.exposureMode = .locked
}
cameraDevice.unlockForConfiguration()

try also to "Smooth Transition" to avoid sudden changes, you can try making a smooth transition to the new format using the AVCaptureDevice.FormatTransition API. This API allows you to animate format changes, which might help prevent the shadow and jumping behavior :/ (not sure about that but it seems logical to me) and maybe reset camera settings to their default values before setting the new activeFormat. Try calling cameraDevice.lockForConfiguration() and then setting properties like focusMode, exposureMode, whiteBalanceMode, etc., to their default values. After that, set the new activeFormat, and then unlock the configuration

Hope it help ^^