AVAssetWriter used with ScreenCaptureKit leads to corrupt files and errors

Question

I'm trying to use ScreenCaptureKit to write an app that will record meetings. However, anytime I instantiate an AVAssetWriter and start streaming CMSampleBuffers to it, it will fail with the error

Optional(Error Domain=AVFoundationErrorDomain Code=-11800 \"The operation could not be completed\" UserInfo={NSLocalizedFailureReason=An unknown error occurred (-12785), NSLocalizedDescription=The operation could not be completed, NSUnderlyingError=0x600000c066d0 {Error Domain=NSOSStatusErrorDomain Code=-12785 \"(null)\"}})

I've got a full code reproduction at this repo - https://github.com/jonluca/buggy-avassetwriter

What's weird is that this repo works, where it's ~95% the same code https://github.com/garethpaul/ScreenRecorderMacOS

I might just not be attuned to the intricacies of AVAssetWriter and the newer ScreenCaptureKit, but I'm just not sure how this error can be happening.

Answer 1

Alright this took a while to figure out but it turns out that you can't only check if a CMSampleBuffer is valid. It looks like it can still be invalid, so I added a util called isValidFrame that checks the SCFrameStatus option, as well as a few other keys

    private func isValidFrame(for sampleBuffer: CMSampleBuffer) -> Bool {

        // Retrieve the array of metadata attachments from the sample buffer.
        guard let attachmentsArray = CMSampleBufferGetSampleAttachmentsArray(sampleBuffer,
                                                                             createIfNecessary: false) as? [[SCStreamFrameInfo: Any]],
              let attachments = attachmentsArray.first
        else {
            return false
        }

        // Validate the status of the frame. If it isn't `.complete`, return nil.
        guard let statusRawValue = attachments[SCStreamFrameInfo.status] as? Int,
              let status = SCFrameStatus(rawValue: statusRawValue),
              status == .complete
        else {
            return false
        }

        // Get the pixel buffer that contains the image data.
        guard let pixelBuffer = sampleBuffer.imageBuffer else {
            return false
        }

        // Get the backing IOSurface.
        guard let surfaceRef = CVPixelBufferGetIOSurface(pixelBuffer)?.takeUnretainedValue() else {
            return false
        }
        let surface = unsafeBitCast(surfaceRef, to: IOSurface.self)

        // Retrieve the content rectangle, scale, and scale factor.
        guard let contentRectDict = attachments[.contentRect],
              let contentRect = CGRect(dictionaryRepresentation: contentRectDict as! CFDictionary),
              let contentScale = attachments[.contentScale] as? CGFloat,
              let scaleFactor = attachments[.scaleFactor] as? CGFloat
        else {
            return false
        }

        return true
    }

    /// - Tag: DidOutputSampleBuffer
    func stream(_ stream: SCStream,
                didOutputSampleBuffer sampleBuffer: CMSampleBuffer,
                of outputType: SCStreamOutputType) {
        // Return early if the sample buffer is invalid.
        guard sampleBuffer.isValid else {
            return
        }

        // Determine which type of data the sample buffer contains.
        switch outputType {
        case .audio:
            self.recordComputerAudio(sampleBuffer: sampleBuffer)
        case .screen:
            guard isValidFrame(for: sampleBuffer) else {
                return
            }
            self.recordFrame(sampleBuffer: sampleBuffer)
        @unknown default:
            fatalError("Encountered unknown stream output type: \(outputType)")
        }
    }