Replacing UIImageJPEGRepresentation with UIGraphicsImageRenderer's jpegData

Question

I am working to add support for wide color photos in iOS 10. When the user takes a photo from the camera, I need to use the new API that supports the new color space to save the photo data - UIGraphicsImageRenderer's jpegData instead of UIImageJPEGRepresentation.

I'm running into some troubles with image orientations. Taking a photo on my iPad in portrait, the image isn't being drawn correctly. See the comments below:

Old API:

let image = info[UIImagePickerControllerOriginalImage] as! UIImage
let imageData = UIImageJPEGRepresentation(image, 1)

New API:

let image = info[UIImagePickerControllerOriginalImage] as! UIImage
let cgImage = image.cgImage!
let ciImage = CIImage(cgImage: cgImage)

let format = UIGraphicsImageRendererFormat()
format.scale = 1
format.prefersExtendedRange = true

let renderer = UIGraphicsImageRenderer(bounds: ciImage.extent, format: format)
let imageData = renderer.jpegData(withCompressionQuality: 1, actions: { context in
    context.cgContext.draw(cgImage, in: ciImage.extent) //draws flipped horizontally
    //image.draw(at: .zero) //draws rotated 90 degrees leaving black at bottom
    //image.draw(in: ciImage.extent) //draws rotated 90 degrees stretching and compressing the image to fill the rect 
 })

What's the correct way to replace UIImageJPEGRepresentation with UIGraphicsImageRenderer's jpegData?

Answer 1

UIImage can have different orientation depending on camera rotation. You can dynamically resolve the transformation needed to be applied to the image depending on that orientation, like this:

let renderer = UIGraphicsImageRenderer(size: image.size, format: format)
let imageData = renderer.jpegData(withCompressionQuality: 1, actions: { context in
    var workSize = image.size;
    workSize.width = floor(workSize.width / image.scale)
    workSize.height = floor(workSize.height / image.scale)
    // No-op if the orientation is already correct
    // if image.imageOrientation == .up { draw image }

    // We need to calculate the proper transformation to make the image upright.
    // We do it in 2 steps: Rotate if Left/Right/Down, and then flip if Mirrored.

    var transform = CGAffineTransform.identity

    switch image.imageOrientation
    {
        case .down, .downMirrored:
            transform = transform.translatedBy(x: workSize.width, y: workSize.height)
            transform = transform.rotated(by: CGFloat(Double.pi))
            break

        case .left, .leftMirrored:
            transform = transform.translatedBy(x: workSize.width, y: 0.0)
            transform = transform.rotated(by: CGFloat(Double.pi / 2.0))
            break

        case .right, .rightMirrored:
            transform = transform.translatedBy(x: 0.0, y: workSize.height)
            transform = transform.rotated(by: CGFloat(-Double.pi / 2.0))
            break

        case .up, .upMirrored:
            break
    }

    switch image.imageOrientation
    {
        case .upMirrored, .downMirrored:
            transform = transform.translatedBy(x: workSize.width, y: 0.0)
            transform = transform.scaledBy(x: -1.0, y: 1.0)
            break

        case .leftMirrored, .rightMirrored:
            transform = transform.translatedBy(x: workSize.height, y: 0.0);
            transform = transform.scaledBy(x: -1.0, y: 1.0);
            break

        case .up, .down, .left, .right:
            break
    }

    // Now we draw the underlying CGImage into a new context, applying the transform
    // calculated above.

    let ctx = context.cgContext

    ctx.concatenate(transform)

    switch image.imageOrientation {
        case .left, .leftMirrored, .right, .rightMirrored:
            ctx.draw(image.cgImage!, in: CGRect(x: 0.0, y:0.0, width: workSize.height, height: workSize.width))
            break;

        default:
            ctx.draw(image.cgImage!, in: CGRect(origin: .zero, size: workSize))
            break;
    } 
 })

Answer based on UIImage+fixOrientation

Answer 2

The image.draw() method should correct the orientation automatically for you:

This method draws the entire image in the current graphics context, respecting the image’s orientation setting. In the default coordinate system, images are situated down and to the right of the origin of the specified rectangle. This method respects any transforms applied to the current graphics context, however.

I'm not sure why you need to use ci.extent. The extent seems to be (I make no claims to be an expert in Core Image APIs) the dimensions of the "raw" image data, which is stored without any image orientation correction. If you raw data needs rotation (non-Up orientations like Left, Right), the extent will still be the original rect that the data is stored.

I use the following code for an image with imageOrientation.rawValue = 3 and my data turns out perfectly fine at the right size with "corrected" imageOrientation.rawValue = 0 when reloaded.

let imageSize = image.Size
let renderer = UIGraphicsImageRenderer(size: renderedSize, format: UIGraphicsImageRendererFormat())
return renderer.jpegData(withCompressionQuality: 0.95) { (rendererContext) in
    let rect = CGRect(x: 0, y: 0, width: imageSize.width, height: imageSize.height)
    image.draw(in: rect)
}

My input image "renders" its data correctly with my code, taking into account the image orientation. On the right, I use draw(in: extent). The image isn't rotated though. Just stretched.