Using two CameraX ImageAnalysis analyzers

77 Views Asked by At

I want to use CameraX to

  1. first analyze the image with MLKit Image Labeling and then
  2. if, and only if, the image contains a dog, send it over an Event channel for further analysis with Flutter. The following code snippets work:
// ***************** Analyze with MLKit imageLabeler
val options = ImageLabelerOptions.Builder()
    .setConfidenceThreshold(0.7f)
    .build()
val imageLabeler = ImageLabeling.getClient(options)
val detectors = listOf(imageLabeler)
val analyzer = MlKitAnalyzer(
    detectors,
    ImageAnalysis.COORDINATE_SYSTEM_ORIGINAL,
    executor
)
{
    val labels = it.getValue(imageLabeler)
    if (labels.isNullOrEmpty()) {
        return@MlKitAnalyzer
    }
    var result = ""
    for (label in labels) {
        val text = label.text
        val confidence = (label.confidence * 100).toInt()
        val index = label.index
        result = result + text + " " + confidence + "%\n"
    }
    Log.d(TAG, "*****************Image Labelling result $result")
}
val imageAnalysis = ImageAnalysis.Builder()
    // Default is OUTPUT_IMAGE_FORMAT_YUV_420_888
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
    .build()
imageAnalysis.setAnalyzer(executor, analyzer)

and

// ****************** Analyze with my own code
val analyzer = ImageAnalysis.Analyzer { image ->
    if (image.format == PixelFormat.RGBA_8888) {
        val planes = image.image!!.planes
        val plane0 = planes!![0]
        val byteBuffer: ByteBuffer = plane0.buffer
        val byteArray = ByteArray(byteBuffer.capacity())
        byteBuffer.get(byteArray)
        sink?.success(byteArray)
        // Thread.sleep(500)
    }
    image.close()
}
val imageAnalysis = ImageAnalysis.Builder()
    .setOutputImageFormat(ImageAnalysis.OUTPUT_IMAGE_FORMAT_RGBA_8888)
    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
    .build()
imageAnalysis.setAnalyzer(executor, analyzer)

But I can't figure out how to do both, and the second one only if the first one has a label containg "Dog". Any ideas, anyone?

0

There are 0 best solutions below