How do I replace GMS TextRecognizer with ML Kit TextRecognizer in Camera Source?

Question

My app starts to detect text from the camera (the surfaceview) continuously and sets it to a textview like this:

cameraView = findViewById(R.id.surface_view);
textView = findViewById(R.id.text_view);


// 1
final TextRecognizer textRecognizer = new TextRecognizer.Builder(getApplicationContext()).build();
if (!textRecognizer.isOperational()) {
    Log.w("MainActivity", "Detected dependence are not found ");
} else {
    cameraSource = new CameraSource.Builder(getApplicationContext(), textRecognizer)
            .setFacing(CameraSource.CAMERA_FACING_BACK)
            .setRequestedPreviewSize(1280, 1024)
            .setRequestedFps(2.0f)
            .setAutoFocusEnabled(true)
            .build();

    // 2
    cameraView.getHolder().addCallback(new SurfaceHolder.Callback() {
        @Override
        public void surfaceCreated(SurfaceHolder holder) {
            try {
                if (ActivityCompat.checkSelfPermission(getApplicationContext(),Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED){
                    ActivityCompat.requestPermissions(MainActivity.this,new String[]{Manifest.permission.CAMERA},
                            RequestCameraPermission);
                }
                cameraSource.start(cameraView.getHolder());
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

        @Override
        public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {

        }

        @Override
        public void surfaceDestroyed(SurfaceHolder holder) {
            cameraSource.stop();
        }
    });


    // 4
    textRecognizer.setProcessor(new Detector.Processor<TextBlock>() {
        @Override
        public void release() {

        }

        @Override
        public void receiveDetections(Detector.Detections<TextBlock> detections) {
            final SparseArray<TextBlock> items = detections.getDetectedItems();
            if (items.size() != 0 ){
                textView.post(new Runnable() {
                    @Override
                    public void run() {
                        StringBuilder stringBuilder = new StringBuilder();
                        for (int i = 0 ;i < items.size();i++){
                            TextBlock item = items.valueAt(i);
                            stringBuilder.append(item.getValue());
                            stringBuilder.append("\n");
                        }
                        textView.setText(stringBuilder.toString());
                        Log.d("Text",stringBuilder.toString());
                    }
                });
            }
        }
    });
}

I want to see if I get the same text detection results if I use ML Kit's TextRecognizer by using the same logic as above. I'm trying to figure out what would be the simplist way to replace it. Because setProcessor accepts a Detector.Processor instance as an argument and TextRecognizer.process an image I'm not sure how to achieve my goal. Textrecognizer.process, from ML Kit, accepts an image as argument like below:

//creating TextRecognizer instance
TextRecognizer recognizer = TextRecognition.getClient();

//process the image
recognizer.process(image)
        .addOnSuccessListener(
                new OnSuccessListener<Text>() {
                    @Override
                    public void onSuccess(Text texts) {
                        processTextRecognitionResult(texts);
                    }
                })
        .addOnFailureListener(
                new OnFailureListener() {
                    @Override
                    public void onFailure(@NonNull Exception e) {
                        // Task failed with an exception
                        e.printStackTrace();
                    }
                });

How do I reconcile the two? So my goal is to replace only the text recognizer with the ML Kit's text recognizer and detect text in a continuous manner, like in my original code.

I have spend a lot of time researching the solution but came up empty. Any help with be greatly appreciated.

Answer 1

I have been tried on my side let me share code may be this can help you

   cameraView = findViewById(R.id.surface_view);
    textView = findViewById(R.id.text_view);
      // Check for camera permission
if (ActivityCompat.checkSelfPermission(this, Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED) {
            ActivityCompat.requestPermissions(this, new String[]{Manifest.permission.CAMERA}, RequestCameraPermission);
        } 
else {
        // Set up camera preview
        cameraProviderFuture = ProcessCameraProvider.getInstance(this);
        cameraProviderFuture.addListener(() -> {
            try {
                // CameraProvider is initialized
                ProcessCameraProvider cameraProvider = cameraProviderFuture.get();

                Preview preview = new Preview.Builder().build();
                CameraSelector cameraSelector = new CameraSelector.Builder()
                        .requireLensFacing(CameraSelector.LENS_FACING_BACK)
                        .build();

                preview.setSurfaceProvider(cameraView.getSurfaceProvider());

                ImageAnalysis imageAnalysis = new ImageAnalysis.Builder()
                        .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
                        .build();

                imageAnalysis.setAnalyzer(ContextCompat.getMainExecutor(this), imageProxy -> {
                    // Process the image using ML Kit TextRecognizer
                    processImage(imageProxy);
                });

                cameraProvider.unbindAll();
                cameraProvider.bindToLifecycle(this, cameraSelector, preview, imageAnalysis);

            } catch (ExecutionException | InterruptedException e) {
                e.printStackTrace();
            }
        }, ContextCompat.getMainExecutor(this));
    }
// Process the image using ML Kit TextRecognizer and create function
 private void processImage(ImageProxy imageProxy) {
        Image mediaImage = imageProxy.getImage();
        if (mediaImage != null) {
            InputImage image = InputImage.fromMediaImage(mediaImage, imageProxy.getImageInfo().getRotationDegrees());

            // Use ML Kit TextRecognizer to process the image
            TextRecognizer textRecognizer = TextRecognition.getClient();
            textRecognizer.process(image)
                    .addOnSuccessListener(visionText -> {
                        // Task successful
                        StringBuilder stringBuilder = new StringBuilder();
                        for (Text.TextBlock block : visionText.getTextBlocks()) {
                            stringBuilder.append(block.getText());
                            stringBuilder.append("\n");
                        }
                        textView.setText(stringBuilder.toString());
                        Log.d("Text", stringBuilder.toString());
                    })
                    .addOnFailureListener(e -> {
                        // Task failed with an exception
                        Log.e("TextRecognition", "Text recognition error: " + e.getMessage());
                    })
                    .addOnCompleteListener(task -> imageProxy.close());
        }
    }

if this is not working then try to Replace the TextRecognizer instantiation with ML Kit's TextRecognizer

FirebaseVisionTextRecognizer textRecognizer = FirebaseVision.getInstance()
        .getOnDeviceTextRecognizer();

// Replace the setProcessor logic with the ML Kit processing
textRecognizer.process(image)
        .addOnSuccessListener(new OnSuccessListener<FirebaseVisionText>() {
            @Override
            public void onSuccess(FirebaseVisionText firebaseVisionText) {
                // Task successful
                StringBuilder stringBuilder = new StringBuilder();
                for (FirebaseVisionText.TextBlock block : firebaseVisionText.getTextBlocks()) {
                    stringBuilder.append(block.getText());
                    stringBuilder.append("\n");
                }
                textView.setText(stringBuilder.toString());
                Log.d("Text", stringBuilder.toString());
            }
        })
        .addOnFailureListener(new OnFailureListener() {
            @Override
            public void onFailure(@NonNull Exception e) {
                // Task failed with an exception
                Log.e("TextRecognition", "Text recognition error: " + e.getMessage());
            }
        })
        .addOnCompleteListener(new OnCompleteListener<FirebaseVisionText>() {
            @Override
            public void onComplete(@NonNull Task<FirebaseVisionText> task) {
                // Close the imageProxy when processing is complete
                imageProxy.close();
            }
        });

for further information you can check this Recognize Text in Images with ML Kit on Android