Get multiple thumbnails from video

Question

I'm using MediaMetadataRetriever to retrieve thumbnails at a specific time in video. This is how I achieve this:

MediaMetadataRetriever metadataRetriever = new MediaMetadataRetriever();
    try {
        metadataRetriever.setDataSource(MainActivity.this, Uri.parse("android.resource://packageName/raw/"+"test"));
        String duration=metadataRetriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION);
        long time = Long.valueOf(duration)/3;
        Bitmap bitmap1 = metadataRetriever.getFrameAtTime(time,MediaMetadataRetriever.OPTION_CLOSEST_SYNC);
        imgone.setImageBitmap(bitmap1);

    }catch (Exception ex) {
        Toast.makeText(MainActivity.this, String.valueOf(ex), Toast.LENGTH_SHORT).show();
        }

This returns a bitmap/thumbnail as expected, the problem is that if I want to get multiple thumbnails at different times in the video like this:

MediaMetadataRetriever metadataRetriever = new MediaMetadataRetriever();
    try {
        metadataRetriever.setDataSource(MainActivity.this, Uri.parse("android.resource://packageName/raw/"+"test"));
        String duration=metadataRetriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION);
        long time = Long.valueOf(duration)/3;
        long time2 = time+time;
        long time3 = time+time+time;
        Bitmap bitmap1 = metadataRetriever.getFrameAtTime(time,MediaMetadataRetriever.OPTION_CLOSEST_SYNC);
        Bitmap bitmap2 = metadataRetriever.getFrameAtTime(time2,MediaMetadataRetriever.OPTION_CLOSEST_SYNC);
        Bitmap bitmap3 = metadataRetriever.getFrameAtTime(time3,MediaMetadataRetriever.OPTION_CLOSEST_SYNC);
        imgone.setImageBitmap(bitmap1);
        imgtwo.setImageBitmap(bitmap2);
        imgthree.setImageBitmap(bitmap3);

    }catch (Exception ex) {
        Toast.makeText(MainActivity.this, String.valueOf(ex), Toast.LENGTH_SHORT).show();
        }

Then it still only returns the same thumbnail, I'm not sure if it is because there is only one thumbnail available for the video or what, but I've tried different video files with the same result.

I've tried changing MediaMetadataRetriever.OPTION_CLOSEST_SYNC to all the available options but still the same result.

Im not sure if FFMPEG would be a better option for this?

Answer 1

Exactly a year later, I noticed that I never provided an answer.

In the original question I wanted to retrieve 3 thumbnails, I ended up retrieving 5. I also mentioned that I'm not sure if FFmpeg will be a suitable option, that's exactly what I used.

So, in OnCreate, I make sure that FFmpeg is supported and then I do the following:

if (FFmpeg.getInstance(getApplication()).isSupported()) {
      @SuppressLint("SimpleDateFormat")
      //ffmpeg expects the time format to be "00:00:00"
      Format formatter = new SimpleDateFormat("00:" + "mm:ss.SS");
      //Get the duration of the video
      long duration = player.getDuration();
      //Since I want 5 thumbnails, I divide the duration by 6 to get the first thumbnail position 
      long img1 = duration / 6;
      //I format the first thumbnail time since ffmpeg expects "00:00:00" format
      String firstTumbTime = formatter.format(img1);

      //Scale the size of the thumbnail output (this can be improved/changed to your liking)
      String scaledSize = displayMetrics.widthPixels / 7 + ":" + displayMetrics.heightPixels / 7;
      //Set ffmpeg command (notice that I set vframes to one, since I only want 1 thumbnail/image)
      String[] a = {"-ss", firstTumbTime, "-i", mStringFilePath, "-vframes", "1", "-s", scaledSize, imageThumbsDirectory + "/" + "thumb1.bmp"};
      //start ffmpeg asynctask for the first thumbnail
      ExecuteThumbFFMPEG(a);

}  else {
     Toast.makeText(TestNewPlayer.this, "Your device doesn't support FFMPEG...", Toast.LENGTH_SHORT).show();
}

The comments in the code above explains everything, now here is my ExecuteThumbFFMPEG method.

public void ExecuteThumbFFMPEG(String[] command) {

    ffmpegImages = FFmpeg.getInstance(this).execute(command, new ExecuteBinaryResponseHandler() {

        @Override
        public void onStart() {
            //ffmpeg started
        }

        @Override
        public void onProgress(String message) {
            //get ffmpeg progress
        }

        @Override
        public void onFailure(String message) {
            //ffmpeg failed
        }

        @Override
        public void onSuccess(String message) {
            //first thumbnail saved successfully, now to get the other 4

            //Scale the thumbnail output (Same as above)
            String scaledSize = displayMetrics.widthPixels / 7 + ":" + displayMetrics.heightPixels / 7;

            try {
                //I first set the path/name for each thumbnail, this will also be used to check if the thumbnail is available or if we should get it
                String imgPath1 = imageThumbsDirectory + "/" + "thumb1.bmp";
                String imgPath2 = imageThumbsDirectory + "/" + "thumb2.bmp";
                String imgPath3 = imageThumbsDirectory + "/" + "thumb3.bmp";
                String imgPath4 = imageThumbsDirectory + "/" + "thumb4.bmp";
                String imgPath5 = imageThumbsDirectory + "/" + "thumb5.bmp";

                //Set the format again (same as above)
                @SuppressLint("SimpleDateFormat")
                Format formatter = new SimpleDateFormat("00:" + "mm:ss.SS");

                //Get the length of the video
                long duration = Player.getDuration();

                //Divide the length of the video by 6 (same as above)
                long time = duration / 6;

                //Since I want 5 thumbnails evenly distributed throughout the video
                //I use the video length divided by 6 to accomplish that
                long img2 = time + time;
                long img3 = time + time + time;
                long img4 = time + time + time + time;
                long img5 = time + time + time + time + time;

                //Format the time (calculated above) for each thumbnail I want to retrieve
                String Img2Timeformat = formatter.format(img2);
                String Img3Timeformat = formatter.format(img3);
                String Img4Timeformat = formatter.format(img4);
                String Img5Timeformat = formatter.format(img5);

                //Get reference to the thumbnails (to see if they have been created before)
                File fileimgPath1 = new File(imgPath1);
                File fileimgPath2 = new File(imgPath2);
                File fileimgPath3 = new File(imgPath3);
                File fileimgPath4 = new File(imgPath4);
                File fileimgPath5 = new File(imgPath5);

                //If thumbnail 1 exist and thumbnail 2 doesn't then we need to get thumbnail 2
                if (fileimgPath1.exists() && !fileimgPath2.exists()) {
                    //Get/decode bitmap from the first thumbnail path to be able to set it to our ImageView that should hold the first thumbnail
                    Bitmap bmp1 = BitmapFactory.decodeFile(imgPath1);
                    //Set the first thumbnail to our first ImageView
                    imgone.setImageBitmap(bmp1);
                    //Set the ffmpeg command to retrieve the second thumbnail
                    String[] ffmpegCommandForThumb2 = {"-ss", Img2Timeformat, "-i", mStringFilePath, "-vframes", "1", "-s", scaledSize, imageThumbsDirectory + "/" + "thumb2.bmp"};
                    //Start ffmpeg again, this time we will be getting thumbnail 2
                    ExecuteThumbFFMPEG(ffmpegCommandForThumb2);
                }

                //If thumbnail 2 exist and thumbnail 3 doesn't then we need to get thumbnail 3
                if (fileimgPath2.exists() && !fileimgPath3.exists()) {
                    //Get/decode bitmap from the second thumbnail path to be able to set it to our ImageView that should hold the second thumbnail
                    Bitmap bmp2 = BitmapFactory.decodeFile(imgPath2);
                    //Set the second thumbnail to our second ImageView
                    imgTwo.setImageBitmap(bmp2);
                    //Set the ffmpeg command to retrieve the third thumbnail
                    String[] ffmpegCommandForThumb3 = {"-ss", Img3Timeformat, "-i", mStringFilePath, "-vframes", "1", "-s", scaledSize, imageThumbsDirectory + "/" + "thumb3.bmp"};
                    //Start ffmpeg again, this time we will be getting thumbnail 3
                    ExecuteThumbFFMPEG(ffmpegCommandForThumb3);
                }

                ////If thumbnail 3 exist and thumbnail 4 doesn't then we need to get thumbnail 4
                if (fileimgPath3.exists() && !fileimgPath4.exists()) {
                    //Get/decode bitmap from the third thumbnail path to be able to set it to our ImageView that should hold the third thumbnail
                    Bitmap bmp3 = BitmapFactory.decodeFile(imgPath3);
                    //Set the third thumbnail to our third ImageView
                    imgThree.setImageBitmap(bmp3);
                    //Set the ffmpeg command to retrieve the fourth thumbnail
                    String[] ffmpegCommandForThumb4 = {"-ss", Img4Timeformat, "-i", mStringFilePath, "-vframes", "1", "-s", scaledSize, imageThumbsDirectory + "/" + "thumb4.bmp"};
                    //Start ffmpeg again, this time we will be getting thumbnail 4
                    ExecuteThumbFFMPEG(ffmpegCommandForThumb4);
                }

                ////If thumbnail 4 exist and thumbnail 5 doesn't then we need to get thumbnail 5
                if (fileimgPath4.exists() && !fileimgPath5.exists()) {
                    //Get/decode bitmap from the first fourth path to be able to set it to our ImageView that should hold the fourth thumbnail
                    Bitmap bmp4 = BitmapFactory.decodeFile(imgPath4);
                    //Set the fourth thumbnail to our fourth ImageView
                    imgFour.setImageBitmap(bmp4);
                    //Set the ffmpeg command to retrieve the last thumbnail
                    String[] ffmpegCommandForThumb5 = {"-ss", Img5Timeformat, "-i", mStringFilePath, "-vframes", "1", "-s", scaledSize, imageThumbsDirectory + "/" + "thumb5.bmp"};
                    //Start ffmpeg again, this time we will be getting thumbnail 5
                    ExecuteThumbFFMPEG(ffmpegCommandForThumb5);
                }

                //If thumbnail 5 exist, then we are done and we need to set it to our ImageView
                if (fileimgPath5.exists()) {
                    Bitmap bmp5 = BitmapFactory.decodeFile(imgPath5);
                    imgFive.setImageBitmap(bmp5);
                }


            } catch (Exception ex) {
                Toast.makeText(Player.this, String.valueOf(ex), Toast.LENGTH_SHORT).show();
            }

        }

        @Override
        public void onFinish() {
            //ffmpeg is done
        }


    });

}

When the user back out of the Activity or OnDestroy gets called, all the thumbnails should be deleted, I do this by calling the following method:

DeleteThumbs.deleteAllThumbnails(getBaseContext());

Here is the DeleteThumbs class for deleting all the thumbnails/images

class DeleteThumbs {

    @SuppressWarnings("unused")
    static void deleteAllThumbnails(Context baseContext){
        //Directory where all the thumbnails are stored
        File imageThumbsDirectory = baseContext.getExternalFilesDir("ThumbTemp");
        //Path to each thumbnail
        File f1 = new File(imageThumbsDirectory + "/" + "thumb1.bmp");
        File f2 = new File(imageThumbsDirectory + "/" + "thumb2.bmp");
        File f3 = new File(imageThumbsDirectory + "/" + "thumb3.bmp");
        File f4 = new File(imageThumbsDirectory + "/" + "thumb4.bmp");
        File f5 = new File(imageThumbsDirectory + "/" + "thumb5.bmp");

        boolean d1 = f1.delete();
        boolean d2 = f2.delete();
        boolean d3 = f3.delete();
        boolean d4 = f4.delete();
        boolean d5 = f5.delete();
    }

}

Since we know the name of each thumbnail, it's easy to delete them all at once.

This provides me with 5 thumbnail images that are scaled to reduce loading time into the ImageView's. Because I divided the duration of the video by 6, I get 5 images that are evenly "distributed" throughout the video.

NOTE:

This can be improved by caching the images into memory or using a library like picasso or glide to handle the image loading for us.

Answer 2

Try this one

 public void detectBitmapFromVideo(int secondcount, int framecount, String videoPath) {
        //int fps = 800000 / framecount;
        int delta_time = secondcount * 1000000; //in microsecs
        //FFmpegMediaMetadataRetriever mmr = new FFmpegMediaMetadataRetriever();
        //mmr.setDataSource(videoPath);
        //String s_duration = mmr.extractMetadata(FFmpegMediaMetadataRetriever.METADATA_KEY_DURATION);
        MediaMetadataRetriever mediaMetadataRetriever = new MediaMetadataRetriever();
        mediaMetadataRetriever.setDataSource(videoPath);
        int duration = getVideoDuration(mediaMetadataRetriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION));
        //int duration = getVideoDuration(s_duration);
        ArrayList<Frame> frames = new ArrayList<Frame>();
        //Log.e("Duration ", "Duration  = " + duration + " Delta time = " + delta_time);
        for (int i = 0; i <= duration; i += delta_time) {
            Bitmap bmFrame = mediaMetadataRetriever.getFrameAtTime(i);
            //unit in microsecond
            if (bmFrame == null) {
                //Log.e(TAG, "frame image " + bmFrame.toString());
                continue;
            }
            //saveBitmapImage(bmFrame,i+"");
            frames.add(new Frame.Builder().setBitmap(bmFrame).build());



            /*Bitmap frame_orig = mmr.getFrameAtTime(i, FFmpegMediaMetadataRetriever.OPTION_CLOSEST);
            if (frame_orig == null) {
                continue;
            }

            frames.add(new Frame.Builder().setBitmap(rotateBitmap(frame_orig, 90f)).build());
            //Log.e("Faces Detected", "Face detection on going  duration = " + duration + " Deleta time = " + i);
        }

    }