ffdec_mpeg4: decoding error with gst_pad_push() = not-linked while using gstreamer-java with pipeline decodebin2 ! ffmpegcolorspace ! video/x-raw-rgb

Question

I am trying to get the following pipeline to work in gstreamer-java (using gstreamer-0.10):

"gst-launch-0.10 filesrc location=big_buck_bunny_480p_surround-fix.avi ! decodebin2 ! ffmpegcolorspace ! autovideosink"

This pipeline works on the command line. Machine vbox vm running ubuntu 14.04 LTS

Can anyone help me identify what my problem might be with the pipeline in the java code?

The java code is as follows:

public static void main(String[] args) {

args = Gst.init("AppSrcTest", args);
/* setup pipeline */
pipeline = new Pipeline("pipeline");
final AppSrc appsrc = (AppSrc) ElementFactory.make("appsrc", "appsrc");

final Element decodebin = ElementFactory.make("decodebin2", null);
final Element ffmpegcolorspace = ElementFactory.make("ffmpegcolorspace", "formatConverter");

SwingUtilities.invokeLater(new Runnable() {

    public void run() {             
        JFrame frame = new JFrame("FakeSrcTest");
        VideoComponent panel = new VideoComponent();
        panel.setPreferredSize(new Dimension(width, height));
        frame.add(panel, BorderLayout.CENTER);
        Element videosink = panel.getElement();

        pipeline.addMany(appsrc, decodebin,ffmpegcolorspace, videosink);
        Element.linkMany(appsrc, decodebin,ffmpegcolorspace, videosink);

        appsrc.setTimestamp(true);

        appsrc.set("emit-signals", true);
        appsrc.connect(new AppSrc.NEED_DATA() {                    
            public void needData(AppSrc elem, int size) {
                try {
                    //other code    
                    ByteArrayOutputStream outputStream = new ByteArrayOutputStream();

                    }
                    System.out.println("Data collected..");
                    Buffer buffer = new Buffer(outputStream.toByteArray().length);
                    buffer.getByteBuffer().put(outputStream.toByteArray());
                    appsrc.pushBuffer(buffer);
                } catch (Exception e) {
                    e.printStackTrace();
                }
            }
        });

The debug info is as follows when i start getting the error:

0:00:06.194558872 [332m14803[00m 0x7f24dc038680 [32;01mINFO   [00m [00;01;37;41m    GST_ELEMENT_PADS gstelement.c:728:gst_element_add_pad:<decodebin20>[00m adding pad 'src0'
0:00:06.194682253 [332m14803[00m 0x7f24dc038680 [32;01mINFO   [00m [00;01;37;41m    GST_ELEMENT_PADS gstelement.c:728:gst_element_add_pad:<decodebin20>[00m adding pad 'src1'
0:00:06.194765369 [332m14803[00m 0x7f24dc038680 [32;01mINFO   [00m [00;01;31m          GST_STATES gstbin.c:2942:bin_handle_async_done:<decodebin20>[00m committing state from READY to PAUSED, old pending PAUSED
0:00:06.194824176 [332m14803[00m 0x7f24dc038680 [32;01mINFO   [00m [00;01;31m          GST_STATES gstbin.c:2962:bin_handle_async_done:<decodebin20>[00m completed state change, pending VOID
0:00:06.194877892 [332m14803[00m 0x7f24dc038680 [32;01mINFO   [00m [00;01;31m          GST_STATES gstelement.c:2365:_priv_gst_element_state_changed:<decodebin20>[00m notifying about state-changed READY to PAUSED (VOID_PENDING pending)
Got TAG event
Tag audio-codec = Dolby Digital (AC-3)
Tag bitrate = 448000
0:00:06.212171413 [332m14803[00m 0x7f24dc038680 [32;01mINFO   [00m [00m              a52dec gsta52dec.c:439:gst_a52dec_reneg:<a52dec0>[00m reneg channels:6 rate:48000
0:00:06.258920113 [332m14803[00m 0x7f24dc043400 [33;01mWARN   [00m [00m              ffmpeg gstffmpegdec.c:2299:gst_ffmpegdec_frame:<ffdec_mpeg40>[00m ffdec_mpeg4: decoding error (len: -1, have_data: 0)
0:00:06.259312652 [332m14803[00m 0x7f24dc043450 [32;01mINFO   [00m [00m             basesrc gstbasesrc.c:2562:gst_base_src_loop:<appsrc>[00m pausing after gst_pad_push() = not-linked
0:00:06.259366067 [332m14803[00m 0x7f24dc043450 [33;01mWARN   [00m [00m             basesrc gstbasesrc.c:2625:gst_base_src_loop:<appsrc>[00m error: Internal data flow error.
0:00:06.259383885 [332m14803[00m 0x7f24dc043450 [33;01mWARN   [00m [00m             basesrc gstbasesrc.c:2625:gst_base_src_loop:<appsrc>[00m error: streaming task paused, reason not-linked (-1)
0:00:06.259418691 [332m14803[00m 0x7f24dc043450 [32;01mINFO   [00m [00;01;31;47m    GST_ERROR_SYSTEM gstelement.c:1964:gst_element_message_full:<appsrc>[00m posting message: Internal data flow error.
0:00:06.268446147 [332m14803[00m 0x7f24dc043400 [33;01mWARN   [00m [00m              ffmpeg gstffmpegdec.c:2299:gst_ffmpegdec_frame:<ffdec_mpeg40>[00m ffdec_mpeg4: decoding error (len: -1, have_data: 0)
0:00:06.273770933 [332m14803[00m 0x7f24dc043400 [33;01mWARN   [00m [00m              ffmpeg gstffmpegdec.c:2299:gst_ffmpegdec_frame:<ffdec_mpeg40>[00m ffdec_mpeg4: decoding error (len: -1, have_data: 0)
0:00:06.274603418 [332m14803[00m 0x7f24dc043400 [33;01mWARN   [00m [00m              ffmpeg gstffmpegdec.c:2299:gst_ffmpegdec_frame:<ffdec_mpeg40>[00m ffdec_mpeg4: decoding error (len: -1, have_data: 0)
0:00:06.275939105 [332m14803[00m 0x7f24dc043400 [33;01mWARN   [00m [00m              ffmpeg gstffmpegdec.c:2299:gst_ffmpegdec_frame:<ffdec_mpeg40>[00m ffdec_mpeg4: decoding error (len: -1, have_data: 0)
0:00:06.277303268 [332m14803[00m 0x7f24dc043400 [33;01mWARN   [00m [00m              ffmpeg gstffmpegdec.c:2299:gst_ffmpegdec_frame:<ffdec_mpeg40>[00m ffdec_mpeg4: decoding error (len: -1, have_data: 0)
Error: code=1 message=Internal data flow error.
0:00:06.291740181 [332m14803[00m 0x7f24dc043450 [32;01mINFO   [00m [00;01;31;47m    GST_ERROR_SYSTEM gstelement.c:1987:gst_element_message_full:<appsrc>[00m posted error message: Internal data flow error.
0:00:06.292844435 [332m14803[00m 0x7f24dc043400 [33;01mWARN   [00m [00m              ffmpeg gstffmpegdec.c:2299:gst_ffmpegdec_frame:<ffdec_mpeg40>[00m ffdec_mpeg4: decoding error (len: -1, have_data: 0)
Got TAG event
Tag minimum-bitrate = -1
Tag bitrate = 0
Tag maximum-bitrate = 0
0:00:06.297233217 [332m14803[00m 0x7f24dc043400 [33;01mWARN   [00m [00m              ffmpeg gstffmpegdec.c:2299:gst_ffmpegdec_frame:<ffdec_mpeg40>[00m ffdec_mpeg4: decoding error (len: -1, have_data: 0)
0:00:06.297382004 [332m14803[00m 0x7f24dc043400 [32;01mINFO   [00m [00;01;31;41m            GST_PADS gstpad.c:3554:gst_pad_event_default_dispatch:<mpeg4vparse0:sink>[00m Sending event 0x7f24dc148c60 (eos) to all internally linked pads

Answer 1

Answering my own question

The best advice I have read online regarding gstreamer is: test your pipeline on the command line before implementing in code. Consequently, if it works on the command line it should work in code.

The following question and answer my question in c code. i.e. it proposes a solution to the error "pausing after gst_pad_push() = not-linked" gStreamer-Sharp Dynamic pads not linking

However, I did not know how to do this in Java code. So I got a little more familiar with dynamic pipelines in gstreamer using the following tutorial: Dynamic Pipelines: http://docs.gstreamer.com/display/GstSDK/Basic+tutorial+3%3A+Dynamic+pipelines

Two points to note from the above tutorial in solving this problem:

1. GSignals are a crucial point in GStreamer. They allow you to be notified (by means of a callback) when something interesting has happened. Signals are identified by a name, and each GObject has its own signals.

2. Demuxers start with no source pads to which other elements can link, and thus the pipeline must necessarily terminate at them. The solution is to build the pipeline from the source down to the demuxer, and set it to run (play). When the demuxer has received enough information to know about the number and kind of streams in the container, it will start creating source pads. This is the right time for us to finish building the pipeline and attach it to the newly added demuxer pads.

Consequently my java code looks as follows:

            public void run() {             
            JFrame frame = new JFrame("FakeSrcTest");
            VideoComponent panel = new VideoComponent();
            panel.setPreferredSize(new Dimension(width, height));
            frame.add(panel, BorderLayout.CENTER);
            final Element videosink = panel.getElement();

            pipeline.addMany(appsrc,  decodebin2,  ffmpegcolorspace, videosink);
            Element.linkMany(appsrc,  decodebin2);
            //we need to finish linking the pipeline later...

            appsrc.set("emit-signals", true);

            appsrc.connect(new AppSrc.NEED_DATA() {  ...

Additionally, you need add a listener for when pads are created in decodebin2 at which point you need to finish linking the pipeline.

            /* listen for newly created pads */
            decodebin2.connect(new Element.PAD_ADDED() {
                public void padAdded(Element element, Pad pad) {
                   System.out.println("New Pad " + pad.getName() + " was created");
                   Element.linkMany(decodebin2, ffmpegcolorspace, videosink);
                }
            });

That's it.

I hope it helps someone else.