Gstreamer and hardware video encoding on tinkerboard

Question

I am starting upp with a tinkerboard r2 (and also working on a 2S) trying to work out a video recording and streaming program. Now: Recording a video from the camera and encoding it to h264 takes a lot of cpu. so:

• Does gstreamer already use the hardware encoding capabilities (automatically) and I am way to optimistic in terms of framerate and resolution

• can the Mali -760 GPU on the tinkerboard be used to take some weight off the cpu? Reading the docs on the T760 (T764) it seems that video encoding should be possible.

• If so, how do enter this into the gstreamer pipeline?

• Finally, if anyone has some really good "for dummies" for gstreamer elements on ARM Linux please share a link,

Setup: Tinkerboard 2r CPU Rockchip Quad-Core RK3288-CG.W processor GPU ARM® Mali™-T764 GPU https://developer.arm.com/Processors/Mali-T760 Memory: 2GB Dual Channel DDR3 OS: Debian 10 - tinker board adapted

I have tried gst-launch-1.0 to make various pipelines. Without ! x264enc ! the files becomes huge because it uses raw data. With the encoding and compression it becomes laggy (or drops frames) because of high cpu usage.

Answer 1

for hardware encoding you can use mpph264enc plugin:

gst-launch-1.0 videotestsrc num-buffers=512 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! mpph264enc ! queue ! h264parse ! mpegtsmux ! filesink location=/home/linaro/Desktop/h264.ts

or

This is the stream i tried with audio and video:

gst-launch-1.0 -e v4l2src num-buffers=512 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! mpph264enc ! queue ! h264parse ! mp4mux0. autoaudiosrc ! audio/x-raw,width=16,depth=16,rate=44100,channel=1 ! queue ! audioconvert ! audioresample ! voaacenc ! aacparse ! mp4mux ! filesink location=/home/linaro/testh264-2.mp4

To give other newbies like myself a little more to work on here is the layout, as I understand it.

gst-launch-1.0 -e v4l2src num-buffers=512 ! video/x-raw,format=NV12,width=1920,height=1080,framerate=30/1 ! queue ! // this sets the camera source as v4l2 and will stop after 512 buffers have been used (approx 4 minutes ) raw vidoe stream is NV12 1920x108 with 30 fps, pretty obvious, add queue to let the computer keep working the camera while it encodes //

mpph264enc ! queue ! // right !encode h.264 via hardware. Nice!

h264parse ! // parse for muxing

mp4mux0. autoaudiosrc ! // link the video and audio mux for intended parcel format, in this case mp4

audio/x-raw,width=16,depth=16,rate=44100,channel=1 ! queue ! audioconvert ! audioresample ! voaacenc ! aacparse // add audio source and encoding

! mp4mux ! f// mux it together into one file

filesink location=/home/linaro/testh264-2.mp4 // sink file where your video will end up!

Presto!