Forums

Video with audio

This question is not answered

With the RR SDK installed on the Leopardboard, is it possible to capture a stream containing both video and audio?

All Replies
  • Hi,

    No, it is not possible right now due to some issues with the audio codecs. Audio and video can be capture separately.

    Regards,

    Cristina

    Cristina Murillo

    Embedded Software Engineer, RidgeRun

  • Thanks Christina

    Any idea when the issue will be resolved?

  • Any news about the issue?

    thank you

    Andrea

  • we are concerning about it too.

    When will it be solved ?

    thanks!

  • And now? Is it possible?

  • It's possible on the DM365 LeopardBoard

  • Could you please explain how? It would really benefit the community and would set things once and for all.

    I have just executed the following pipeline successfully (the mp4 file is written with audio+sound) however it seems accelerated (not in real-time speed) when executed on a desktop computer with Ubuntu.

    gst-launch -e alsasrc buffer-time=500000 slave-method=1 ! 'audio/x-raw-int, endianness=(int)1234, signed=(boolean)true, width=(int)16, depth=(int)16, rate=(int)44100,channels=(int)2' ! dmaienc_aac bitrate=224000 maxbitrate=224000 outputBufferSize=131072 ! priority rtpriority=18 scheduler=1 ! queue ! qtmux name=m v4l2src always-copy=false ! video/x-raw-yuv,format=\(fourcc\)NV12, width=1280, height=720, framerate=\(fraction\)23/1 ! dmaiaccel ! dmaienc_mpeg4 ! dmaiperf ! priority nice=-19 ! m. m. ! filesink location=file.mp4

    PLEASE help us solve this problem.

  • - Take out the slave-method from the alsasrc, will cause you the sync problems.

    - You probably should avoid the priority elements unless you really need them. Running gst-launch with nice -n -5 should be good enough for you.

    That would likely get you a decent file. Otherwise you can also try using h264 for such high resolution (should be faster in platinum mode).

    Regards,

    Diego

  • Hi Diego,

    I still have the same problem: the video is not playing at normal speed on my desktop computer.

    I have tried this:

    gst-launch -e alsasrc buffer-time=500000 ! 'audio/x-raw-int, endianness=(int)1234, signed=(boolean)true, width=(int)16, depth=(int)16, rate=(int)44100,channels=(int)2' ! dmaienc_aac bitrate=224000 maxbitrate=224000 outputBufferSize=131072 ! queue ! qtmux name=m v4l2src always-copy=false ! video/x-raw-yuv,format=\(fourcc\)NV12, width=1280, height=720, framerate=\(fraction\)23/1 ! dmaiaccel ! dmaienc_h264 ! priority nice=-5 ! dmaiperf ! m. m. ! filesink location=file.h264

    But it seems that the FPS is only about 5 (Timestamp: 0:20:41.207906559; bps: 274205; fps: 5; ). Is it normal?

    Thanks!

  • nice -n -5 gst-launch -e alsasrc buffer-time=500000 ! 'audio/x-raw-int, endianness=(int)1234, signed=(boolean)true, width=(int)16, depth=(int)16, rate=(int)44100,channels=(int)2' ! dmaienc_aac bitrate=64000 outputBufferSize=131072 ! queue ! qtmux name=m v4l2src always-copy=false ! video/x-raw-yuv,format=\(fourcc\)NV12, width=1280, height=720, framerate=\(fraction\)23/1 ! dmaiaccel ! dmaienc_h264 encodingpreset=2 ratecontrol=2 targetbitrate=1500000 ! dmaiperf ! queue ! m. m. ! filesink location=file.h264

    Try first recording on /tmp (assuming that's a ramfs), your problem could be produced if you are recording into network or other device that is too slow.

  • Thank you Diego.

    The framerate has much more improved (30fps) but there is still the problem with the speed when playing the file on an other computer. I have recorded a mp4 file for 20s on the DM365 but when analyzing it using MediaInfo on Ubuntu, I am getting a total time of 5s !

    As you could imagine, the video (and the audio) are playing at a speed 4 times higher than it should be on the desktop. It is really strange.

    Please tell me how I can improve the total time / speed of the video.

    Thanks!

  • There is a known issue where if you switch the audio sample rates the clocking will break until next reboot. Since the timing of the gstreamer pipe is determinated by the the audio clocking, is likely that you are seeing this problem.

    Remove all the audio caps, and be sure to record after booting the Leopard to see if that makes a difference. Another test is record the audio only pipe and see if the audio sounds fine.

    Diego

  • Thank you Diego, but now I have only a DM355 :-(

    Have you idea if is an hardware or software problem? Do you know if the development and support for this board is finished?

    I like resolve the problem on DM355, but I don't know where to start !!

    Andrea

  • Andrea,

    This is a software problem: the codec available by Ittiam stops if used in parallel with the driver for audio capture. We tried to get support for the audio codec but never received any solution for it. You can encode if you use a codec that isn't from Ittiam for DM355 (like flac).

    Development for Leopardboard is at this point focused on DM365, so don't expect any new SDK or improvements for DM355 unless there is someone founding them.

    Regards,

    Diego

     

  • Hi Diego,

    Unfortunately, removing the caps after rebooting does not change anything, the audio (and video) are accelerated 4x:

    nice -n -5 gst-launch -e alsasrc buffer-time=500000 ! dmaienc_aac bitrate=64000 outputBufferSize=131072 ! queue ! qtmux name=m v4l2src always-copy=false ! video/x-raw-yuv,format=\(fourcc\)NV12, width=1280, height=720, framerate=\(fraction\)23/1 ! dmaiaccel ! dmaienc_h264 encodingpreset=2 ratecontrol=2 targetbitrate=1500000 ! dmaiperf ! queue ! m. m. ! filesink location=file.h264

    Same result for:

    gst-launch -e alsasrc ! 'audio/x-raw-int,rate=(int)44100,channels=(int)2' ! dmaienc_aac bitrate=128000 ! qtmux ! filesink location=audio_aac.mp4

    And for:

    gst-launch -e alsasrc ! dmaienc_aac bitrate=128000 ! qtmux ! filesink location=audio_aac.mp4

    Thanks for your support!