First impressions with new RidgeRun SDK and Leopardboard 365

Hi Alex,

Thanks for your positive feedback on our new LeopardBoard 365 SDK.  We have been working hard for many months to make the experience better than the beta version.

I haven't posted an announcement as we are working to resolve common problems beginners encounter.  For example, you encountered a problem with the installer's default TFTP image transfer setting.  I just checked in a change so the default is now serial.  However, we don't have a mechanism in place for you to register to receive SVN check-in emails, so you have no way of knowing what improvements are now available.

I am glad you are able to see video images with the 5M camera using DVI output.  I added your pipeline to

www.ridgerun.com/.../DM365_Leopard_Pipelines

If you refer to

processors.wiki.ti.com/.../Getting_Started_Guide_for_Leopard_Board_DM365

you see the Aptina MT9P031 sensor on the LI-5M03 is not an SOC sensor.  The Aptina SOC sensors support internal color saturation, white balance, noise reduction, dark current management.  Since the Aptina MT9P031 sensor is not an SOC sensor, Auto Exposure / Auto White Balance (AE/AWB) software is required.  The evaluation SDK does not include AE/AWB software.

You might find you have better luck with an SOC sensor, like the one on the LI-LBCM3M1 board (but I haven't tested that one myself).

Todd

Hi Todd,

Many thanks for the super-fast reply and lots of appreciation to all of the RidgeRun team for supporting the community this way :-)

Am I right in thinking that the AE/AWB software is included in the professional SDK? If I know that the issue will definitely be fixed by upgrading to the professional version then I'm happy to ignore it for now. However you haven't said anything about the frame rate or the glitch which is particularly serious. These problems aren't related to the AE/AWB are they?

Alex

Alex,

We are working on a AE/AWB application, but haven't achieved what we feel is acceptable image quality.    Leopard Imaging has an AE/AWB library you can try out (www.leopardimaging.com/Download.html).

GStreamer was mainly developed on desktop machines, where processing power, memory, and floating point are readily available.  We find that GStreamer works great with TI's Davinci and OMAP chips, but requires more hand tuning.  For instance, we added an element that allows you to control thread priority.  We needed priority control to resolve audio dropout issues.  We have also been working hard to be able to create zero-copy pipelines.  The enhancements are in the DDOMPE branch of the GStreamer plug-in for TI chips (http://gstreamer.ti.com).

We are shooting for network streaming 720p live video at 30 fps.  As customers fund this effort, RidgeRun will pass the improvements along to the LeopardBoard community.

Todd

Hi Alex,

Both TI and Appro have proprietary AE/AWB. For the open source community, I actually have a good news for you. We will release a "semi-Auto" exposure and white balance very soon at arago example (arago-project.org/.../examples-davinci.git) . It improves the image quality significantly by one-time adjustment. What you need to do is that, simply point the camera to a white area ( a wall or a piece of white paper), press a key to ask the software to tune the image parameters, and then it will generate a set of parameters for this light condition.

I have submitted the example source code to the owner of arago git & example. Hopefully, they will post it very soon.

-Leon

Hi Leon,

That sounds great. I'll look forward to trying it out.

Alex

Hi Todd,

That makes sense and thank you for the pointer to the Leopard imaging code. I can't help but think there's a bit more too it than just raw performance though. The video glitch I'm seeing doesn't look like a frame got skipped, but more like the board displayed a frame out of sequence. As I wave my arm across the screen I can see my arm starts on the left, reaches the middle and then briefly appears to fly back to the left before resuming it's path across to the right. It's like the frame display was 1,2,3,4,5,6,7,1,8,9,10. Of course everything happens so fast it's hard to be sure, but that's what it looks like to me.

I also don't understand why the framerate is locked to 23fps. The RidgeRun release says that the driver only supports fixed framerates which is fair enough, but why 23fps? Are you saying it can't handle 30fps even without any H264 encode or streaming?

Alex

I fixed the upside down video by modifying: kernel/linux-2.6.32/drivers/media/video/mt9p031.c as follows:

static const struct v4l2_queryctrl mt9p031_controls[] = {

{

.id = V4L2_CID_VFLIP,

.type = V4L2_CTRL_TYPE_BOOLEAN,

.name = "Flip Vertically",

.minimum = 0,

.maximum = 1,

.step = 1,

.default_value = 0,

}, {

Directly modifying this file isn't very maintainable since it's created by applying a RidgeRun patch. Todd, if you're OK with this change perhaps you could check it in?

Hope that helps someone else,

Alex

Alex,

Thanks for the suggestion.  I updated the patch and checked it back into SVN.

SVN and patches don't play nicely together.  If you have a development directory already check out from SVN, you need to

cd $DEVDIR/kernel/linux-2.6.32/patches

quilt pop -a

svn update

quilt push -a

cd $DEVDIR

make kernel

make install

I need to add a Makefile target to take case of this detail (on my todo list before we announce the SDK availability).

Todd

Todd,

That's great. Thanks for the patching instructions too. On the frame rate glitch I discovered that it disappears if I turn off accelFrameCopy so my pipeline becomes:

gst-launch -e v4l2src always-copy=false ! video/x-raw-yuv,format=\(fourcc\)UYVY, width=1280, height=720, framerate=\(fraction\)23/1 ! dmaiaccel ! dmaiperf ! TIDmaiVideoSink videoOutput=DVI sync=false accelFrameCopy=false videoStd=720P_60

That's got to be a bug hasn't it? The framerate is the same with or without the accelFrameCopy, but with it turned on I get these out of sequence frames.

By reading through the sourcecode for the camera driver I discovered that it will also accept 45fps as a valid option. Reading the datasheet for the sensor, that's the maximum it can handle with binning turned on. However when I try this option in the pipeline it works but I still don't get any more than 23fps. It's looking to me like the framerate is being throttled after the camera driver, either in TIDmaiVideoSink or perhaps in the format conversion between the camera's native RGB and UYVY. I'll keep digging, but any light you could shed would be a big help.

Alex

Alex,

My hint is for you to check the framerate using fakesink, so you can take TIDmaiVideoSink out of the picture.

I will pass along the accelFrameCopy issue to Diego, as I believe he wrote that code.

I am not the video capture driver expert on our team, so I don't know why we see one spec for the sensor frame rate, yet the driver doesn't capture at that rate.

Todd

Hi everybody,

I have been trying to get rid of the Ethernet cable when using the RidgeRun SDK (because I want to make it more portable) but then I'm getting this error:

Root-NFS: Unable to get nfsd port number from server, using default            

Looking up port of RPC 100005/1 on 192.168.1.101                                

rpcbind: server 192.168.1.101 not responding, timed out                        

Root-NFS: Unable to get mountd port number from server, using default          

Root-NFS: Server returned error -5 while mounting /home/ninja/trunk/fs/fs      

VFS: Unable to mount root fs via NFS, trying floppy.                            

VFS: Cannot open root device "nfs" or unknown-block(2,0)

Any idea on how to do it?

Thanks!

Hi gentlemich,

The SDK defaults to mounting the filesystem over NFS. If you want to get rid of the ethernet cable you will need to write the filesystem to the leopardboard's flash memory. As things are, the Leopardboard is totally dependent on an ethernet connection to the host pc to mount its file system.

I haven't done this before so I'm afraid I can't give you specifics on how to write the flash memory. There is probably some documentation on the process in the RidgeRun wiki.

Alex

It is easy

cd $DEVDIR

make config

then go into the file system menu and change from NFS to NAND.  Save and exit

make

make install

Todd

Hi gentlemich

Another way to get rid of the network cable is saving the filesystem in an SD card. For more information please visit

https://www.ridgerun.com/developer/wiki/index.php/How_to_boot_a_board_from_a_SD_card