RealBasic: Computer Vision -> OpenSURF feature matching


This is the translation of the OpenSURF Computer Vision library from Chris Evans to RealBasic, plus I’ve added some code to compare two pictures. The conversion to RealBasic is not as fast as the C# or C++ version. In fact it is rather slow, but you can use it for educational purposes. If by any chance you’re one of these guys who want to spend some time speeding up this code, please let me know because it can benefit a lot of programmers out there.

SURF (Speeded Up Robust Feature) is a robust local feature detector, first presented by Herbert Bay et al. in 2006, that can be used in computer vision tasks like object recognition or 3D reconstruction.

Christopher Evans on the OpenSURF library:

The task of finding point correspondences between two images of the same scene or object is an integral part of many machine vision or computer vision systems. The algorithm aims to find salient regions in images which can be found under a variety of image transformations. This allows it to form the basis of many vision based tasks; object recognition, video surveillance, medical imaging, augmented reality and image retrieval to name a few.

Simply said:
The concept of feature detection refers to methods to find edges, corners, blobs etc within images that can be used to detect, identify and compare images and is robust against different image transformations like roations, skews, etc…

The code is to complicated to put in this article. The engine is here so you can use it without worrying about the complicated maths. The class has an easy-to-use interface and you can start new Computer Vision experiments with your favorite computer language! 🙂

In fact, this is the only code you need to use it:

  ' declare an opensurf and run it on the first image
  OSurf1 = new ABOpenSURF
  OSurf1.ExecuteSURF(Image1,1,true)
  
  ' declare a second opensurf and run it on the second image
  OSurf2 = new ABOpenSURF
  OSurf2.ExecuteSURF(Image2, 1, true)
    
  ' get the matching points between the two images
  OSurf2.GetMatchingPoints(OSurf1)
  
  ' and draw them
  OSurf1.PaintSURF(mBuffer1, DebugMode, true)
  OSurf2.PaintSURF(mBuffer2, DebugMode, true)
  Canvas1.Refresh(true)
  Canvas2.Refresh(true)  

Guess it can’t get any simpler to do such a complicated task 🙂

Note 1: An application of the SURF algorithm is patented in the US!
Note 2: In the next article I’ll give you the Realbasic translation of the FAST (Features from Accelerated Segment Test) engine!

The RealBasic implementation of OpenSURF: http://www.gorgeousapps.com/ABopenSURF.zip

Also check out the official OpenSURF library in C++/C#: http://www.chrisevansdev.com/computer-vision-opensurf.html

Advertisements

About Alwaysbusy

My name is Alain Bailleul and I'm the Senior Software Architect/Engineer at One-Two. I like to experiment with new technologies, Computer Vision and A.I. My projects are programmed in B4X , Xojo, C#, java, HTML, CSS and JavaScript. View all posts by Alwaysbusy

13 responses to “RealBasic: Computer Vision -> OpenSURF feature matching

  • Geoff Perlman

    For what it’s worth, if your code is math-intensive, then the reason it’s slower than C# is because the Real Studio compiler is not an optimizing compiler. However, we are in the process of moving the backend of our compiler to LLVM which IS an optimizing compiler. So once that’s done, your code to speed up considerably.

    • Alwaysbusy

      Thanks for the update Geoff! Looking forward to see what the LLVM compiler is capable of. I’m also experimenting with the RealBasic plugin feature to see where I can gain more speed.

  • Raphael

    GREAT! I tried this myself and did not succeed, neither have the time to make me understand the whole thing. I’m good with RB, but have no clue (yet) on C / Objective C programming or reading and compiling libraries etc.

    Out of that reason – do you think you could show a tutorial on how to compile a cross-platform static library of openCV or for mac / linux and how to use it inside a Real Studio application? Is that possible via declares?

    I would really love to know how to do that and how to interpret the man pages of openCV to continue building with it inside Real Studio.
    I kinda want to unleash the power of the library and MAYBE be able to compile updates myself.

    Or have you succeeded in building a plugin?
    Would be really interested in that.

    Your work is awesome, but atm it really is at least “kind of” slow.

    Thank you very very much for this already !
    I think without you I would possibly really have to learn Objective C before I could do anything with openCV. Now there’s a chance I could work with what I already now and learn C and all the more advanced compiling backgrounds spear over time.

    Best regards,
    Raphael

    • Alwaysbusy

      Hi Raphael,

      Ive tried making OpenCV into a RB lib before but I never succeeded. I’m afraid learning Object C is going to be needed. However, having the java version could help you if you’re willing to translate some code to RB. Some functions (like the above) are real clever but are not using advanced C or java, so you can translate them with ease to RB. I converted some of the OpenCV functions for an inhouse project at OneTwo to create Scaper (some kind of OCR app).

      It does take some time to get browse through the OpenCV code to find what you’re looking for. But It is worth it!

      Cheers!
      Alwaysbusy

  • Raphael

    There also seems to be a Java version. Can’t we link that and use it as well? Except I don’t know how, it should be possible, right? At least with the Monkeybread Plugins?

  • Dave Fessler

    This only seems to work for the first pair of pictures selected. Selecting a second pair without quitting and rerunning the program does not result in any matches. I’m running on OS X 10.10.

    Thanks.

    • Raphael S. E.

      Dave, I cannot confirm this …
      Either it has been fixed by #Alwaysbusy meanwhile or there seems to be another issue.

      I am on 10.10.2 ran it in debug mode and also compiled the application using Xojo.
      Maybe you forgot to use the “Find features” button after loading your second set of images?

      @alwaysbusy
      Am I mistaken, or does the time it takes to find the feature vectors scale with the amount of features the image has?
      I have not taken any closer look at the code for a long time, but it feels as if larger pictures can be processed faster than the included sample pictures, if there are fewer features in the image itself ?!?

      • Dave Fessler

        Thanks for your response.

        I’m still playing with it, it may be that I’m expecting matches where the algorithm isn’t finding them. As I test this with more images I’ll let you know the results.

    • Alwaysbusy

      It has been a long time since I worked on this, but wouldn’t reinitializing ABOpenSURF (= new ABOpenSURF) just start from scratch?

      @Raphael : Never benchmarked this. Maybe the number of possible features has indeed a bigger impact on speed than the size of the image.

  • Raphael

    Hey there!
    I have meanwhile had the opportunity to learn Java and C#, which in turn made me able to understand C / C++ a bit more easily than before.
    Interestingly, this little exploration of languages led me to kind of leave Xojo behind (really not happy at all with the new interface and the slow bug fixing … I have colleagues with probably the largest RealStudio projects on sale to date and they have almost no say in the decision making over at Xojo; meaning, they will be stuck in RS because Xojo is just not usable enough for quick paced developers who don’t want to click ten times to get to where they want …)
    So although I am not using Xojo for serious development anymore, I still use it for little experiments or modules and classes for clients …

    Just out of interest though:

    Do you have any word now on the LLVM compiler?
    Or are you still working on any OpenCV implementations for Xojo?

  • Alwaysbusy

    Hi Raphael,

    I agree Xojo really has to step up if they want to play with the big boys. The IDE (at least on Windows) is still the reason for a lot of frustration. 90% of the projects in our company are also still in RealStudio and only if a certain feature is needed (like we needed support for proxy logins), we move it to Xojo.

    But we avoid it if possible, also because of the huge DLLs that are included in the build. Our product consists of 50+ EXEs and EVERY exe would have the same DLLs included, making the product unnecessary big. Unacceptable…

    Sometimes I have the feeling Xojo is losing touch with reality more and more. And the new framework is not going to help I’m afraid.

    About the LLVM? All I get is the tiring ‘When it is ready’. And as Geoff replied on this post almost three years ago, I wouldn’t hold my breath for the near future.

    I’ve given up working on OpenCV I’m afraid. As vision is only for hobby stuff for me, I wrote ABXVision that can handle my needs. Still believe OpenCV is the best, but probably better used with more advanced languages instead of Xojo.

    I’m in the middle of moving a big RS project on PC to tablets with B4A and it is a big relief working with the tools of Anywhere Software. No crashes, superior debugging tools, fast solutions if there is a problem from the makers, clear statements from the company what to expect in the future. Makes me feel nostalgic to the good old RealBasic days…

    Cheers!

    Alain

  • Raphael S. E.

    Hey Alain!
    I more and more get the feeling the Xojo / RealStudio world has divided into three groups: the beginners who like experimenting (Xojo sure is a nice playground for them), serious application developers who grew out of playground environments for a long time and then the last group, the Xojo staff.
    The only positive feedback seems to come from the new users, which also seemed to be the reason behind the iOS support hype. The staff doesn’t seem to think about existing pro users, but “new” beginner user clients.

    I think they have yet to understand that the “new” user search is just like draining a pool. New users might come, but on the other end, pro and long term members resign; and they would be the real source for money …
    I really don’t understand the kind of decision making going on there.

    When the new subscription based system was implemented I thought it was a joke. But yes, it sure is the best payment solution for someone who wants steady payments, without having to work …
    Only problem is: As soon as people realize nothing happens, payments stop, holidays are over. And then: Oh, let’s do iOS – it takes a long time to do and payments are justified again. But let’s do it even slower!

    They should really look at IDE’s like those from JetBrains, or even Apple and Microsoft. Those are the ones that just work. For anyone willing to commit. Pro and beginner. And they take a lot of work out your hands!

    Anyhow. Enough ranting and raving …

    I did follow the new Anywhere Software empire.
    But there doesn’t seem to be a Mac IDE, or I am too dumm to find it.
    However, I appreciate there’s someone who’s trying to make products the way customers expect them !

    I do know, that I don’t even know how much I don’t know 😉 but this is exactly the reason that keeps me going with long term professional IDE’s and languages at the moment. REALBasic -was- a good start, but it’s not my future. I rather develop my applications in two separate languages to support three or more platforms, than working with a platform that slows me down.

    To finish up on topic though:
    I am not a math guru, so I appreciate the work you’ve put into the Xojo OpenSURF version. It did help to draw a connection between the source and your code and made it a lot easier!

    I say nothing against the Basic language. It always has been easy to understand and to explain things. I just don’t agree with the Xojo approach.

    And Alain, if you still need any help with translations, hit me up again!
    I always appreciate simple tasks in between real work 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: