Xojo: Mouse (Touch) Gestures Engine

Random Generated Gestures

Random Generated Gestures

Here is a complete framework to make ‘gestures’ with the mouse (or touch) to perform some action. It’s a Xojo translation (with some additions) of the work of Dr Per Ola Kristensson. The engine can recognize user build gestures on the fly.

As always, I’ve tried to make using the engine as easy as possible. The Engine contains the following classes:

Gesture Engine Structure

Gesture Engine Structure

You can use this with very little code, so let’s start!

In the Open() of the window, Initialize a ABMGRecognizer:

  ' MUST BE INITIALIZED AS SOME FUNCTIONS ARE USED TO BUILD THE TEMPLATE BITMAP IN VALUEPAIR !
  Recognizer = new ABMGRecognizer

Load some templates in the engine. A template is build by adding keypoints. Suppose we want to add the following template:

Example Template

Example Template

dim ID as string = "My template"
dim Templates(-1) as ABGMTemplate
' set the points
dim newPoints as new ABMGPoints
newPoints.Vals.Append(new ABMGPoint(0,0)) ' start point (always 0,0)
newPoints.Vals.Append(new ABMGPoint(0,1)) ' One Down (0 + 1 = 1)
newPoints.Vals.Append(new ABMGPoint(-1,2)) ' One Left (0 - 1 = -1), One down (1 + 1 = 2)
newPoints.Vals.Append(new ABMGPoint(-1,0)) ' two Up (2 - 2 = 0)
newPoints.Vals.Append(new ABMGPoint(-2,0)) ' One left (-1 - 1 = -2)
newPoints.Vals.Append(new ABMGPoint(-2,-1)) ' One Up (0 - 1 = -1)
' create and add the template
Templates.Append(new ABMGTemplate(ID, newPoints))

So every point is relative to the start point (0,0).

Next we add the Templates() to the the engine:

  ' Creates the continuous gesture recognizer for the template set. This
  ' template set can be changed later by calling the recognizer's
  ' setTemplateSet method. The second parameter is the distance between
  ' sampling points in the normalized gesture space (1000 x 1000 units).
  ' A lower value improves precision at the cost of increased memory and
  ' processing time.
Recognizer = new ABMGRecognizer(Templates, 5)

In the canvas we handle the movement of the mouse cursor. We add the new position to Input (global defined ABMGPoints). And run a recognize function to check for the pattern.

Function MouseDown(X as integer, Y as integer) as Boolean
    ' Initilaize our Input
    Input = new ABMGPoints
End Function

Function MouseDrag(X as integer, Y as integer)
    ' add the new point
    Input.Vals.Append new ABMGPoint(X,Y)
    Recognize
End Function

Function MouseUp(X as integer, Y as integer)
     ' add the last point
    Input.Vals.Append new ABMGPoint(X,Y)
    Recognize
End Function

Sub Recognize()
  if Input = nil then Return
  if UBound(Input.Vals) < 1 then Return ' we need at least to points
  if Recognizer = nil then Return ' precaution
  dim Results() as ABMGResult = Recognizer.Recognize(input)
  if Ubound(Results) = -1 then
    Return ' nothing found
  end if
  ' do something with the results! They are ordered with the most likely one first.
End Sub

A more extented example is added at the bottom of this article.
Here you can see the extended example running:

This engine works on Xojo for Windows and OSX.

Download the full source code from:
http://gorgeousapps.com/ABMouseGesture.zip

Bye!

Alwaysbusy

Click here to Donation if you like my work

Advertisements

About Alwaysbusy

My name is Alain Bailleul and I'm the Senior Software Architect/Engineer at One-Two. I like to experiment with new technologies, Computer Vision and A.I. My projects are programmed in B4X , Xojo, C#, java, HTML, CSS and JavaScript. View all posts by Alwaysbusy

One response to “Xojo: Mouse (Touch) Gestures Engine

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: