Monthly Archives: April 2014

Xojo: Mouse (Touch) Gestures Engine

Random Generated Gestures

Random Generated Gestures

Here is a complete framework to make ‘gestures’ with the mouse (or touch) to perform some action. It’s a Xojo translation (with some additions) of the work of Dr Per Ola Kristensson. The engine can recognize user build gestures on the fly.

As always, I’ve tried to make using the engine as easy as possible. The Engine contains the following classes:

Gesture Engine Structure

Gesture Engine Structure

You can use this with very little code, so let’s start!

In the Open() of the window, Initialize a ABMGRecognizer:

  Recognizer = new ABMGRecognizer

Load some templates in the engine. A template is build by adding keypoints. Suppose we want to add the following template:

Example Template

Example Template

dim ID as string = "My template"
dim Templates(-1) as ABGMTemplate
' set the points
dim newPoints as new ABMGPoints
newPoints.Vals.Append(new ABMGPoint(0,0)) ' start point (always 0,0)
newPoints.Vals.Append(new ABMGPoint(0,1)) ' One Down (0 + 1 = 1)
newPoints.Vals.Append(new ABMGPoint(-1,2)) ' One Left (0 - 1 = -1), One down (1 + 1 = 2)
newPoints.Vals.Append(new ABMGPoint(-1,0)) ' two Up (2 - 2 = 0)
newPoints.Vals.Append(new ABMGPoint(-2,0)) ' One left (-1 - 1 = -2)
newPoints.Vals.Append(new ABMGPoint(-2,-1)) ' One Up (0 - 1 = -1)
' create and add the template
Templates.Append(new ABMGTemplate(ID, newPoints))

So every point is relative to the start point (0,0).

Next we add the Templates() to the the engine:

  ' Creates the continuous gesture recognizer for the template set. This
  ' template set can be changed later by calling the recognizer's
  ' setTemplateSet method. The second parameter is the distance between
  ' sampling points in the normalized gesture space (1000 x 1000 units).
  ' A lower value improves precision at the cost of increased memory and
  ' processing time.
Recognizer = new ABMGRecognizer(Templates, 5)

In the canvas we handle the movement of the mouse cursor. We add the new position to Input (global defined ABMGPoints). And run a recognize function to check for the pattern.

Function MouseDown(X as integer, Y as integer) as Boolean
    ' Initilaize our Input
    Input = new ABMGPoints
End Function

Function MouseDrag(X as integer, Y as integer)
    ' add the new point
    Input.Vals.Append new ABMGPoint(X,Y)
End Function

Function MouseUp(X as integer, Y as integer)
     ' add the last point
    Input.Vals.Append new ABMGPoint(X,Y)
End Function

Sub Recognize()
  if Input = nil then Return
  if UBound(Input.Vals) < 1 then Return ' we need at least to points
  if Recognizer = nil then Return ' precaution
  dim Results() as ABMGResult = Recognizer.Recognize(input)
  if Ubound(Results) = -1 then
    Return ' nothing found
  end if
  ' do something with the results! They are ordered with the most likely one first.
End Sub

A more extented example is added at the bottom of this article.
Here you can see the extended example running:

This engine works on Xojo for Windows and OSX.

Download the full source code from:



Click here to Donation if you like my work


%d bloggers like this: