Here is a complete framework to make ‘gestures’ with the mouse (or touch) to perform some action. It’s a Xojo translation (with some additions) of the work of Dr Per Ola Kristensson. The engine can recognize user build gestures on the fly.
As always, I’ve tried to make using the engine as easy as possible. The Engine contains the following classes:
You can use this with very little code, so let’s start!
In the Open() of the window, Initialize a ABMGRecognizer:
' MUST BE INITIALIZED AS SOME FUNCTIONS ARE USED TO BUILD THE TEMPLATE BITMAP IN VALUEPAIR ! Recognizer = new ABMGRecognizer
Load some templates in the engine. A template is build by adding keypoints. Suppose we want to add the following template:
dim ID as string = "My template" dim Templates(-1) as ABGMTemplate ' set the points dim newPoints as new ABMGPoints newPoints.Vals.Append(new ABMGPoint(0,0)) ' start point (always 0,0) newPoints.Vals.Append(new ABMGPoint(0,1)) ' One Down (0 + 1 = 1) newPoints.Vals.Append(new ABMGPoint(-1,2)) ' One Left (0 - 1 = -1), One down (1 + 1 = 2) newPoints.Vals.Append(new ABMGPoint(-1,0)) ' two Up (2 - 2 = 0) newPoints.Vals.Append(new ABMGPoint(-2,0)) ' One left (-1 - 1 = -2) newPoints.Vals.Append(new ABMGPoint(-2,-1)) ' One Up (0 - 1 = -1) ' create and add the template Templates.Append(new ABMGTemplate(ID, newPoints))
So every point is relative to the start point (0,0).
Next we add the Templates() to the the engine:
' Creates the continuous gesture recognizer for the template set. This ' template set can be changed later by calling the recognizer's ' setTemplateSet method. The second parameter is the distance between ' sampling points in the normalized gesture space (1000 x 1000 units). ' A lower value improves precision at the cost of increased memory and ' processing time. Recognizer = new ABMGRecognizer(Templates, 5)
In the canvas we handle the movement of the mouse cursor. We add the new position to Input (global defined ABMGPoints). And run a recognize function to check for the pattern.
Function MouseDown(X as integer, Y as integer) as Boolean ' Initilaize our Input Input = new ABMGPoints End Function Function MouseDrag(X as integer, Y as integer) ' add the new point Input.Vals.Append new ABMGPoint(X,Y) Recognize End Function Function MouseUp(X as integer, Y as integer) ' add the last point Input.Vals.Append new ABMGPoint(X,Y) Recognize End Function Sub Recognize() if Input = nil then Return if UBound(Input.Vals) < 1 then Return ' we need at least to points if Recognizer = nil then Return ' precaution dim Results() as ABMGResult = Recognizer.Recognize(input) if Ubound(Results) = -1 then Return ' nothing found end if ' do something with the results! They are ordered with the most likely one first. End Sub
A more extented example is added at the bottom of this article.
Here you can see the extended example running:
This engine works on Xojo for Windows and OSX.
Download the full source code from: