Motion Gestures Touch Demo
This demo showcases Motion Gesture's embedded machine-learning solution for touch gesture recognition. With Motion Gestures, users can quickly develop new ML models to recognize custom, user-defined gestures. The recognition models can then be deployed back to the embedded application and integrated within Microchip's 2D Touch Surface Library. The Motion Gestures solution is suitable for deployment on Microchip Arm® Cortex®-based 32-bit microcontrollers and microprocessors.
Figure 1: Integrated Graphics and Touch (IGaT) Curiosity Development Kit with the Motion Gestures Touch Demo
How It Works
The Motion Gestures recognition engine is deployed as a static C library within the embedded application and it interfaces with Microchip's 2D Touch Surface Library in order to detect gestures input by the user on the 2D touch surface.
New gesture recognition models can be developed rapidly because only a single example of each desired gesture is required to train the model. The Motion Gesture's Software Development Kit (SDK) takes care of generating the training dataset from the one provided example of the desired gesture so that developers can skip the long data collection process that is typically required for ML solutions.
Figure 2: Motion Gestures Development Flow
Gesture Definitions
The eight distinct gestures that the demo library is trained to recognize are shown below. The start of each gesture is marked by a dot and the end of the gesture is marked with an arrow. Recognition is path-dependent. Refer to the gesture definitions to see how each gesture should be drawn. Keep in mind that the finger should not be lifted until the gesture is complete.
The eight distinct gestures that the demo library is trained to recognize
These eight distinct gestures map to six labels: M, Check Mark, S, 2, Alpha, Star. There are two variations of the M and 2 gestures. In the demo, these variations will map to the same predicted label, so that a capital M and a lowercase m will both be mapped to the M gesture label.
Hardware Platforms
The Motion Gestures demo is available on two different hardware platforms. On both platforms, the demo is integrated within Microchip's 2D Touch Surface Library so that all of the standard 2D Touch and Gestures functionality is available as well. The Motion Gestures Demo Library will detect the complex demo gestures while Microchip's 2D Touch Surface Library will detect fundamental single-finger gestures such as taps, swipes, wheels, and even dual-finger gestures such as pinch, zoom, or dual-swipe.
SAME51 Integrated Graphics and Touch (IGaT) Curiosity Development Board
Figure 4: IGaT Curiosity Development Kit Running the Legato Showcase Demo
The SAME51 Integrated Graphics and Touch (IGaT) Curiosity Development Board comes preprogrammed with the initial version of the Legato Showcase Demo. This demo application is a part of the MPLAB® Harmony 3 Graphics application examples for SAM D5x/E5x Family. The latest firmware can be found on GitHub or it can be installed from the Harmony Content Manager within MPLAB X IDE. The Motion Gestures Touch Demo is integrated within the AI/ML portion of the Legato Showcase Demo.
SAMC21 Xplained Pro with QT8 Xplained Pro Touch Surface
Figure 5: SAMC21 Xplained Pro with the QT8 Xplained Pro Connected to EXT1
To use the Motion Gestures Touch Demo on the SAMC21, download and program the board, then use Microchip's 2D Touch Surface Utility on a Windows® machine to view the gesture recognition results.
If you are not familiar with Microchip's 2D Touch Surface Utility, please download the "QT8 Xplained Pro User's Guide."
The 2D Touch Surface Utility can be used to simultaneously display gesture recognition results from the Motion Gestures Demo Library and Microchip's 2D Touch Surface Library. The Current Gesture window will display the most recent recognized gesture. Results from the Microchip Touch Library will be displayed in blue and results from the Motion Gestures library will be displayed in green.
Figure 6: Microchip's 2D Touch Surface Utility Showing That the M Gesture Has Been Recognized
Summary
With the Motion Gestures solution, complex touch gesture recognition models can be developed in a matter of minutes thanks to the power of machine learning. With just one example of each desired gesture, the Motion Gestures SDK can generate highly accurate recognition models. To learn more, visit the Motion Gestures website.
