Machine learning you can teach in seconds
This page recognises gestures drawn with a mouse, trackpad, finger or stylus. It comes with a few starter gestures (swipe left, right, up and down, a circle and a check mark) so it works straight away. The real fun is teaching it your own. Draw a shape, such as a zigzag, a letter or a spiral, type a name and press Teach. Do that three to five times, drawing a little differently each time, and the predictor will start recognising the new gesture. Your examples are saved in this browser's local storage, so they are still there next time, and nothing is uploaded.
How recognition works
A stroke is a list of points, but two people drawing the same shape produce different numbers of points at different sizes and speeds. So every stroke is first normalised in three steps. It is resampled to 32 points spaced evenly along the path, which removes the effect of drawing speed. It is moved so its centre sits at the origin, which removes position. And it is scaled to a standard size, keeping its proportions, which removes size while keeping a horizontal swipe horizontal. Direction is preserved, so left and right swipes stay different.
To classify a new stroke, the predictor measures the average distance between its 32 points and those of every stored example, then lets the k nearest examples vote, with closer examples counting more. This is the k-nearest-neighbours algorithm, the same idea behind the well-known $1 gesture recogniser from HCI research. It needs no training phase: adding an example takes effect instantly, which is why it can learn from just a handful of samples. The bars show the share of the vote each gesture received, and a note appears when the stroke is unlike anything learned so far.
Tips for good results
- Give each gesture three to five varied examples rather than one perfect one.
- Gestures that differ only in size are hard to tell apart, because size is normalised away.
- Drawing direction matters: a clockwise and an anticlockwise circle are different gestures. Teach both if you want both recognised.
- Set k to 1 when you have only one example per gesture.
Where this is used
Gesture shortcuts in drawing apps, handwriting shortcuts on tablets and game controls use similar techniques. Larger systems use neural networks trained on thousands of examples; this lightweight approach shows how far careful preprocessing and a simple algorithm can go.