Javascript pen recognition
Here is a demonstration of a Graffiti-style block recognizer in javascript. It's based on the same algorithm as my previous C# implementation, and uses the firefox for the new <canvas> element in Firefox 1.5 (codename DeerPark).
The canvas is one of the many new features in Firefox 1.5. Its a screen area that can be painted programmatically. Apple first introduced it in Safari and the recent Firefox 1.5 (currently in beta) implements it as specified (Web Applications 1.0 spec from the WHATWG group).
Limitations:
The graffiti recognition is not perfect, as I didn't implement some of the manual tuning/heuristics from my previous implementation. Moreover, the alphabet that I used isn't quite the official Graffiti one, but that could be fixed.Also, I was suprised at how much CPU is consumed by just tracking the mouse and drawing on the canvas; my 1GHz machine gets slown down even with this lightweight canvas demo.
Some further ideas:
Firefox 1.5 has a subset of SVG built-in. It should be easy to write an interface based on SVG rather than the canvas. This would be nice for allowing to the post resulting "ink" back in an XML format.The canvas element in general opens lots of possibilities for games and my graffiti script could be used for some too (sudoku, crosswords?). Besides that, I've been thinking about a canvas-based graphical interpreter such as Logo (great for teaching programming to kids and reminds me of some good old APPLE II days).
______________________________________Julien, great demo.
I've been thinking for a while about a mouse gesture lib. With all the drag-and-drop going on, i think it would be nice to be able to delete something by striking it out, for example. With canvas, you could easily show the strokes like the FF gestures extension. (I don't think there's a conflict with that extension as ajax mouse gestures would use left mouse button.)
BTW On the pen demo, any chance of swapping the alerts for a "pad" showing the history of letter guesses?
Posted by: Michael Mahemoff (September 29, 2005 11:37 AM) ______________________________________Hey Micheal,
What kind of gesture are you thinking for scratching? A single line (left to right or vice versa) or a more complex scratching gesture?
The latter wouldn't work with my current implementation, as the number of back and forth would vary.
I removed the alert box. I actually had de-activated the "pad" code before putting it online ;-)
Posted by: Julien (September 29, 2005 12:23 PM) ______________________________________In the case of deletion, probably a single diagonal stroke is good enough provided it did the usual gesture-like things - "You appear to be using a mouse gesture for the first time. ..." - and a visual indication of what will happen before the button is actually released.
More generally though, I had in mind an engine like the FF extension, which transforms gestures into a sequence of numbers, so can easily be used generically. (I don't have the extension right now, I think it uses a number 1-8 for each possible direction.) You could then say:
portlet.onGesture = function(gesture) {
if (gesture.strokesMatch(1,8)) {
// User went left-up then right-down
}
}
I have just converted this over to work with dojo (dojotoolkit.org) and this trick so that it works with IE too:
http://me.eae.net/archive/2005/12/29/canvas-in-ie/
Email me if you want any further details.
I haven't made it public though because you are using GPL.
Morris
Posted by: Morris Johns (February 14, 2006 05:44 AM) ______________________________________Morris,
Nice work. I'd be happy to see it.
GPL doesn't prevent you from making it public, on the contrary. What it means is that you have to provide the source of any modifications that you make. It basically "prevents" someone from taking from the community without giving back too.
Let me know when you post it.
Posted by: Julien Couvreur (February 20, 2006 11:11 AM)