Curiosity is bliss    Archive    Feed    About    Search

Julien Couvreur's programming blog and more

Tablet PC programming


One problem, maybe the main one, with Tablet PCs today is the lack of ink-based applications. Microsoft's "Think in Ink" contests are definitely are great way to get more developer traction to the ink-enabled platforms. But my attempts at coding ink applications have led me to think that it'll take quite some time before rich ink applications flourish. More samples and articles demonstrating interesting techniques are needed to get developers to invest more effort in ink-specific challenges.

The problem
To begin, I believe that ink really becomes a powerful interface when it allows a "smart" paper behavior. Journal and document annotations are examples of this, because they replicate the natural and free form paper interface and extend it.

The Tablet PC SDK allows you to develop such applications, but it doesn't necessarily make it much easier. What I mean is that the SDK makes it easy to write certain classes of applications: paint programs and form-based text input. The applications that experiment with other models for input (maths instead of english) and interaction (gestures instead of windows widgets) are still quite difficult to write.
The .NET Show episode on the Tablet PC is certainly impressive, because it shows how easy it is to ink-enable an application and make use of the powerful text recognition engine of the Tablet PC. But as soon as you want features that are pretty different from the Journal, you are on your own.

I'm not saying that the Tablet PC's SDK is at fault here, as it is very elaborate (ink parsing and handwriting recognition) and it certainly provides useful abstractions that allowed applications like Corel's Graphigo and xThink's calculator to be developed. I'm also glad that the companies that developed such products are making money out of it and I agree they should definitely keep their techniques secret.
But the fact is the expertise or knowledge to develop things like Graphigo's simple shape recognition and xThink's equation analysis aren't readily available to the developer community. I know free form sketch recognition is still an open problem in research, but there are very few samples that actually explain even the basic techniques and this raises the barrier to entry for developing rich ink applications.

Last year, I spent some time on a unistroke recognizer in C# (but not on top of the Tablet SDK), similar to Palm's Graffiti. When I was searching the web for existing solutions for this problem, there wasn't many sources of information and they were either too intricate for me (Rubine algorithm) or not really detailed (just comments in source files).

Some ideas
Obviously this is a bootstrapping problem and if ink applications were becoming mainstream more developers would know how to write them or would be interested in learning to. But another way to feed the virtuous adoption cycle would be to help developers write cool apps today. And Microsoft could help there in a couple of different ways:
- publishing more research papers explaining ink parsing techniques,
- making the source for some PowerToys available and explaining it in some MSDN Mag articles,
- publishing more code samples,
- making small parts of the SDK available under some shared source license (providing a powerful API is nice, but teaching a powerful algorithm is better), like some shape recognition APIs,
- keeping the community site more active (keep Tablet PC Developer alive and improve it).

In terms of platform improvements, besides for Ink interop which I don't see coming anytime soon, I'll hold my rants until after Lonestar and the 1.7 SDK are released ;-) I know I'll love the new specialized TIP to enter urls in a browser, with spaces being removed and the url history used.

Other posts on the problem
As I was finishing this post, I saw Peter's post on How Microsoft could help Tablet PC ISVs over on his Tabula PC blog.

One thing I don't necessarily agree on is his "Don't dwell on code sharing", when it comes to contests. I understand his argument, but these contests aren't just the "Tablet PC app awards". You get money or prizes if you win, so showing a cool app isn't enough in my mind. At least the binaries of the winning apps should be made available, if you really think sharing the code would drive developers away.

One thing on which I definitely agree with him is the "Educate developers". But I think they need to be educated further than just on how to use the SDK. The same way there are many people out there that know how do write an HTTP server or proxy using the socket APIs, we need to have more widespread skills for writing powerful ink applications.

Loren also had a thread on this developer adoption problem, which led to starting an open Journal project called WriteFree.

Here are some more applications that go beyond the basic text recognition in forms:
Music Composition Tool.
Text edition with the pen, that demonstrates gesture interaction with text to edit it, and the newer demo.
Lesynski's inShape SDK that allows you to implement shape recognition amongst other things. takes a look at Tablet PCs after one year (found via Julia Lerman's Blog, I wonder why ;-). It is always hard to evaluate the success of new technologies, but the article clearly shows that more progress and adoption is needed.

comments powered by Disqus