Microsoft Research to ship Kinected Browser JavaScript library for Kinect-enabled websites

Kinected Browser JavaScript library

Dan Liebling and Meredith R. Morris from Microsoft Research will be presenting today at the 2012 ACM International Conference on Interactive Tabletops and Surfaces about their project “Kinected Browser: Depth Camera Interaction for the Web“, a toolkit which enables web developers to detect Kinect gestures and speech using a simple JavaScript library.

With the introduction of Kinect for Windows earlier this year and the increasing adoption of HTML5 and CSS3 browsers which support rich multi-touch events, Dan and Meredith saw an opportunity to expand the developer reach of the Kinect SDK beyond its C# and C++ roots.

Depth-camera based gestures can facilitate interaction with the Web on keyboard-and-mouse-free and/or multi-user technologies, such as large display walls or TV sets. We present a toolkit for bringing such gesture affordances into modern Web browsers using existing Web programming methods. Our framework is designed to enable Web programmers to incrementally add this capability with minimum effort by leveraging Web standard DOM structures and event models.

The team recognizes some existing work by the community of Kinect and browser integrations: Swimbrowser, an app that uses Kinect to “swim” around websites but does not offer any developer endpoints, and DepthJS, a Safari and Chrome plugin that exposes a similar JavaScript library with more focus on high-level gestures.

In contrast to these, Kinected Browser is a low-level approach that only exposes raw sensor (RGB and depth) data. It works by two components, a browser plugin that talks directly Kinect C++ SDK and a JavaScript library which a web developer can include and redistribute on the page. In the future, the browser plugin can also be abstracted to work with a number of depth camera systems.

The benefits of this “versatile” approach is that they support many more exciting scenarios such as a “Web applications using depth data, e.g., to scan physical artifacts and input them into 3D printing services” or even multi-user interactions on a single website enabled by facial recognition and skeleton tracking.

Compared to Xbox game titles or apps, distributing and promoting a website is much easier. It would be fascinating to see if websites becomes a popular platform for new Kinect hacks and games.

The Kinected Browser library should be available for download from the Microsoft Research project site later today.