Kobito: "Magic" on Your Breakfast Table

You wake up in the morning and sit at your breakfast table. There comes a can of Earl Grey, magically sliding towards you. But, who's suggesting to have a cup of tea? A group of students at Tokyo Institute of Technology created an installation called "Kobito: Virtual Brownies," which allows us to see and feel who it is: kobitos (dwarves).

[conceptual overview. illustration by cami dodson.]

The system was developed using Computer Vision, Artificial Intelligence and Haptic Feedback technologies. The dwarves appear in a movable display device called Kobito Window (see below) that blends video images and virtual 3D avatars of dwarves. Kobito Window is like a magic lens through which people can view the world differently from the way our naked eyes can see: the device recognizes its orientation using a rotary encoder and displays 3D avatars based on a current view angle. Moreover, you can feel the dwarves through haptic feedback. If you hold the tea can, you can feel the force of the dwarves. If you push the can against the dwarves, they will react to it.

[system at work. the display device at the lower bottom corner is Kobito Window.]

This mpeg movie shows the overview of the system with English captions.

Kobito will also be showcased at SIGGRAPH 2005 which will soon take place at Los Angeles Convention Center (July 31 - August 4). A description of their installation can be found here.

 
[left: vision system for tracking objects, right: mechanical component hidden under the table]

The camera mounted on the ceiling tracks the position and the orientation of the tea can. Based on this tracking technology, the software application determines how 3D avatars act. The system presents the 3D avatars visually (in Kobito Window) and physically (on the table as movements of the tea can). The tea can is controlled by a mechanical component hidden under the table (see the above photo). This component, whose position can be programmatically controlled, integrates a magnet for moving the metal can without physical contact.

People who created this installation think systems like this "can be used in the fields of design, amusement, and healthcare."

More video clips:
Kids playing with Kobito1
Kids playing with Kobito2

Related Website:
Robot Technology Research Forum at Tokyo Institute of Technology
(in Japanese)