RoomWare, Augmented reality and new exiting stuff happening fast

Posted on June 4, 2009


This is a collection of AR and RoomWare related items and inventions.

High definition 3D city maps

Using aeroplanes, high resolution camera’s and software to crunch the images, a sub-division of Saab has created a solution to map cities and object in 3D. Watch the video here (

Augmented reality using trackpoints from the environment

The video below shows the result of an experiment using tracking points in the real world to get the 3D mapping coordinates to do Augmented reality.

On the iPhone:

While riding a train:

Tracking movement and gestures to interface

Microsoft and Sony are both releasing new technologies that presents the next step of human computer interfacing.

Microsoft Natal allows you to use gestures almost anywhere in the room and directly manipulate things in a 3D simulated environment. I heard from a reliable source (August 2009) that Natal still has trouble tracking objects, but I guess it is a matter of time before we will see it in action.

Sony has done some work on a new controller which (using camera feeds) does quite a good job as well:


Alexander Zeh and I are currently shaping a new presentation regarding RoomWare. RoomWare is about buildings and spaces as living entities with which you can interact. We are mainly focusing on lo-tech solutions now, but these new emerging technologies are really very exiting. They will enable us to blend reality with data and virtual reality in a way I personally did not think was possible for the next five years.

My first reaction today (June 4, 2009)  working on my Flash RAD framework and seeing the Augmented Reality demo’s I thought: “Fuck – I quit.”

On the other hand: now is the time to start implementing these new things.

Breaking language barriers: Translations on the fly

Google has this translation service for a while now. I have seen the first demo’s in Wave, translating French to English on the fly. The translations are dodgy still. But give it time and it will very likely improve and useful for on the fly translations in any language you need.

[August 20, 2009]

Do It Yourself “Minority Report” interface in Flash

Last week (August 10 to August 16) I developed the first of a series of cheap interfacing means using Flash and the Web cam to create different interaction scenarios.Check this page.


SixtSense is a combination of a on-body portable beamer, a cell phone and a web-cam. It responds to the environment and is a portable version of the “Minority Report” interface. Developed by Pranav Mistry. Watch this page and the videos. It is awesome.

ARTag: Using AR table wide

The current Flash AR / FLAR applications are what I  mockingly call “some movable shit on a paper”. The possible use is wider than that, using the AR tags as location markers on tables and enabling you to use large scale surfaces to do AR on. As I was outlining my own “next step” with FLAR toolkit, I came across ARTag via a friend. ARTag is already using combined AR tags. See here where AR with Flash will move to.

TrackMate: Reactable revisited

TrackMate shows a simple and portable implementation of the “Reactable” concept, using object with markers to create interactions with a computer system. See their page here.

Crystal: Integration of devices in the room

The guys from Crystal are showing a nice integration of devices and media using a “touch table” with top projection.

What makes it so exiting and new?

All examples (except Sony) show new ways where real world/ virtual world connections are happening on the fly. We are not far away from the point where “facial recognition” is done with environments as they are and our mobile phone will show overlays of data. Interfacing with spaces can be done using your body. No hardware required. Mapping cities as Saab does will make virtual explorations over and through a city get within the reach of our hands. “Where am I” gets new tools to find answers.

Posted in: Uncategorized