Multimodal Interaction with a Wearable Augmented Reality System: the perfect combination

The computing applications of the wearable computers set up the need for a more context-oriented interface comparing with the classical desktop computers. This article tries to describe what a multimodal interface is and how can it improve the user’s situational awareness and provide new functionality.

By using an augmented reality system, one can see additional information that is otherwise invisible. As an example can be the situation from the action games, where on a HUD you could see a small map of the surrounding infrastructure, your hit points and the amount of ammo you have. A modern augmented reality system can provide such information, as room temperature, wireless signal strength, other important environment parameters, that can be processed by special applications and provide tenance information, emergency response and reconnaissance missions. A wearable computer that also includes GPS can provide you with a compass and with your recent coordinates. In order to be able to interact with such data on a wearable computer, there must be included additional ways of interaction between the user and the machine, as for example the voice commands and gesture recognition. In combining these types of interfaces with the classical ones, one can receive an intuitive and effective way of input.

Augmented reality systems have some great advantages, compared to classical desktop user interaction. The main characteristics of an augmented reality system are:

1. real time user interaction,
2. the combination of the virtual information with the surrounding environment, and
3. is registered in 3-D.

But the ease of use and great interactivity of the augmented reality systems require additional computing resources.

Such kinds of systems have already been tested both indoor and outdoor and there already are previous user experiences. The authors postulate that the new-generation interaction metaphors must be developed together with user interfaces that can control such levels of user input.