Auszüge aus/Excerpts from :
"Human-Computer Interaction for Distributed Systems", SICS (Swedish Institute of Computer Science)
5.1 The Problem
Generally speaking, the human--computer interfaces found in today's computer systems are both complex and confusing. This problem will potentially become even more serious in the near future, when interfaces are extended from dealing with a single person interacting with one computer to dealing with users interacting in an open space of information, computing resources and other users. A further problem arises from the fact that an increased number of computer-naive users will be confronted with such systems through the widespread use of information technology in their work and leisure.
After some 30 years of development and evolution, interface designers still seem to be catering more to the demands of the computer itself than to the goal of providing users with efficient and transparent ways of getting their work done. This trend is also apparent in many electronic devices found in everyday life, such as the remote control of a conventional VCR. There is always a significant cognitive load or mental strain associated with using present-day computer-based applications.
5.2 Current State of the Art
Research on human--computer interaction has largely been concerned with two-dimensional interfaces based on icons, windows, menus and buttons. Although this is now beginning to change, the main input mechanisms to computer systems are still the conventional keyboard and the mouse. These techniques have proved to work fairly well for a single user interacting with applications having relatively limited functionality. However, they have serious shortcomings when the user has to deal with large amounts of heterogeneous information, when complex functionalities are needed to solve a problem, and when people have to work under time pressure. Also, existing techniques do not provide any natural representation of other users as needed in collaborative systems. Ideally, the interface of the future should try to alleviate these problems by using aspects of our natural behaviour in the world as a model --- moving around, manipulating objects, gesturing and talking.
On the positive side, however, several developments of the underlying technology can potentially enhance the opportunities for providing more efficient human--computer interaction:
The availability of rapid and broad-band multimedia communication will enable distributed applications that utilize multimodal interaction.
The increasing hardware performance will contribute to making computationally hard tasks like natural-language and speech processing feasible.
The availability of low-cost high-performance 3D-graphics platforms makes it possible to migrate high-quality interactive visualization to a large number of users.
The integration of tele- and computer communication will allow an integrated platform for synchronous and asynchronous communication, as well as mobile access to information and computing resources, thereby allowing for a high degree of independence of both time and place. Thus, we have potentially at our grasp the possibility of drastic improvements of the human--computer interface within the next few years.
5.3 Desirable Properties
The widespread adoption of distributed computing connected by broadband networks will generate both new requirements on, and create new opportunities for, the human--computer interface of the future. In particular, interfaces will have to meet the following desiderata:
To support single-user interaction as well as computer-mediated communication and collaboration between groups of users, powerful interaction metaphors are needed. Examples include personal assistants acting on a user's behalf in an environment of agents, and the spatial metaphor embodied by virtual realities.
Interaction metaphors of the above kinds typically involve advanced or human-like forms of communication in which several modalities interplay. Multimodal interaction will thus play an increasingly important role, and will include facilities like speech, gestures, tactile feedback and full support for multichannel video and audio. It is particularly important that Swedish is available for the spoken- and written-language modality, so that Swedish users can communicate in their native language.
To guide and support people using large, shared, heterogeneous and globally distributed information spaces, drastically improved means for selection, extraction and filtering of information will be needed. Systems that provide efficient support for users in navigating and exploring distributed information sources will thus be needed.
5.5 Technical Description....
5.5.1 Distributed Virtual Reality Systems
A virtual reality system is a support system for interaction and navigation in a 3D workspace. A user perceives the environment as a ''world'' - he is given a location and a ''presence'' in the 3D environment and can interact with objects and other users present in the same environment. Virtual reality has attracted vast attention for its potential to provide transparent and natural means of manipulating objects, accessing services communication between humans.
To realize their full potential as the communication, information and entertainment tools of tomorrow, virtual reality systems need to be able to support a large number of participants spread over wide geographical areas. These participants need to be able to work and interact with each other within large distributed information spaces. In order to build such systems one can identify three necessary or mission critical components.
There is a need for a high capacity distribution infrastructure incorporating a number of local and wide area computer networks together with switching and protocol capabilities to match.
We need a framework within which a multiplicity of applications can be developed and supported, that is able to handle the requirements from the underlying distribution layers without causing overload and bottle necks.
There has to be a user, object, and interaction model to aid both users and the underlying systems to cope with the communication intensive environments. A strong concern of such a model is that it has to be scalable.
Visualizing, populating and vitalizing distributed information systems
It is a little disputed fact that information is of most use when combined with an ability for people to communicate and co-operate with one another. In traditional information managing system communication and cooperation is, of course, also occurring but this is occurring outside the system itself and the user is left completely unaided and possibly even hindered by the system.
We envision a distributed information systems where communication and cooperation interactions will appear inside the environment and the enabling mechanisms necessary are inherent part of the system itself. Research in the field of Computer Supported Cooperative Work (CSCW) suggests that the notion of mutual awareness between users is critically important in many co-operative situations. (Some of the classical studies are in the domain of Air Traffic Control and the supervision of trains in the London Underground). As a consequence, awareness may also require sufficiently rich user embodiment within the environment itself. Support for cooperation also involves consideration of how information is shared and of further issues such as synchronization, history, version control and subjectivity etc.
Traditional information retrieval systems tend to view information as passive entities available (in many cases only) for inspection. When the amount of available information grows too large for a single user to inspect and manage, this approach to representing and visualizing information becomes insufficient. The alternative view is to regard information as active and responsive, realize that information may interact with other information and multiple users, and try to provide active and opportunistic services to users.
A goal of this research programme is to develop systems based on the integration of the information-linking and browsing functionality, as exemplified by the existing and evolving WorldWideWeb infrastructure, and the powerful visualization, interaction and collaboration capabilities of emerging multi-user, distributed virtual reality systems. Such an environment would offer an impressive set of capabilities in nearly all regards outlined above. For example, the system would be inherently multi-user and collaboration friendly, completely open and extendible, support many different kinds of media, provide for a range of platforms, offer multimodal interface facilities, and enable the incorporation of reactive agent-based software-techniques etc.
Augmented Reality and Augmented Virtuality
With the current state of competence in distributed virtual worlds, sensing algorithms, and mobile robotics we are building a foundation for a number of useful applications that will exploit our combined current research. One such application is the integration of real world mobile robots into a virtual world for the purpose of remote-operator robot control.
The virtual environment consists of a model of the real environment in which the physical robot operates. The real robot, which may be located in a different physical location than the user, is equipped with e.g. a camera, sonar range, and contact sensors. The video is sent back to the virtual environment and displayed for the user. The video can be used to enhance the user environment in two ways:
augmented reality: viewing the live real world images enhanced with graphical computer overlays;
augmented virtuality: using the video to enhance the virtual world.
Both of those graphical enhancements serve to aid the interaction with both the real and the virtual worlds. In addition, by having an agent performing actions in the real world, we begin to blur the division between real and virtual worlds even further.
Using DIVE as a basic framework we are working on the integration of the real and virtual world. For example if the task is to locate an object in a scene, if that object is recognized by a computer vision algorithm it could be graphically enhanced to both show its location to the user as well as inform the user of the robot's awareness of that object. Real images can be segmented and warped onto virtual objects to give them a more familiar appearance.
In the interface, we would like to explore how the user can best use the robot as a tool inside the virtual reality. Giving the user full access, in a natural way, to the robot capabilities is not trivial. The interface should be configured in a way that would allow the robot to act like an agent and expand and learn with user experience. This agency will allow the robot, when seen as a tool, to become more useful over time.
Some Research Issues
A number of research issues have been identified, e.g.:
Visualization: developing techniques for visualizing large collections of multimedia information. These techniques will allow users to obtain overviews of information; examine specific information in detail; visualize the results of applying search queries and filters; and create and manipulate information.
Embodiment: exploring the representation of users to one another in relation to the information they are using. A range of issues will be explored including conveying presence, position, identity, activity, availability, capability, as well as also enriched representations of human behavior, such as conveying gesture and facial expressions.
Awareness: developing flexible mechanisms so that users can selectively manage their awareness of other users and so that they can encourage others' awareness of them. In particular, mutual awareness might be related to proximity in an information space or to the degree of commonality of information being accessed.
Communication: supporting users in communicating with one another, both in real time and asynchronously using a variety of media, including audio, text, graphics and video. Communication also includes issues of synchronization (e.g. techniques supporting concurrent editing of multimedia information).
Distribution Mechanisms developing efficient distribution layers for multimodal information for wide range of network characteristics (enhancement of DIVE and SID developed by SICS).
Heterogeneity: cooperative multimedia systems may be accessed from a wide variety of equipment including desktop (i.e. screen-based), 2-D, and 3-D systems. Specific issues include interface architectures for multiple presentations of information; resolving perceptual problems arising from different presentations of shared spaces; and coping with differences in computing power and capability.
Scalability: developing techniques to support cooperation between large numbers of users. Mechanisms are required to allow users to flexibly and dynamically assign their computing and cognitive resources to selected users of greatest interest, so as to reduce problems of computational and cognitive overload.
Persistence and history: transforming information systems into 'places' for cooperative work requires introducing a notion of persistence, so that users become aware of changes that have taken place since their last visit and can associate these changes with others that have caused them.
Information sharing: developing techniques for populating information systems with entities and sharing these entities between users.
Reactive agents: explore and develop techniques for building reactive or ultra-responsive environments. One promising approach is to mate a virtual reality system with a state-of-the-art concurrent constraint language. This should take into account that the incorporation of this functionality into the virtual environment/interface must be done in a way that is both unobtrusive and empowering for the user.
Networking requirements: the combination of multimedia information systems with virtual reality systems will make significant demands of underlying communication networks. For example, traffic types will vary from continuous media (e.g. in video interaction and VR-world descriptions) to relatively small, but time critical, chunks of information (e.g. changes in position, collision detection and other events). Furthermore, the issue of perceived lag is critical to any sense of presence within a computer generated information space, and this is likely to be affected by network latency. It is therefore important to understand the kinds of traffic that such systems will generate and the effects of bandwidth and especially latency on the experiences of end users.