Augmented reality (AR) is a technology in which a user's view of the real world is enhanced or augmented with additional information generated from a computer model. The enhancement may consist of virtual artifacts to be fitted into the environment, or a display of non-geometric information about existing real objects. AR allows a user to work with and examine real 3D objects, while receiving additional information about those objects or the task at hand. By exploiting people's visual and spatial skills, AR brings information into the user's real world. That is, AR allows the user to stay in touch with the real environment. This is in contrast to virtual reality (VR) in which the user is completely immersed in an artificial world and cut off from the real world. In VR systems, there is no way for the user to interact with objects in the real world. Using AR technology, users can thus interact with a mixed virtual and real world in a natural way. AR systems bring the computer to the user's real work environment, whereas VR systems try to bring the world into the user's computer. This paradigm for user interaction and information visualisation constitutes the core of a very promising new technology for many applications.

However, real applications impose very strong demands on AR technology that cannot yet be met. Some of such demands are listed below. In order to combine real and virtual worlds seamlessly so that the virtual objects align well with the real ones, we need very precise models of the user's environment and how it is sensed. It is essential to determine the location and the optical properties of the viewer (or camera) and the display, i.e. we need to calibrate all devices, combine all the local coordinate systems centered on the devices and objects in the scene in a global coordinate system, register models of all 3D objects of interest with their counterparts in the scene, and track them over time when the user moves and interacts with the scene. Realistic merging of virtual objects with a real scene requires that objects behave in physically plausible manners when they are manipulated, i.e. they occlude or are occluded by real objects, they are not able to move through other objects, and they are shadowed or indirectly illuminated by other objects while also casting shadows themselves. To enforce such physical interaction constraints between real and virtual objects, the AR system needs to have a very detailed description of the physical scene. There may be many useful applications of this technology. Just as the personal computer has changed the daily routine of the average office worker, computer technology will, in the future, very likely create even more dramatic changes in the construction, design and manufacturing industries. In order to get some idea of what this change will entail, let us examine how a typical construction worker may do his job in the future.

Mechanical Repair & Training: An engineer with some form of trackable, head mounted display and a see through visor can proceed with his work while seeing a continually updated, annotated display which assists him in identifying the components before him and reminds him of the tasks he must perform.

By simply putting three distinctive marks at known distances from each other, a wearable camera with known focal length can recover the 3D location of the plane defined by these three marks. By extrapolation from an on line technical manual, the rest of the object's 3D location can be derived. Thus, when a repair technician walks up to a broken machine, the machine can transmit its diagnostics to the technician's wearable. The wearable automatically determines the problem, locates the 3D position of the object, and overlays specific 3D real time step by step guidelines on the object for the technician to follow.

Interior Design and Modeling: An interior architect designs, remodels, and visualises a room, using furniture models from a database that are superimposed on video images of the room.

Computer Aided Surgery: A doctor performs surgery using Augmented Reality tools to see medical image data (including volume rendered versions and computer graphics renderings of physical implants or other devices) visually superimposed on a human subject.

Electric Manufacturing and Diagnostics of Printed Circuit Boards: A technician diagnoses what is wrong with a particular printed circuit board (PC board), using Augmented Reality and Artificial Intelligence technology to analyse the reported symptoms, highlight the detected trouble spots in a video image of the PC board, and overlay relevant information that is pulled out of user manuals.

Apparel Retail: A customer shops for clothes, using an augmented reality system to look through an extensive electronic catalogue and electronically "wear" selected items to see how they fit.

Industrial Plant Maintenance: Maintenance personnel try to find a pipe with a certain functionality among the jungle of existing pipes. The technician sees an adjustable video image of the factory floor on a monitor. Functional data (e.g. labels, measurements such as temperature and pressure) are superimposed on the image to help him identify and inspect the correct pipe.

Road Repair and Maintenance: A repair crew is about to dig a road and wants to avoid hitting major water pipes. The workers aim a camera to the road and get an image of where the main water pipes under the road are. This is accomplished with the aid of an augmented reality system which uses a GPS system and a database of the water pipes in the city.