Augmented reality refers to the utilization of computers to modify reality, usually in order to provide additional assistance when a human user is interacting with the real environment. The real world around humans provides a wealth of information which the human user must absorb and process through their senses. The most useful and informative of all human senses is the sense of vision and a huge amount of information about the ambiance is required to be sensed as well as processed by the human visual system. Computers are useful because they can provide an overlay of information to assist with the human processing of the information which they perceive through their senses, mostly the visual sense. As an example, a human who is walking around an urban area can be provided additional information through a head mounted display which superimposes textual information about identification of buildings and other important landmarks so that the human is provided additional guidance about identification of buildings and landmarks or other useful information about what they are viewing. Obviously, the computer too has to sense the environment in order to provide any additional information and this is done through a camera that captures the same view as the human. The camera provides images which are identified by a computer and virtual scenes are created in the virtual world generated by a computer. The computer program then generates additional assistive information that is presented to the human user in order to assist them in their interactions with their environment. Augmented reality software program is required to process information which a computer senses through sensors and the real space is transformed into a virtual space within the computer, with computer software identifying, recognizing and transforming the external inputs from the real world in order to transform these inputs into assistive outputs for the user of an augmented reality system. Augmented reality can, therefore, be considered to be a variation of virtual reality which completely immerses a user inside a synthetic environment. Augmented reality makes it possible for a human user to view the world with virtual objects that are generated by a computer and then superimposed or merged with it. The characteristics of virtual reality systems include a combination of the real and the virtual, interactions in real time and registration in 3-D.
Operation of an Augmented Reality System
Augmented reality systems usually add to the information that is received from the real world. In augmented reality vision systems, objects from the virtual world generated by a computer can be added to the scene from the real world. However, augmented reality systems can also remove sensations from the real world and as an example; it is possible to remove a table which a human using the augmented reality system may be viewing as a result of the computer painting it over. Although augmented reality usually refers to a computer modifying the real information that is presented to the human vision system, vision is not the only sense which a computer can interfere with. A system in which multi – sensory input is provided may turn on a heat lamp when a user approaches a spot in the virtual environment that is exposed to the sun or the smell of roses may be directed to a user when they approach a rose garden in mixed environment. A computer can also direct that a fan be switched on in order to provide a flow of air to a user when they approach a spot in the virtual environment that is exposed to wind. Thus, augmented reality involves modification of reality that is presented to a user by a computer which also senses reality and which has created a virtual model based on what has been sensed. Examples of the application of augmented reality include superimposition of internal information over external surfaces or the augmentation of viewed environment with informational labels.
When thinking about augmented reality, it is appropriate to think about the physical space, the virtual space and the measured space. The concept of physical and virtual space should be clear, however the concept of measured space refers to the representation of the physical space inside a computer and the manner in which sensory data is used to present the real space to a user. The generation of augmented and virtual spaces require the joining together or overlapping of the space maps associated with the physical and the virtual spaces, with the interaction being managed by computers and people using their sensory capabilities. Virtual space does not necessarily represent the physical space and this space is a model of the physical space that is created the real world and its views as perceived by sensors. Virtual space contains inputs from the physical space which are incorporated into the electronic space by the computer. The generation of augmented reality involves superimposing electronic generated space on to the real objects and space. The virtual space inside a computer must correspond to the real space which has to be mapped into the computer using some sort of a global reference such as a global positioning system.
Relative measurements of objects in physical space are transferred to the computer so that the electronic space may be generated, even though it may not be known where the physical space is with regard to the rest of the universe. Thus, a car’s interior dimensions are relative to each other, although it may not be known where the car is in the universe. A close mapping may exist between the physical and the virtual space if the electronic space is required to be a close representation of the physical space. It may be required to represent different parameters from the real space into a virtual space, such as the variation of light intensity or temperature. Some event in the physical space may also be required to generate a trigger and some how alter the electronic virtual space. Different types of sensors may be used to provide information about the physical space such as the dimensions of the physical space being measured by ultrasound or sonar. Sensor inputs are required to be fed into a computer, processed and then used to present a reflection in the electronic space. Triggers from the physical space are also measured by sensors and as an example; the presence of a human object in an area of the physical space may be registered by a capacitive sensor which will transmit this information into the computer. The computer that is being used may then take some action to process this trigger and generate some outputs or variations in the electronic space. Virtual spaces that are generated inside a computer after some processing of the data related to physical spaces are translated and made available in the real world by projection involving video, audio, tactile or hepatic devices or even by using the sense of smell. The virtual world of the computer is projected at some point or location in the physical world which is known as the point of projection. Device such as a screen, a virtual reality goggle or PDA etc are used for such projections. Projections of the virtual world into the real world through devices present users with an illusion of occupying some part of the physical space such as the space behind a projection screen or in front of a holographic plate. In purely virtual reality environments, the sensing of an object from the physical space may have an impact on the projection of the virtual space into the physical space, however, the object that is being sensed will not be a part of the virtual space. In mixed or augmented spaces, however, the sensed object will be incorporated into the virtual space and hence the link between the object that is being sensed and the projection is important. Location in space is usually measured in terms of some sort of coordinates which could be Euclidean coordinates such as polar, spherical or ordinance grid coordinates or alternatively, the location may be relative to a zone with objects being detected as being in a zone such as a room or a part of a room. Information about position may also be relational with some object being detected as being close to another object. The level of accuracy with which measurements in the physical space need to be made and information in the virtual space need to be displayed will vary for different applications, but the requirements related to accuracy will determine the amount of data that needs to be exchanged between the physical and virtual spaces.
As an example, head mounted displays and cameras that are mounted on these displays can be used to present computer modified reality to a user. The cameras sense reality and feed a video signal to a computer which is then modified by the computer according to the programmed algorithms for the generation of virtual space involving identification of objects, addition of data to images, image manipulation or object cancellation etc. The desired additions or modifications to the human view of reality are then projected on to the human eye through head mounted monitors or optical diversion and mixing of the real and virtual space.
Apart from optical see – through augmented reality displays, it is also possible to generate augmented reality using monitors and video see-through displays. Optical see-through displays in which the real world images are mixed with the virtual reality images that are generated by a computer using optical mixing are different from video see-through displays which project images that are desired to be seen by a human user on to the eye using a video display without any optical mixing from the real world.
User Benefits of an Augmented Reality System
Augmented reality systems attempt to present a world which consists of a combination of computer generated imagery or other sensory inputs and real world perceptions that are available to a human user. Thus, augmented reality systems can present an enhanced view of the world to the users and the enhancements provide additional information which can be of benefit to the user in a variety of ways. The additional information which can be superimposed on the perception of a user can take a variety of forms depending on the application or the intended use of the augmented reality application. As an example, an augmented reality system may be designed to superimpose a pipeline system which has been designed for an industrial setting, or the system may assist a surgeon by superimposing internal imagery of a patient’s diseased organs on the image of their exterior anatomy, assisting the surgeon to perform surgery. All augmented reality systems assist humans to perform a task by enabling the human user to visualise, readily access additional information or to superimpose objects that are not visible on to real views. Augmented reality systems can enhance human understanding of what they are able to perceive and thus humans are assisted in solving problems which may be difficult if they were not provided any additional pieces of information. However, the applications of augmented reality are very broad and this technology has proven to be useful in very many applications. Thus, augmented reality may be used for entertainment or gaming, providing additional input on views of sporting events as well as assisting humans in more serious endeavours such as the battlefield, archaeology or architecture or urban design etc. Objects that are superimposed on real world views using augmented reality may be required to be presented in correct perspective depth as well as being accurately positioned with respect to other real or virtual objects and this can assist in human users being able to perform delicate work due to the additional understanding that they are able to acquire as a result of using augmented reality systems. Miniaturization of computing elements and advances in wireless as well as general technological advances on a broad front have made it possible for augmented reality systems to be miniaturized and to be made wearable or portable, adding to their general usefulness both indoors and outdoors. Thus, augmented reality systems are able to assist, entertain, inform or aid humans by enhancing their perceptions in a wide variety of applications.
However, virtual reality technology is still evolving and with advances in computer science, even greater sophistication is likely to be possible in a large number of applications. Advances in wearable computing, software, miniaturisation, display technologies as well as sensors and radio networking have had a profound impact on the application of augmented reality. Augmented reality systems are now far less bulky, portable and can be worn by a human user who may want to be assisted in a variety of ways in order to perform a task. Systems making use of augmented reality are now often used outdoors in harsh weather conditions. Despite the possibilities, virtual reality implementations do have some limitations as well as difficulties with implementation. Registration errors or problems associated with sensing or bandwidth of the display devices can cause deterioration in the performance of augmented reality systems. Objects in the real and the virtual world have to be aligned correctly in relation to each other otherwise the illusion which is created as a result of the two worlds coming together will not be comfortable for humans. Sensing devices which provide data inputs for the augmented reality computer have to be able to track the field of vision of a user. Sensing errors, mechanical alignments, incorrect viewing parameters such as field of view or tracker –to- eye position or orientation as well as optical distortion in the display system are some of the problems which can affect the performance of an augmented reality system.