Virtual Public Sector Reality – An Overview of Immersive Use Cases

Suddenly regaining sight – an unimaginable prospect for many people who suffer from visual impairment or blindness. But, with complex image processing algorithms and a head-mounted display, this is indeed possible¹. Technologies related to Mixed Reality and co. are already being used in many fields. Things can not only be made to “come alive” in the gaming and entertainment industry, or in marketing, but also in aviation, for the training of pilots, in the education and publishing industry for displaying the content of books, or in medicine, in operations, and also to provide life-changing support regarding the visual perception of individuals who suffer from visual impairment.

MR meets AI

Due to the blending of Mixed Reality technologies with functions of artificial intelligence, many factors play a role in the user experience. How an application is designed can have a huge effect on a more intensive user experience, as visual and haptic impulses are supplemented with acoustic functions. The interaction with the application thus reaches a new level of user experience. The resulting advantages can be loosely summarised as follows:

• Fast transmission of content (Time-to-Content) and information processing in real time
• Sensory stimulation: Simultaneously addressing different senses that sustainably support communication processes and structures
• Stimulation of user activities (e.g. through curiosity, longer retention times, etc.)
• Increasing the user experience through moments that “come alive”
• Multitasking
• Collision detection of real and virtual objects

The possible applications for Mixed Reality are limitless. Particularly in connection with artificial intelligence and in the context of the recognition of people, objects, or language, it is not just the user experience that is being taken to the next level, but more and more Use Cases are also being facilitated. With enormous amounts of data and new analysis approaches, algorithms that explain behaviours can enable a multitude of new business models in the virtual world in the future.

Image analysis / image recognition: Information can be filtered out of images with the assistance of image analysis or image recognition. Text and content from images can be understood and categorised, and individual objects or faces can be filtered out and recognised.

Face recognition / facial expressions / gestures: The face recognition function typically enables the recognition and analysis, as well as the comparing and grouping, of faces in photos. Emotions can be recognised through facial expressions and gestures.

Voice control / speech recognition: Often also called text-to-speech or speech-to-text, this function enables the evaluation and conversion of natural language. When processing natural languages, commands can be understood, speakers can be identified, or language can be converted into text format.

Thanks to new hardware, artificial intelligence, and the increasing use of smartphones, technologies have been enjoying a boom in all areas for some years now. New Use Cases and the growing availability of well-developed input devices are spurring on the market. In the public sector, however, it still seems to be a niche issue. However, there is still a lot of potential in this area.

But where are the opportunities that will make the public sector fit for the future with Mixed Reality technologies? And what exactly does Mixed Reality mean?

Virtual Value for the Public Sector

The question also arises for CIOs and IT professionals in the public sector as to which initiatives should be taken and which technologies and platforms should be invested in if they want to strategically advance digital applications with Mixed Reality.

In addition to spatial and environmental planning, e.g. for museums, sports facilities, neighbourhood centres, and playgrounds, displays and Smart Glasses can also facilitate decision-making in urban planning tasks. To master the challenges that come with the overhaul of the infrastructure offering and to consider the equality of all those involved in the entire planning process, planning innovations are required. Mixed Reality provides the opportunity for so-called Gender Planning to visualise different scenarios or different life situations and interests, and to thus assist in decision-making, e.g. regarding streetlights.

Nowadays, museums in particular struggle to give due consideration to the zeitgeist and to design exhibitions in an appealing way. To actively involve visitors and give them the opportunity to become a part of the exhibition, digital concepts create a new way of conveying content and self-exploration. With augmented reality technology, museums can relay additional information content, meanings, and contexts to their visitors that go beyond the visual perception of pure reality. For example, visual cross-fading could offer an insight into the past or perspectives that would otherwise be hidden. Mixed Reality technologies, such as augmented or virtual Reality, are also increasingly influencing the tourism landscape. Travel and tourism applications that enhance the travel experience are becoming more and more important. To this end, tourism associations, municipalities, and cities can, for example, make it easier for visitors to navigate on-site and use cross-fading to present information on sights, restaurants, or shops in the vicinity. Moreover, Mixed Reality devices equipped with GPS could also capture landscapes and visualise geographic information at any location, such as property boundaries, tunnels, electricity, water, and gas lines, and much more from other databases. For land registry offices and land surveying offices in particular, Mixed Reality technologies could be used to access a detailed and complete image of the earth in the form of a 3D cadastre in the future. All digitally available construction conditions or property security can expedite the application process and create a uniform basis for all authorities, from the agency for technical relief to the Federal Office for Water Management or the German Federal Office for Hydraulic Engineering. These functions can also simplify the inventory and shelf arrangement in libraries. However, for every new technology, one has to ask oneself what the purpose of the application will be within the company. The reason the initiative still fails today is mainly the entry hurdles related to costs and, in some cases, technical shortcomings regarding the purpose of the application. Many potential users still only have a very vague idea of how they can benefit from the possibilities and the use of technologies within their own company, or how to even understand them.

From Reality to Virtuality

With the assistance of user interfaces, such as special data glasses, smartphones, or sensors, etc., users can be sent into varying depths of a fictional reality. The extent to which a user is immersed into this or the extent to which a user perceives virtuality as reality is referred to as the degree of immersion. The maximum degree is reached when the user feels completely cut off from the real world. The area of application or the context-dependent provision of information defines the relationship to the real world and, depending on the reality technology, is reinforced by the real time processing of the information. While virtual reality describes a completely virtual world or a world simulated in real time that is mapped with the assistance of headsets or glasses and seen by users as the real perception, augmented reality is a system or an environment in which the real world is enriched or superimposed with digital information.

The term Mixed Reality refers to the mixing of the real world and the virtual world, and does not include any technology of its own. Mixed Reality solutions move along the two end points and contain those scenarios in which both environments are combined. Mixed Reality is the most suitable variant for business and the most flexible form, as it can include a particularly large number of virtual objects and environments without losing touch with reality.

However, it is not easy to clearly distinguish between virtual reality and Mixed Reality in practice. The lines are blurred, in particular due to the available hardware. There are VR glasses with tracking systems that record the spatial relationships; this integrates part of the real environment and models Mixed Reality. Other tracking technologies, on the other hand, only measure the change in position of the head, but not the position of objects in the user’s surroundings. Thus, there is no reference to the real world and consequently to Mixed Reality.

As a partner and expert in Mixed Reality technologies, Cloudflight works with you to develop applications that digitally map your company’s business processes according to your individual needs and take them to a new level of immersion. Sign up here.

¹ https://business.esa.int/news/satellite-image-processing-gives-sight-to-visually-impaired