{"id":209719,"date":"2021-03-24T16:00:01","date_gmt":"2021-03-24T13:00:01","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/mixed-reality-gets-a-machine-learning-upgrade\/"},"modified":"2021-03-24T16:00:01","modified_gmt":"2021-03-24T13:00:01","slug":"mixed-reality-gets-a-machine-learning-upgrade","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/mixed-reality-gets-a-machine-learning-upgrade\/","title":{"rendered":"#Mixed reality gets a machine learning upgrade"},"content":{"rendered":"

“#Mixed reality gets a machine learning upgrade<\/strong>”<\/p>\n

\n
\n
\n
\"Mixed
\n Fig. 1 Proposed MR system: Integration of semantic segmentation into MR. Credit: Osaka University
\n <\/figcaption><\/figure>\n<\/div>\n<\/div>\n

Scientists from the Division of Sustainable Energy and Environmental Engineering at Osaka University employed deep learning artificial intelligence to improve mobile mixed reality generation. They found that occluding objects recognized by the algorithm could be dynamically removed using a video game<\/a> engine. This work may lead to a revolution in green architecture and city revitalization.<\/p>\n

Mixed reality (MR) is a type of visual augmentation in which real-time images of existing objects or landscapes can be digitally altered. As anyone who has played Pok\u00e9mon Go! or similar games knows, looking at a smartphone screen can feel almost like magic when characters app<\/a>ear alongside real landmarks. This approach can be applied for more serious undertakings as well, such as visualizing what a new building will look like once the existing structure is removed and trees added. However, this kind of digital erasure was thought to be too computationally intensive to generate in real time on a mobile device.<\/p>\n

Now, researchers at Osaka University have demonstrated a new system that can construct a MR landscape visualization faster with the help of deep learning. The key is to train the algorithm with thousands of labeled images so that it can more quickly identify occlusions, like walls and fences. This allows for the automatic “semantic segmentation” of the view into elements to be kept and others to be masked. The program also quantitatively measured the Green View Index (GVI), which is the fraction of greenery areas including plants and trees in a person’s visual field, in either the current or proposed layout. “We were able to implement both dynamic occlusion and Green View Index estimation in our mixed reality viewer,” corresponding author Tomohiro Fukuda says.<\/p>\n

\n
\n
\"Mixed
\n Fig. 2 MR-based landscape visualization with dynamic occlusion handling in field validation. Credit: Osaka University
\n <\/figcaption><\/figure>\n<\/div>\n<\/div>\n

Live video is sent to a semantic segmentation server, and the result is used to render the final view with a game engine on the mobile device. Proposed structures and greenery can be shown even when the viewing angle is changed. “Internet speed and latency were evaluated to ensure real-time MR rendering,” first author Daiki Kido explains. The team hopes this research will help stakeholders understand the importance of GVI on urban planning.<\/p>\n

\n
\n
\"Mixed
\n Fig. 3 Planting simulation with MR and greenery estimation. Credit: Osaka University
\n <\/figcaption><\/figure>\n<\/div>\n<\/div>\n\n
\n
\n More information:<\/strong>
\n Daiki Kido et al. Assessing future landscapes using enhanced mixed reality with semantic segmentation by deep learning, Advanced Engineering Informatics<\/i> (2021). DOI: 10.1016\/j.aei.2021.101281<\/a><\/p><\/div>\n
\n Provided by
\n Osaka University
\n
\n <\/svg><\/a><\/p><\/div>\n

<\/p>\n

\n

Citation<\/strong>:
\n Mixed reality gets a machine learning upgrade (2021, March 24)
\n retrieved 24 March 2021
\n from https:\/\/techxplore.com\/news<\/a>\/2021-03-reality-machine.html<\/p>\n

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
\n part may be reproduced without the written permission. The content is provided for information purposes only.<\/p><\/div>\n<\/p><\/div>\n