Enhancing the Customer Experience by Mixed Reality in the Retail Industry

Abstract

Nowadays, customization by mixed reality to enhance the customer experience plays an important role in the retail industry. Customers can choose and customize products with their images and labels in a virtual reality environment. However, the existing asset creation pipelines are labor-intensive and time-consuming to display the images and labels (aka logos) on 3D product models, and cannot be easily customized by customers in real-time. In this paper, we thus propose a real-time 3D logo mapping framework for converting 3D logo mesh from a specified image and fitting it to the 3D product models. In the framework, Convolutional Neural Network (CNN) is adopted to reconstruct 3D logo/product models from their images. The detailed 3D information and the logo location provided by a customer are used to select the effective sampling points to mesh deformation. This method can preserve both the visual quality and details of 3D product models. Experimental results, carried out on various sizes of logos and types of products, show that our method can produce accurately and quickly customized logos on 3D product models.