Model-Free Authoring by Demonstration of Assembly Instructions in Augmented Reality

IEEE Trans Vis Comput Graph. 2022 Nov;28(11):3821-3831. doi: 10.1109/TVCG.2022.3203104. Epub 2022 Oct 21.

Abstract

Among the most compelling applications of Augmented Reality are spatially registered tutorials. The effort of creating such instructions remains one of the obstacles precluding a wider use. We propose a system that is capable of extracting 3D instructions in a completely model-free manner from demonstrations, based on volumetric changes. The instructions are visualised later in an interactive Augmented Reality guidance application, on a mobile head-mounted display. We enable a technology that can be used by anyone in an ad-hoc tabletop setup for assemblies with rigid components.