Collider-based movement detection and control of wearable soft robots for visually augmenting dance performance

Abstract

The fusion of wearable soft robotic actuators and motion-tracking sensors can enhance dance performance, amplifying its visual language and communicative potential. However, the intricate and unpredictable nature of improvisational dance poses unique challenges for existing motion-tracking methods, underscoring the need for more adaptable solutions. Conventional methods such as optical tracking face limitations due to limb occlusion. The use of inertial measurement units (IMUs) can alleviate some of these challenges; however, their movement detection algorithms are complex and often based on fixed thresholds. Additionally, machine learning algorithms are unsuitable for detecting the arbitrary motion of improvisational dancers due to the non-repetitive and unique nature of their movements, resulting in limited available training data. To address these challenges, we introduce a collider-based movement detection algorithm. Colliders are modeled as virtual mass-spring-damper systems with its response related to dynamics of limb segments. Individual colliders are defined in planes corresponding to the limbs' degrees of freedom. The system responses of these colliders relate to limb dynamics and can be used to quantify dynamic movements such as jab as demonstrated herein. One key advantage of collider dynamics is their ability to capture complex limb movements in their relative frame, as opposed to the global frame, thus avoiding drift issues common with IMUs. Additionally, we propose a simplified movement detection scheme based on individual dynamic system response variable, as opposed to fixed thresholds that consider multiple variables simultaneously (i.e., displacement, velocity, and acceleration). Our approach combines the collider-based algorithm with a hashing method to design a robust and high-speed detection algorithm for improvised dance motions. Experimental results demonstrate that our algorithm effectively detects improvisational dance movements, allowing control of wearable, origami-based soft actuators that can change size and lighting based on detected movements. This innovative method allows dancers to trigger events on stage, creating a unique organic aesthetics that seamlessly integrates technology with spontaneous movements. Our research highlights how this approach not only enriches dance performances by blending tradition and innovation but also enhances the expressive capabilities of dance, demonstrating the potential for technology to elevate and augment this art form.

Keywords: activity recognition; colliders; dance; inertial measurement units (IMUs); movement detection; soft robots; wearable sensors.