Kinect VR

The Kinect VR project emerged from Art-A-Hack's Summer 2016 program as an exploration into controller-free virtual reality experiences powered by full-body motion tracking. Working with the KinectVR toolkit, Microsoft Kinect sensors, and Samsung Gear VR headsets, our six-member team set out to demonstrate how natural body movement could transform immersive digital environments. The project challenged the conventional reliance on handheld controllers by creating a wireless system where participants' complete skeletal structure was rendered in virtual space, allowing them to navigate and interact with digital worlds using only their physical movements.

As one of six collaborative team members alongside Filip Baba, Loren Abdulezer, Lisa Russell, Pierre Bernard, and QiuYi Wu, I contributed to designing one of the distinct virtual environments featured in the showcase. Each team member created their own unique digital world that participants could explore, demonstrating the versatility of the KinectVR system across different artistic visions and interaction paradigms. This collaborative structure allowed us to showcase a range of possibilities within the same technical framework, from abstract spaces to more representational environments.

My participation in this project centered on pushing the boundaries of embodied interaction in virtual reality. Working with the team, I helped develop and refine the systems that translated raw Kinect sensor data into smooth, natural avatar movements within the virtual environments. This involved grappling with technical challenges like noisy sensor data and the complexities that arose when multiple bodies overlapped in the tracking space, which could cause avatar distortion and affect the user experience.

The project is particularly interesting because it addresses a fundamental question in virtual reality: how do we bring our physical bodies into digital spaces in meaningful ways? By rendering full-body avatars that mirror natural limb movements and enabling multiple players to see each other simultaneously in the same virtual environment, we created a mirrored physical-virtual experience that felt more socially connected than traditional VR interactions. This multi-user capability transformed solitary VR experiences into shared social spaces where physical presence maintained its significance even within digital realms.

The showcase at ThoughtWorks in New York presented a playable prototype that demonstrated these concepts in action, and the team committed to advancing the open-source software for broader developer adoption. This project represents an important moment in exploring how creative technology can dissolve boundaries between physical embodiment and virtual experience, making VR more accessible and intuitive by leveraging the most natural interface we have: our own bodies.