Lumen: Headset-free Mixed Reality Storytelling Platform

Lumen is a storytelling flashlight that creates layers of fiction on top of the objects in your home through projection mapping. It tries to reimagine immersion and provokes a new way of looking at mixed reality without using headsets.

Arvind Sanjeev is an Interaction Designer who recently graduated from Copenhagen Institute of Interaction Design.  He is the developer of Lumen, a Mixed Reality storytelling platform that lets people immerse in alternate realities in their natural space through machine learning and projection mapping technologies.
The project tries to explore the creation of a new kind of media that takes advantage of the physical world while overlaying a layer of digital fiction on top of it. The main inspiration behind Lumen was imagining a future with no more screens and trying to see how people will interact with the environment around them.
Below, Arvind describes the Lumen Project in his own words while exploring how immersive virtual reality experiences can be created while avoiding the isolation of conventional head-mounted displays.

The Lumen Project

The Lumen Project emerged as a part of my 10-week design thesis at CIID. The research phase for Lumen started with the question: “How might we immerse people in alternate reality experiences without isolating them in headsets?”. 
I was trying to imagine different ways in which people can feel immersed in an alternate reality without using headsets. Several sacrificial concepts were sketched out and used for talking with experts in the realm of AR and MR. Key interactions were identified from these story sketches.
Moreover, the contributions from a LARP gamemaster through several interview and prototyping sessions were really helpful for understanding the elements that create believable stories. As part of the research, escape rooms and museums were also studied for understanding the immersion they are capable of creating.
For exploring the interactions, several different working concept prototypes were made such as for storytelling with kids, storytelling in art galleries, storytelling in homes, etc. And after testing out each of these prototypes with people, it was understood that the potential for this kind of tool lies in it being a platform. A storytelling platform that lets people tell their stories through magical interactions with their environment. And that’s how Lumen as a platform was born.
Lumen was also tested with different participants to understand how immersive the key interactions were and was used to learn and iterate.

The Technology Behind Lumen

All popular AR/VR technologies today that lets you immerse relies on headsets or screens, Lumen challenges this trend and explores how people can feel immersed in their natural space by merging bits with atoms.
Lumen uses the Yolo Darknet machine learning platform to classify objects which are then processed by the onboard algorithm that generates stories on top of those classified objects. Narratives for this platform are designed by storytellers and game designers who will have access to the background graphical interface that serves as the story builder and dashboard.
The hardware consists of a laser projector in combination with a camera and depth sensors that work to create the best projection mapping experience for the user.

Leave a Reply