This is the project that I am currently working on.

Overview:

Team Kalpana is creating an interactive Virtual Reality(VR) experience. Using live action in VR, the team wants the guest to share the space with the characters and interact in the space. The main objective of the team is to explore interactive storytelling and building an effective pipeline for it. Leveraging the immersive power of VR, team Kalpana is designing an experience with the aim of exploring the emotional impact of interactive VR space versus traditional film.

We’ve built 2 interactions in VR as of now, gaze and voice. Using gaze player can select objects in the world and using voice recognition system player can interact with people. We also created our custom plugin in order to playback live action footage in Unity 3D without any frame rate drop. We also integrated google’s speech api into Unity 3D and applied machine learning model in order to extract the emotion and tone from player’s voice.

Development duration: 15 weeks (8 weeks complete)

Technologies used: Unity3D, OpenCV, Google Speech API

My Role: Producer, Programmer in a team of 5 (2 programmer, 1 director, 2 artists)

Development Blog: http://www.etc.cmu.edu/projects/kalpana/index.php/blog/

Project Website: http://www.etc.cmu.edu/projects/kalpana/

Team: