I’ve been doing a lot of R&D for my thesis ‘Discerning the Natural from Simulation in Extended Reality Applications” and decided to make an AR app to demonstrate some of my findings. I used photogrammetry scans of fruit and lit everything with an HDRI in Unity to attempt to keep the textures and objects looking realistic. I used animations (some vertex, some baked from Cinema4D) to distort the fruit and break the illusion. Audio was a large part of the experience so I implemented spatial 3D audio using the Resonance SDK to place sounds in 3D space as well as move it around. When using the AR app, where the user moves effects the sound just like the real world.
For narration and VO’s, I wanted character with a subtle attitude. I built a demo using IBM Watson but their service failed and I ditched it for good ole reliable Amazon Web Services. All the voices are AI using machine learning and Speech Synthesis Markup Language (SSML) to modify the speech patterns, pacing and distortions. I branded the concept ‘Xscape’ for the Computer Science Showcase as well of ease of communication. All this ‘XR’ nomenclature made me think that using an ‘X’ in front of my apps may be a clever idea.
The video below was captured from an iPad.