From presenting information to sharing stories online, multimedia has become the dominant tool of communication on smartphones over the past years. Case in point: the stunning pictures, inventive GIFs, and creative videos produced by Samsung’s new Galaxy S9/S9+’s camera capture emotions that are simply beyond words.
But the Galaxy S9/S9+ is taking it up a notch with an all-new variety of intelligent and personalized emojis. AR Emoji is designed to bring your unique facial expressions and add more fun to the emoji experience. But how did this AR-based feature come to life? Get a look behind Samsung’s lab doors for a close-up view.
Regular emojis add more colour to the conversation and are a fun way to communicate, but there are a lot of times when the existing set of emojis just don’t cut it. These lack personality – one smile looks no different from another. The task was to create emojis that would bring out more character and personality.
The team at Samsung realised that many people use messaging apps to communicate even with people right next to them and rely heavily on emojis rather than text. The focus quickly became how to create an image of the user and deliver multiple facial expressions and emotions along with that.
By leveraging the Galaxy S9/S9+’s camera technology, taking a selfie and transform it into a custom avatar and emoji would add something unique to the fun of communicating. The first step toward this was to use augmented reality to generate an avatar. With facial recognition technology, the Galaxy S9/S9+’s camera starts by producing an initial 3D rendering, using a data-based machine learning algorithm. Next, the camera analyses the 2D image of the user and pulls information from its deep learning database to create the user’s facial features on the final 3D AR Emoji. To bring people’s unique expressions to life, we decided to include Face Tracking Technology, which tracks over 100 facial features and allows us to capture individual emotions on the spot.
In addition to this, Blend Shape technology helps deliver an even greater range of different facial expressions. Based on 75 basic facial expressions, Blend Shape allows us to reflect a wider set of user emotions by blending multiple expressions together. This is also the technology that has made Samsung’s Disney emoji offerings possible.
The end product of this is AR Emoji, which among other features offers a realistic or cartoon-like option for creating your own personalized My Emoji. These can be transformed into a set of 18 different expressions – gradually to expand to 54 – which were carefully created as universal visual representations of human emotions like joy, anger, sorrow, and pleasure.
The idea is to offer emojis that would be able to deliver the same emotions across the world. In addition to the variety of emotions, AR Emoji also comes with a lot of versatility, allowing users to create the 3D avatars, either as a GIF, video or PNG file.
