Meet Rephrase.ai - the team behind Mondelez’s AI-led #NotJustACadburyAd that empowers small businesses

A conversation with Ashray Malhotra, co-founder and CEO of the media production platform.

Mondelez India wants to make the festive season a memorable one for small business owners and local stores across the country. So, it has launched the second edition of Cadbury Celebrations’ much appreciated ‘Not Just A Cadbury Ad’. It is a part of the ‘Iss Diwali Aap Kise Khush Karenge?’ campaign.

Conceptualised by two agencies, Ogilvy India and Wavemaker, the campaign was first launched in 2020.

Featuring Bollywood star Shah Rukh Khan (SRK), the current edition takes the campaign up a notch. In addition to the brand films, the local retailers now get to make a version of these ads for their own stores, using the AI-powered technology developed by Rephrase.ai. The initiative aims to help small businesses create unique and scalable avenues of engagement with their consumers.

Companies like Cameo, GoNuts, Tring, etc., are into celebrity video gifting. They help their users send a personalised video message to their loved ones from their favourite celebs.

With technology aggregated to this small idea, Rephrase was able to pull off a similar initiative for Mondelez, but at a scale that’s five times bigger. ‘Not Just A Cadbury Ad 2.0’ is Rephrase’s second engagement with Mondelez.

Co-founded in 2018 by Ashray Malhotra, Shivam Mangla and Nisheeth Lahoti, Bengaluru-based Rephrase.ai is a synthetic media production platform that uses artificial intelligence, or AI, for video personalisation. With the help of its AI technology, Rephrase can clone a celebrity and bring his/her digital avatar to you, similar to what can be seen in these campaigns.

Speaking to afaqs! about the first collaboration with Mondelez for the 2021 Raksha Bandhan campaign, Malhotra, who is also the CEO of Rephrase, says, “We wanted to provide a personalised video that is specially curated for a particular customer, along with every chocolate box. So, we took this idea to Mondelez first. Wavemaker and Ogilvy then came back to us, and we did the Rakhi campaign.”

For this particular (Diwali) campaign, Malhotra informs, the ideation was done by Ogilvy and Wavemaker. Both the agencies have also worked with Rephrase before and, hence, were aware of what their technology is and what they can do for the brand.

“It was a pretty wild concept as it’s the first time that an ad film like this is being edited at such a huge scale in real-time. This was stretching the boundaries of everything that is possible with AI today,” mentions Malhotra.

“The brief that they (the two agencies) gave us was pretty similar and the final video looks like it. There were a few categories, namely grocery, retail, consumer electronics, home décor, clothing, furniture and jewellery, that they wanted us to use Khan and edit his video to match the store name for a particular pin code.”

Machine learning was used to recreate Khan’s voice. As it was impossible to cover all the small stores, the brand gave the power to the people to create their own versions of ‘Not Just A Cadbury Ad’. And with Khan’s personalised message, the store owners can now promote their businesses on their respective social media channels.

All that the store owners need to do to create their own personalised ad film, featuring Khan, is to simply visit notjustacadburyad.com and fill in the form with basic information.

Executing this campaign came with its own set of challenges for Rephrase. The earlier (Raksha Bandhan) campaign was just a product-level integration. You had to purchase a Cadbury chocolate box to access the QR code printed on it. The code had to then be scanned to create the personalised video message.

In the latest campaign, any retailer can go to the website and create a video.

“The first campaign was easier as (actor) Hrithik Roshan was just looking into the camera, while speaking. Whereas, here, Khan is moving and is in different lighting conditions. Technology is sensitive to all this. For us, it was equivalent to doing five projects, as we ended up creating five different avatars of the actor,” reveals Malhotra.

Talking about working on a the second edition of the campaign that was quite successful when it was first launched last year, he says, “When we saw this campaign last year, our technology was still nascent, but we had a vision that the right way of doing it is by creating personalised videos. That is when a brand truly creates an ad for a local store.”

Taking us through the process of executing this campaign, Malhotra shares, “This was the most technically advanced project that has been done. First, we had to recreate Khan actually moving around (and not static), while speaking to the camera. The technology can primarily redo a certain variety of movements, but not that much.”

“Second, this was not shot in a studio, but a set. So, we had to deal with the background noise and get voice cloning models to work with a small amount of data that was available. We worked with 15 minutes of voice data recorded in a set, and not a soundproof studio. The voice aspect of it was really challenging.”

As the ad is a mix of Hindi and English, it was important for AI to get the (bilingual) pronunciations correct. For example, most Indian names have to be pronounced in Hindi, while the other words in the sentence are in English. It was quite a challenge for Rephrase.

“For every single set, the actor’s voice was slightly different and we had to create different voice models for each specific location,” states Malhotra.

According to him, the campaign has received a much better response than what they had expected. With the recent controversy surrounding Khan, the response could have gone either way. People could have even boycotted it.

Says Malhotra, “I wouldn’t say that we completely blocked the entire controversy in our heads, but we didn’t give it a serious thought because it’s something beyond our control. The two agencies and Mondelez were in a better position to figure things out.”

Rephrase has also done a campaign with Colors and actor Ranveer Singh in the past. It also works with companies to create digital avatars of their CXOs and senior leaders to help them communicate with their employees directly.

Malhotra states that going forward, we will see this technology become mainstream, as it’s changing the definition of having a brand ambassador.

“This has already evolved from just onboarding a celebrity for TV ads, to what digital media has to offer now. Hyper-personalised content creation with brand ambassadors is going to be Digital 2.0,” Malhotra signs off.