Dr. Jyoti Joshi is the founder and CEO of Kroop.ai. Kroop.ai is a deep tech startup that has developed a sophisticated audio-visual deep learning platform. She is a first-generation entrepreneur and an AI scientist with a PhD from the University of Canberra, Australia. She worked for mental health analysis and put in stints at the University of Waterloo, Canada and Monash University, Australia.
Can you tell us about your AI journey?
My first rendezvous with AI was at the University of Canberra (UC) with Prof. Roland Goecke. I worked as a research assistant on sports data. To formally learn the nuances of pursuing research, I started my PhD in computer vision and speech analysis with Prof. Roland Goecke and proposed multimodal frameworks for unipolar depression analysis. During my PhD, I was fortunate to visit the Queen Mary University of London and the University of Pittsburgh and experience different AI-centric research environments. Later I moved to Canada to pursue a postdoc at the University of Waterloo with Prof. Jesse Hoey. I worked on affect analysis to achieve an effective human-computer interaction using digital avatars. I also worked at Monash University and ANU and later moved to the industry through an entrepreneurship venture. The experience at Canberra and Waterloo played a seminal role for me in co-founding Kroop AI.
What is Kroop AI?
Kroop AI is a deep tech startup working in the space of automatic facial and voice animation. In the form of The Artiste AI framework, we offer an easy-to-integrate API and a cloud-based interface. Using our platform talking, avatar-based videos are generated with just audio or text input. Using our API, 2D or 3D facial characters in Metaverse or games can be animated within seconds. We have also made deep strides in more human and natural-sounding text-to-speech and voice cloning, allowing avatar-specific voices accompanied by high-quality facial movements.
Our cloud-based editor, The Artiste AI Studio, can be accessed at https://studio.theartiste.ai/
The AI-based platform finds strong application in lip-sync correction, dubbing, hyper-personalised marketing and gaming/Metaverse.
In your opinion, how will AI affect the future of the film and entertainment industry?
There is a significant increase in video consumption due to social networking and OTT platforms. A considerable amount of video data consumed is multi-lingual, typically dubbed across a different language. Dubbing is an expensive and time-consuming exercise. Progress in computer vision and deep learning is now enabling faster dubbing. Our Artiste AI API is an example of that.
When dubbed from one language to another, many movies still lack a proper lip-sync at many points. This lack of lip-syncing not only affects the engagement and immersive experience of the user but also affects viewers with hearing disabilities. But, again, AI, in the form of machine learning, computer vision and signal processing, can correct the lip-sync. At Kroop AI, we automatically fix lip-sync issues faster and help increase the footprint of the content as our technology is language agnostic.
We now see that AI-based frameworks are both expediting works in pre-and post-production in movies and TV programs. In the very near future, actors’ voices will be heard in different languages as if the actor originally spoke in a foreign language. This is achieved with training data of about 20 minutes in its current form. This requirement is going to go drastically down in the coming months.
Why is it essential to detect deep fakes and present evidence as a valuable tool?
A primary concern for content creators is the misuse of their audio-visual creations by illegally copying content. Using a deep fake detector, content creators can validate the authenticity of the content and check if any manipulation has been done to the audio or video signal. At Kroop AI, we see it as our responsibility to offer an ethical data generation platform. Therefore, we provide an audio-visual deep fakes detection API for anyone to validate content.
Can you explain your experience at the Cannes Film Festival?
The overall experience at Cannes was phenomenal. The energy at the India pavilion and the passion of fellow founders to showcase their ideas was amazing. This event provided me with a huge platform to network with content creators, filmmakers, directors and metaverse experts. They were highly impressed with the product and tech we offer as it can significantly reduce cost and help produce content in various languages with proper lip-sync much faster.
What are the significant challenges you faced as a woman in reaching where you are right now?
Women have been underrepresented in AI and STEM, sometimes posing barriers to growth. I remember starting a support network during my graduation days at University to expose women to like-minded peers. The idea was to get connected with others who have been struggling with similar issues and learn from their experiences how they have been able to overcome challenges. My professional journey has been no different from any working mother. Little kids always struggle, which affects how one can maintain a healthy work-life balance. For me, prioritising some day’s work takes precedence and vice versa.
What’s your advice for other women who want to pursue a similar journey?
Everyone has their unique way of dealing with challenges. For me planning and being persistent have worked. Clear communication with teammates also helps. I have been fortunate to have understanding teammates at Kroop AI. We have established a culture of equality. We are striving to balance empathy and objectiveness.
Source: indiaai.gov.in
