A former product manager who has worked with Microsoft, Snap and Waymo, entered a hackathon in May this year. The solo woman hacker won against hundreds of other hackers where Replit CEO, Amjad Masad, was one of the judges. “If you’re going to start a company, I’m going to invest in you,” assured Masad. However, having a full time job and not really confident of starting a company at that time, Priyaa K, went about her work. 

After a few months, she quit her job and joined South Park Commons (community of technologists and builders), and the word got out. Masad reached out – “I heard you quit your job. So, when are you going to raise money?” he asked. Thus, Lica began. 

Companies such as Midjourney, Runway and the latest Pika, that employ generative AI to transform text to image/video have been built for a specific niche where creators use these tools for the creative/movie industry alone. Furthermore, the generation is from a pure text prompt. 

However, a storytelling platform for all modes of writing was missing, and a couple of Indian-origin women were on a mission to change that.  

Priyaa K and Purvanshi Mehta have been working on a project called Lica, a platform that works on converting every form of writing such as documents, presentations, emails, and many others, into captivating videos. 

“We are building something where video storytelling which is the most effective form of storytelling that can be democratised for everyday people, who don’t have access to the most powerful video editors or even have the knowledge to be able to use them effectively,” said Priyaa K, co-founder of Lica World, an AI startup in San Francisco, in an exclusive interaction with AIM. 

Need for Lica

“Lights, Camera, Action,” or Lica, arose from the need to address common modes of office communication, such as powerpoint and other office products, that were built in the 90s and early 2000s and have not evolved since. Priyaa believes that every time a story needs to be told, for example, a developer presenting a technical documentation, or a journalist writing a brief for their executive team, they are constrained within the paradigm of a tool. 

“We realised video storytelling, specifically interactive videos, are really going to be a powerful tool for storytelling because you can embed more information within a video than in text. You can encode more things more succinctly in a video because it’s visual and it’s the closest proxy you have to watch something live in front of you. So that’s why we chose videos,” said Priyaa. 

Furthermore, with a number of large language models and research teams working on similar platforms of text-to-video, there still exists a big market gap as Priyaa believes that a person needs an interface to connect with all these AI models because they are all “disparate models in disparate silos in disparate applications.” 

Just like how people use powerpoint templates created by someone, Priyaa firmly believes that people will use design agents created by developers, and that these design agents will be able to generate visuals, graphics, audio, music and that there would be an orchestration layer to put these things together. “We are building an LLM that can predict the design action that a user needs to get to a certain output, and that output today is an interactive video. 

A Mix of Models

“We use a couple of modules, and what we are trying to build is an orchestrator module, which would be our proprietary model with IP there. Its job is to coordinate which downstream model to call. It would be a multimodal model, which understands how the downstream models perform. So, the orchestrator model will be taking all these design actions on its own. So it would be an action transformer kind of model,” said Mehta, co-founder of Lica. 

The models will involve different levels of human intervention which will allow a person to create customised video depending on the occasion, and even fine tune at various stages to change  background, voiceover sound, and many more as per one’s directorial style. Currently, they are working on integrating a number of downstream models such as lip sync models, video generation models, voice cloning models, music generation models, and others. “We use GPT-4 for API’s, but soon we want to have our own in-house models which are open source, and fine tune them for some of the screenplay generation that we are doing,” said Mehta. 

Journey to Effective Communication

The startup is not only backed by Amjad Masa, but also Replit VP of AI Michele Catasta, and Village Global, a venture capital firm chaired by Reid Hoffman and backed by Jeff Bezos, Bill Gates, Mark Zuckerberg and others. Lica recently closed a pre-seed round of funding. 

After winning the hackathon where Priyaa met Amjad, he tweeted about the same which went viral. VC Vinod Khosla recently referred to that tweet while talking about AI predictions. 

Amjad’s tweet after Priyaa’s win at the hackathon. Source: X

Combining Forces

At Microsoft, Priyaa had worked with the designer team to make presentation, design and automation possible, and also started Microsoft Designer which used to look different back then. “After working there for a couple of years, I got a lot of faith and I could see a lot of feedback from users, like no matter how cool AI is, it’s always five steps behind what’s there on TikTok or Instagram, where consumers enjoy a form of communication that business users don’t,” said Priyaa.

Mehta comes from a machine learning and applied research background of building large language models, or rather ‘the intersection of graph intelligence and multimodal models,’ and has also worked with Microsoft, building features for Microsoft 365 suite. “We kind of joined forces together to build Lica where anyone can create videos in AI,” said Priyaa. 

Stemmed from the need to effectively build communication between audiences, such as teachers who want to find ways to engage with their students who liked TikTok and not powerpoint, Priyaa always had the idea of building agents for video. Playing on the idea of how to tell an effective story and having been a public speaker and participating in competitive debating throughout school and college, Priyaa has learned how to tell stories. She also believes that anyone can tell a good story, but tools are the limitation. 

Still in the experiment phase, the beta programme is expected to start in January 2024 as a gradual rollout with specific users for feedback. In the future, Lica looks to probably give access to companies that are brand compliant, to put up their own models on the platform. “So, today any of the independent tools in the market could become an agent on our platform, because we are more interested in the infrastructure problem than specifically solving one vertically integrated problem that one particular industry has got,” said Priyaa. 

The post Lights Camera Action: These Two Indian Womenpreneurs Reinvent Text-to-Video AI appeared first on Analytics India Magazine.