LetsSignTalk, developed by an Indian startup called DeepTechVision.AI, is an artificial Intelligence-powered, device-agnostic solution that enables easy two-way communication for people with hearing/speech impairment.

Hearing loss may be on the rise, as per some estimates by WHO, and more people are increasingly using sign language to communicate. In 2018, more than 466 million of the world’s population (6.1% people) suffered from some form of hearing impairment. This could rise to 900 million by 2050, with as many as 1 people affected in every 10. Even closer home, close to 10% (~100m people) suffers from some level of functional hearing loss in India.

Affected people face numerous challenges in their day-to-day lives. In some cases, it also includes denial of their right to education because of the lack of schools teaching in sign languages. Research says that more than 90% of hearing-impaired people are either uneducated or school drop-outs. They may also have to deal with a massive lack of job opportunities, as well as social and workplace isolation. 

How Can Technology Help?

According to experts, speech solutions available today for hearing/speech impaired communities are incomplete, non-scalable, and dependent on special hardware/software, which is inconvenient for day-to-day use with ease. Also, the existing systems do not provide any customisation features like creating a person’s digital voice for reading aloud interpreted text and creating an animated avatar from voice description for displaying sign gestures.

Even if few systems did not require additional hardware for sign language recognition, they fall short of detection accuracy or use techniques which cannot scale for multiple sign languages like Indian Sign Language, American Sign Language, etc. or variations of sign languages like finger-spell or word gestures or phrase gestures. Moreover, these systems require internet connection or backend services for interpretation.

This is where LetsTalkSign platform can add tremendous value by bringing to patients a comprehensive tool to communicate, and break the barrier between hearing/speech impaired and normal people LetsTalkSign has been developed by DeepVisionTech.AI, a budding startup which specializes in machine learning technology and helps enterprises build smart and intelligent solutions. The company was conceptualized in late 2018 and established in mid-2019 with a handful of solutions for a variety of verticals like disability (hearing/speech impairment), agriculture (remote sensing for crop health monitoring), entertainment (gestures s a new user interface for apps & games) and healthcare (identifying diabetic retinopathy from OCT scans). Its patented computer vision-based solutions and techniques are targeted at giving businesses a competitive advantage.

The Solution

To solve the issues that we see above, the founder of DeepVisionTech.AI, Jayasudan Munsamy described the tech behind LetsTalkSign. The solution leverages artificial Intelligence-powered device-agnostic platform that enables easy two-way communication between people with hearing/speech impairment who use sign language for communication and others who do not know sign language.

Some of the techniques used are –

1) Computer Vision (CV) for video/image recognition to identify sign language signs/gestures.

2) Generative Adversarial Networks (GANs) for speech/text to sign language translation and synthetic voice creation.

3) Natural Language Processing (NLP) to form sentences post interpretation of sign language signs.

The solution is device-agnostic meaning it can be used across smartphone, tablet, laptop, PC, TV, Raspberry Pi, etc. The coolest thing is that inference happens on device in real-time, and easily extendable to interpret other sign languages. According to Jayasudan, it also supports customizable vocabulary to support multiple domains and industry verticals and no additional hardware is required. Just a camera and minimal processing power are needed to make it work. No internet required as all the capabilities are built-in the app.

“LetsTalkSign is the proof that ‘AI for Accessibility’ isn’t just a slogan but a reality…we are here to leverage the power of technology to empower hearing/speech impaired communities and transform their lives,” told Jayasudan.

So, can it really help deaf patients? We interacted with a patient, and using the app, we could interpret from his sign language a message which proves the solution may actually be transformational. “I have not seen an app like this. Very easy to use on a mobile phone wherever I need. When will this be available for everyone? I can also learn ISL signs through this app. Will this support IT-related signs?” told the deaf patient.

The Roadmap

For 2020, the majority of the focus will be on gesture-enabling solutions – mainly LetsTalkSign platform. Then, according to the founder, the startup plans to complete building all the key functionalities, implement pilots, start acquiring customers of select domains in the next two quarters.

“By the end of 2020, we should start covering direct customers, as well as small and medium business customers. Also, our plan is to kick-off GesturizeMe solutions for the entertainment and gaming industry by the end of 2020. For 2021, we plan to extend LetsTalkSign’s vocabulary to multiple domains and approach large B2B customers. Solutions for different sectors – like agriculture and healthcare – will also be completed and rolled out soon,” said Jayasudan Munsamy.

Here is a short demo of LetsTalkSign.

The post How DeepTechVision.AI Startup Is Enabling Hearing Impaired People To Communicate Freely appeared first on Analytics India Magazine.