[Employee of Signvrse wearing motion capture suit and translating audio into sign language so it can be used in app]
[Screen showing avatar being trained by an employee]
[Signvrse founder Elly Savatia entering the office]
[Savatia and colleagues in the office]
[Signvrse employee doing sign language while wearing a motion capture suit]
[Office]
Elly Savatia (interview): “Signvrse is an assistive technology, a social enterprise, with a mission to bridge communication barriers for the deaf community. And, how we do this is? We’ve built Africa’s first AI translation platform called ‘Terp-360.’ We like to call it Google Translate for sign language – and how this works is it interprets text and speech input and outputs very fluid natural avatars, which is a digital human that now signs the same way a human being does at scale.”
[Employee wearing a motion capture suit]
[Employee translating audio into sign language]
Elly Savatia (interview): “Our biggest challenge has been access to sign language data. Looking at the nature of our solution, it heavily relies on localised sign language data to power our platform, and accessing that has been a problem. One, African sign languages are not documented online. You would easily find British Sign Language, American Sign Language data on the web. So we have had to collect our own sign language data, which is expensive. So we really had to push our fundraising efforts to enable us to really generate localised, diverse Kenyan sign language data.”
[Employee translating audio into sign language while wearing motion capture suit]
Elly Savatia (interview): “I think AI is becoming an ongoing conversation and it is an enabler and with companies like Signvrse that are building assistive devices, there is only less that human interpreters are able to do. They are very limited. Context: if I am to take a human interpreter from a legal kind of setting to a medical kind of setting. Now what we are doing is that we are now using AI to have interpretation at a scale and this is the power of AI that we are leveraging on.”
[Employee translating audio into sign language while wearing motion capture suit]
Elly Savatia (interview): “The techniques that we are using to digitise sign language we use motion capture systems to accurately record sign language motion. So we are looking at the body movement, hand rotation and articulation, facial expressions that enables us to accurately capture the linguistic and vibrant nature of sign language as a language and that is what makes our platform unique.”
[Waiter Kelvin Munene, who is deaf, walking up to a table to take order from customers at Pallet Cafe using Signvrse’s Terp 360 app]
[Munene taking an order using the app]
Kelvin Munene (interview): “I had a lot of challenges when searching for a job and sometimes when I go to hospital I have a lot of challenges but I think when using this app, it will help those challenges and it will make communication easier.”
[Munene delivering order]
[Leakey Nyabaro, a member of the deaf community and the founder and director of Galaxy Sign Language Training Centre in Kenya]
Leakey Nyabaro (interview): “I have also tried to find out how they transcribe signs, we find that there are a lot of errors. It’s not that perfect but somehow I find it more effective when it is sign to voice (translation). We find that it is voiced correctly, benefitting more of the hearing community but not the deaf. But as well, I think in the nearby future, as it grows into perfection, which it will be, it may reduce the job opportunities for sign language interpreters.”
[People interacting at the Galaxy Sign Language Training Centre]
This script was provided by The Associated Press.