Technology

Technology for community

There are few aspects more important in the formation of culture and community, than that of human language and our ability to communicate. Modern technology has revolutionized the accessibility of communication. Translation software enables us to communicate across borders, and different assistive tools ensures a more including society. However, while automated translation software has revolutionized communication across barriers of verbal language, it has yet to provide the same service for users of sign language.

DeepSign seeks to extend the technologies of machine translation into the world of sign language, by contributing to the developing field of sign language translation (SLT). By giving computers the ability to understand sign language as well as spoken or written language, we can incite a paradigm shift in the integration of sign language communities to society at large. We do this by ensuring communication across the verbal and sign language communities, at all levels of society.

Our technology

Neural Machine Translation (NMT) – using neural networks to performance language translation – has seen huge developments in the last decade. These methods employ specialized neural network architectures trained on millions of sentence pairs. On languages with large datasets to train on, such as English and French, NMT is starting to rival human translations in quality and accuracy.

As sign language is a purely visual language, translation to and from sign language poses a different and more challenging problem than translation between text-based languages. The natural digital format of sign language is video, which is much more abstract and information dense than a text-based sentence. Additionally, sign language NMT efforts suffer from a lack of large, high quality datasets. This has resulted in a lack of viable translation software on par with what text-based languages can offer.

DeepSign believes that recent developments in gesture recognition and real-time 3D animations makes it possible to close this gap, and push consumer sign language translation software to the level enjoyed by text-based languages. Modern gesture recognition and video processing software, combined with the ever increasing computational power of consumer devices, allows DeepSign to accurately track bodily movements, to recognize individual signs. Combined with the state of the art machine translation architecture, similar to the one employed by Google Translate, DeepSign is building real-time sign language translation software.