Literature Review: Advancements in Sign Language Translation

I am thrilled to continue sharing my journey through the development of myproject, which focuses on creating a groundbreaking Saudi Sign Language (SASL) translation system. Today, I will dive into some fascinating literature that has informed my research and helped shape the direction of my work.

To build a robust SASL translation system, it's crucial to understand the state-of-the-art techniques and technologies that have been developed for similar purposes. Here are some key studies and their contributions to the field:

Convolutional Neural Networks (CNN) for Gesture Recognition

Two pivotal studies have employed CNN to recognize Arabic sign language gestures and translate them into text. These studies underscore the power of deep learning in accurately identifying and translating sign language.

[1] focuses on facilitating communication between deaf and non-deaf individuals using a Pepper robot. The system not only recognizes gestures but also processes natural language, converting spoken words into text that is displayed on the robot's screen. This dual functionality enhances communication in a dynamic and interactive way.

[2] specifically targets Saudi signs, training a system to identify 40 different gestures with an impressive accuracy of 99.47%. The focus on Saudi-specific signs makes this study particularly relevant to my project, highlighting the potential for high accuracy in localized sign language translation.

Dynamic Dataset Creation and Feature Extraction

Another interesting approach is the dynamic creation of an Arabic Sign Language (ArSL) dataset using videos.

[3] research compiled 1,500 video files showing 100 ArSL signs, utilizing a gradient-based key frame extraction method to isolate signs from the frames. Features were extracted using intensity histograms integrated with Gray Level Co-occurrence Matrix (GLCM) features. This method provides a rich dataset that is essential for training accurate recognition models.

Support Vector Machines (SVM) for Sign Recognition

While deep learning is popular, some studies have explored alternative machine learning techniques like Support Vector Machines (SVM) for sign recognition.

achieved 96% accuracy in recognizing Arabic sign language alphabet signs using the Dense SIFT technique for feature extraction. The high accuracy demonstrates the effectiveness of SVMs in this

study employed the Spatio-Temporal Local Binary Pattern (STLBP) feature extraction technique,showcasing another viable method for sign language recognition.

Combining Multiple Techniques for Enhanced Recognition

Some studies have experimented with combining various techniques to improve recognition accuracy

research utilized a blend of Modified Fourier Transform (MFT), Local Binary Pattern (LBP), Histograms of Oriented Gradients (HOG), and a combination of HOG and Histogram of Optical Flow (HOG - HOF). While this approach achieved 99.11% accuracy, it required significant processing power, which could be a limitation in practical applications.

Incorporating Depth Information with Kinect Sensors

Going beyond 2D image processing, some innovative approaches have included depth information to enhance recognition.

This study used a Kinect sensor to capture both depth and 2D images, processing them to identify signed words. The use of Dynamic Time Warping allowed for the coordination and matching of each signed word with a database, providing both text and pronunciation of the sign. This method shows promise for creating more comprehensive translation systems.

Moving Forward

These studies provide a rich foundation of knowledge and techniques that I can leverage for my SASL translation system. By combining the strengths of various approaches such as CNN for high accuracy, and dynamic dataset creation for robust training - I aim to develop a system that not only meets the current needs but also pushes the boundaries of what is possible in Saudi Arabia Sign language translation technology.

Stay tuned until next time!

References

[1]      D. A. Alabbad, N. O. Alsaleh, N. A. Alaqeel, Y. A. Alshehri, N. A. Alzahrani, and M. K. Alhobaishi, “A Robot-based Arabic Sign Language Translating System,” Proc. - 2022 7th Int. Conf. Data Sci. Mach. Learn. Appl. CDMA 2022, pp. 151–156, 2022, doi: 10.1109/CDMA54072.2022.00030.

 

[2]      A. H. Al-Obodi, A. M. Al-Hanine, K. N. Al-Harbi, M. S. Al-Dawas, and A. A. Al-Shargabi, “A Saudi Sign Language Recognition System based on Convolutional Neural Networks,” Int. J. Eng. Res. Technol., vol. 13, no. 11, pp. 3328–3334, 2020, doi: 10.37624/IJERT/13.11.2020.3328-3334.

 

[3]      R. Ahmed et al., “Arabic Sign Language Translator,” J. Comput. Sci., vol. 15, no. 10, pp. 1522–1537, Oct. 2019, doi: 10.3844/JCSSP.2019.1522.1537.

 

[4]      M. A. Ali, M. R. Ewis, G. E. Mohamed, H. H. Ali, and H. M. Moftah, “Arabic Sign Language Recognition (ArSL) Approach Using Support Vector Machine,” 27th Int. Conf. Comput. Theory Appl. ICCTA 2017 - Proc., pp. 17–21, Oct. 2017, doi: 10.1109/ICCTA43079.2017.9497164.

 

[5]      S. Aly and S. Mohammed, “Arabic Sign Language Recognition using Spatio-Temporal Local Binary Patterns and Support Vector Machine.” 2014.

 

[6]      A. addin I. Sidig, H. Luqman, and S. A. Mahmoud, “Arabic sign language recognition using optical flow-based features and HMM,” Lect. Notes Data Eng. Commun. Technol., vol. 5, pp. 297–305, 2018, doi: 10.1007/978-3-319-59427-9_32.

 

[7]      A.-G. A.-R. Abdel-Samie, F. A. Elmisery, A. M.Brisha, and A. H. Khalil, “Arabic Sign Language Recognition Using Kinect Sensor,” Res. J. Appl. Sci. Eng. Technol., vol. 15, no. 2, pp. 57–67, Feb. 2018, doi: 10.19026/RJASET.15.5292.