Data Collection Methodology Using MediaPipe Framework

The data-collection program has been completed, and the next step is to begin collecting data to construct the dataset. For this purpose, 38 different signs will be collected, with a minimum of 800 data records expected for each sign. The data collection procedure will be executed in the following order:

1.The program will be executed via the terminal with the specified sign name.

2.The individual will present the hand sign to the webcam and commence data collection by pressing ‘0’.

The advantage of this data collection program is that the background does not affect the quality of the collected data. This is due to the capability of the MediaPipe framework to recognize the hand regardless of other visual noise in the background. Additionally, the distance between the hand and the camera does not impact data quality, as the program normalizes the joint distances from the wrist joint.

To enhance the robustness of the collected data, the following measures will be implemented:

1. The orientation of the hand will be slightly varied during the data collection process for each hand sign.

2. Multiple individuals will collect data for the same hand sign.

3. During data collection, the hand will be moved back and forth to capture different distance measures.

These measures are necessary for the following reasons:

If only upright hand positions are used for training, the model may struggle to correctly recognize gestures if the hand is tilted. Since some gestures are distinguished by minor differences in the position of one or two fingers, slight changes could confuse the multi-layer perceptron (MLP) model. Furthermore, different individuals have varying hand sizes, resulting in different joint distances. Training the model on data from a single individual would limit its ability to recognize signs made by hands of different sizes and shapes, which is undesirable for a sign language translator intended for diverse users.

My project guide Dr.Maha Saadeh suggested measures to address these issues.The data collection process will involve four individuals, with each person collecting at least 200 data records per hand gesture with slight orientation variations. Consequently, 800 records will be collected per hand gesture, ensuring that the MLP model can correctly identify hand gestures despite variations in hand size and shape. Further analysis may include more hands if necessary. The data collection program currently accepts input from one hand. By changing the `max_num_hands` parameter to 2, the program has to be altered in such a way that it can accept data points for signs that use both hands. However, mixing single-handed and double-handed sign data in the same model could cause confusion. Therefore, separate models will be trained for single-handed and double-handed signs.

Comments

Post a Comment

Popular posts from this blog

Hey!!! Hi, I'm Megha. Welcome to my blog, where I will chronicle the journey of my MSc thesis project. Here, you will find updates on my research progress, challenges, solutions, and implementation details. Background Deaf individuals primarily use sign language for communication, but most hearing people do not understand it, leading to communication barriers. While significant research has been done on various sign languages, Saudi Sign Language (SSL) lacks technological applications. Globally, about 14% of the population has some form of disability, with many experiencing hearing loss. In Saudi Arabia, approximately 720,000 individuals are deaf, highlighting the need for effective communication technologies. Saudi Vision 2030 aims to provide resources and facilities that enable disabled individuals to access education and employment opportunities, promoting their independence and participation in society. Project Inspiration : The Need for an SSL Translator Unlike American Sign L...
Read more

Understanding Saudi Arabic Sign Language Gestures

In Saudi Arabia, a nation rich in traditions and diverse cultures, Saudi Sign Language (SSL) serves as a crucial communication method for the deaf community, embodying both the Kingdom's heritage and adaptability. The Origin of Saudi Sign Language (SSL) developed naturally within deaf communities in Saudi Arabia. Initially based on informal sign systems used in families and local settings, SSL evolved into a recognized language with unique grammar, syntax, and regional variations. Government Acknowledgment and Support Efforts:   to formally recognize and support SSL have increased, with the Ministry of Education and other institutions promoting SSL as a cultural and linguistic asset. Steps have been taken to integrate SSL into educational settings, providing resources and support for deaf students. Usage and Community Influence: Estimating the number of SSL users is challenging, but it remains a primary communication tool for many deaf individuals in Saudi Arabia. SSL ...
Read more

Understanding and implementation of Multilayer Perceptrons (MLP)

Image
This blog provides an in-depth look on the R&D phase about MLP, along with detailed insights from the training and testing phases of the Multilayer Perceptron (MLP) model. In the realm of artificial intelligence, neural networks have become a cornerstone for solving complex problems. One such neural network that has garnered significant attention is the Multilayer Perceptron (MLP). After creating the dataset, we need to have a clear understanding of MLPs. But before we dive into the technical details, let’s first get acquainted with what an MLP truly is. A Multilayer Perceptron is a type of neural network composed of multiple layers, including an input layer, one or more hidden layers, and an output layer. Each of these layers contains neurons, the fundamental building blocks of the network. MLPs are renowned for their versatility and are widely used in tasks like forecasting models and image pattern recognition. [1] Exploring the Architecture and Functioning of a Multilaye...
Read more