Minimizing Communication Barriers with AI: Project Aims and Methodology

Hey everyone!

In this post, I will outline the aim and objectives of my project, delve into the methodology I am using to achieve the same.

Aims and Objectives

The primary aim of my project is to minimize communication barriers between the deaf and non-deaf communities in Saudi Arabia using deep-learning techniques.

Here are the key objectives I am working towards:

  • Understanding Saudi Arabic Sign Language Gestures: Analyzing and comprehending the nuances of SSL to accurately capture and translate gestures.
  • Creating a Gesture-Capturing Tool: Developing a program to capture SSL gestures and build a comprehensive dataset.
  • Building and Training a Deep Learning Model: Using the dataset to train a Multi-Layer Perceptron model, ensuring accurate gesture recognition.
  • Developing a Desktop Application: Creating an intuitive interface for users to interact with the trained model, facilitating real-time gesture translation.
  • ROS node Development: The implemented system will be ported as a ROS node, so it can be used across any robotic system which is ROS compatible.

Brief Methodology

To achieve these objectives, I have outlined a detailed methodology that combines data collection, model training, and application development:

  1. Data Collection and Gesture Analysis: Utilizing online resources to analyze and understand SSL gestures, studying hand movements within the context of Saudi Arabic Sign Language.
  2. Gesture Detection and Dataset Creation: Developing a program using Python and MediaPipe to detect hand landmarks and create a robust dataset.
  3. Model Training: Training a Multi-Layer Perceptron model using the collected dataset to recognize SSL gestures.
  4. Real-Time Gesture Recognition: Developing a program for real-time gesture identification and displaying recognized gestures on-screen.
  5. User Interface Development: Building a desktop application using TKinter for easy interaction with the model.
  6. Model Evaluation: Ensuring the model's accuracy through testing with real-world datasets.

Research Questions

How can deep learning be used to overcome the lack of tools to translate Saudi Arabia sign language (SSL) into Arabic to provide seamless communication between the deaf community and society?

Deliverables

  • A tool to extract data points using MediaPipe library
  • GUI for user interaction
  • Make complete project as a ROS node for deployment in Pepper Robot
  • Thesis Report

Evaluation Strategy

  • Conduct live tests with volunteers fluent in SSL to evaluate real-time performance.
  • Gather feedback on the desktop application's interface and usability, and iterate based on feedback.
  • Test the model under various conditions to ensure robustness.
  • Analyze errors and refine the model to improve accuracy.

Gantt Chart

Link for Gantt Chart as it will be updated continuously:Gantt Chart

Thanks until next post!

Comments

Post a Comment

Popular posts from this blog

Hey!!! Hi, I'm Megha. Welcome to my blog, where I will chronicle the journey of my MSc thesis project. Here, you will find updates on my research progress, challenges, solutions, and implementation details. Background Deaf individuals primarily use sign language for communication, but most hearing people do not understand it, leading to communication barriers. While significant research has been done on various sign languages, Saudi Sign Language (SSL) lacks technological applications. Globally, about 14% of the population has some form of disability, with many experiencing hearing loss. In Saudi Arabia, approximately 720,000 individuals are deaf, highlighting the need for effective communication technologies. Saudi Vision 2030 aims to provide resources and facilities that enable disabled individuals to access education and employment opportunities, promoting their independence and participation in society. Project Inspiration : The Need for an SSL Translator Unlike American Sign L...
Read more

Understanding Saudi Arabic Sign Language Gestures

In Saudi Arabia, a nation rich in traditions and diverse cultures, Saudi Sign Language (SSL) serves as a crucial communication method for the deaf community, embodying both the Kingdom's heritage and adaptability. The Origin of Saudi Sign Language (SSL) developed naturally within deaf communities in Saudi Arabia. Initially based on informal sign systems used in families and local settings, SSL evolved into a recognized language with unique grammar, syntax, and regional variations. Government Acknowledgment and Support Efforts:   to formally recognize and support SSL have increased, with the Ministry of Education and other institutions promoting SSL as a cultural and linguistic asset. Steps have been taken to integrate SSL into educational settings, providing resources and support for deaf students. Usage and Community Influence: Estimating the number of SSL users is challenging, but it remains a primary communication tool for many deaf individuals in Saudi Arabia. SSL ...
Read more

Understanding and implementation of Multilayer Perceptrons (MLP)

Image
This blog provides an in-depth look on the R&D phase about MLP, along with detailed insights from the training and testing phases of the Multilayer Perceptron (MLP) model. In the realm of artificial intelligence, neural networks have become a cornerstone for solving complex problems. One such neural network that has garnered significant attention is the Multilayer Perceptron (MLP). After creating the dataset, we need to have a clear understanding of MLPs. But before we dive into the technical details, let’s first get acquainted with what an MLP truly is. A Multilayer Perceptron is a type of neural network composed of multiple layers, including an input layer, one or more hidden layers, and an output layer. Each of these layers contains neurons, the fundamental building blocks of the network. MLPs are renowned for their versatility and are widely used in tasks like forecasting models and image pattern recognition. [1] Exploring the Architecture and Functioning of a Multilaye...
Read more