Final Step - Integrating the whole system with ROS

Why Use ROS for Robotic Development?

ROS (Robot Operating System) is a widely-used framework for developing robotic systems. Despite its name, ROS is not an actual operating system - it is a middleware framework that runs on top of a real operating system. In this post, I will explain why ROS is so popular for robotics development, and why I've chosen to use it in my Saudi Sign Language (SSL) translation project.

What Makes ROS Stand Out?

There are several key reasons developers turn to ROS when building complex robotic systems:

1. It’s Free and Open-Source

Unlike other robotics development software that requires expensive licenses, ROS is completely free. Originally designed to run in Linux environments, getting started with ROS doesn’t cost anything. This makes it a fantastic choice for both research and production without the financial burden of proprietary software.

2. ROS is Distributed

One of the most powerful features of ROS is its distributed nature. In ROS, the system is broken down into components called Nodes, which communicate with each other using channels known as Topics. Each node can either publish information to a topic or subscribe to a topic to receive information.

This distributed structure offers flexibility and fault tolerance. For example, if one node encounters an issue, the rest of the system can continue to function without requiring a complete shutdown. It also makes it easier to update individual components without disrupting the entire system.

3. Strong Community Support

ROS boasts a large, active developer community. This ongoing support makes ROS a continuously evolving platform with regular updates and improvements. The community-driven nature also means there's a wealth of resources and solutions available when tackling development challenges.

Why Use ROS in the SSL Project?

In the early stages of the SSL translation project, the entire system was condensed into a single Python file. However, by converting the project into a ROS-compatible framework, we can ensure it integrates smoothly with any ROS-based robotic system. This opens up the potential for deploying the SSL translation system in a wide variety of robots without significant modifications.

System Architecture Overview

The ROS implementation of the SSL project consists of two ROS nodes working together:

1.ssl_translator_audio.py : Gesture Recognition

- This node is responsible for capturing images from the camera and identifying the hand gestures using the trained MLP (Multi-Layer Perceptron) models. - Once a hand sign is recognized, it publishes the corresponding label to a topic named 'gesture_label'.

2.SayIT: Text-to-Speech

- The second node subscribes to 'gesture_label' to receive the detected sign label. - It sends the label to Google Cloud's Text-to-Speech API, retrieves the audio version of the label, and plays it out loud.

The system's architecture can be visualized through the rqt graph, which shows the communication flow between the two nodes. Once both nodes are running, users won’t notice any difference between this ROS-based implementation and the earlier Python-only version.

How to Set Up and Run the SSL ROS Package

If you'd like to try out the ROS version of the project, follow these steps:

1. Download the ROS package

The project can be downloaded from the following URL: arsl_ros_package

2. Grant Execution Permissions

Navigate to the `src` folder of the package and grant execution permissions to the two Python nodes by running the following command in the terminal: chmod +x NODENAME.py

3. Build the Workspace

From the workspace folder, build the project by running: catkin_make

4. Source the Terminal

Source the terminal by running: source devel/setup.bash

5. Run the ROS Master Node

Start the ROS master node by executing: roscore

6. Launch the ROS Nodes

In two separate terminals, run the following commands to launch the two nodes: rosrun arsl SSL_translator_audio.py , rosrun arsl say_it.py

If all dependencies are met, you should see a camera preview with the identified sign label displayed on the screen. The system will also speak the label aloud, enabling real-time interaction with the SSL translation system.

Conclusion

By using ROS, we've made the SSL translation system more robust and adaptable to various robotic platforms. This setup allows for future enhancements and integrations with more complex robotic systems, further advancing communication between deaf and non-deaf communities.

Real-time Implementation Demo Video: demo_video

GitHub Repository for the Entire Project: whole scource code and datasets

Feel free to explore the demo video and the repository for detailed insights into the project’s real-time implementation and codebase!

A huge thank you to everyone who took the time to read through these blog! Your interest and support in exploring the Saudi Sign Language translation project make this journey even more rewarding. I look forward to sharing more updates and insights with you as this project continues to evolve. Stay tuned!

Comments

Post a Comment

Popular posts from this blog

Hey!!! Hi, I'm Megha. Welcome to my blog, where I will chronicle the journey of my MSc thesis project. Here, you will find updates on my research progress, challenges, solutions, and implementation details. Background Deaf individuals primarily use sign language for communication, but most hearing people do not understand it, leading to communication barriers. While significant research has been done on various sign languages, Saudi Sign Language (SSL) lacks technological applications. Globally, about 14% of the population has some form of disability, with many experiencing hearing loss. In Saudi Arabia, approximately 720,000 individuals are deaf, highlighting the need for effective communication technologies. Saudi Vision 2030 aims to provide resources and facilities that enable disabled individuals to access education and employment opportunities, promoting their independence and participation in society. Project Inspiration : The Need for an SSL Translator Unlike American Sign L...
Read more

Understanding Saudi Arabic Sign Language Gestures

In Saudi Arabia, a nation rich in traditions and diverse cultures, Saudi Sign Language (SSL) serves as a crucial communication method for the deaf community, embodying both the Kingdom's heritage and adaptability. The Origin of Saudi Sign Language (SSL) developed naturally within deaf communities in Saudi Arabia. Initially based on informal sign systems used in families and local settings, SSL evolved into a recognized language with unique grammar, syntax, and regional variations. Government Acknowledgment and Support Efforts:   to formally recognize and support SSL have increased, with the Ministry of Education and other institutions promoting SSL as a cultural and linguistic asset. Steps have been taken to integrate SSL into educational settings, providing resources and support for deaf students. Usage and Community Influence: Estimating the number of SSL users is challenging, but it remains a primary communication tool for many deaf individuals in Saudi Arabia. SSL ...
Read more

Understanding and implementation of Multilayer Perceptrons (MLP)

Image
This blog provides an in-depth look on the R&D phase about MLP, along with detailed insights from the training and testing phases of the Multilayer Perceptron (MLP) model. In the realm of artificial intelligence, neural networks have become a cornerstone for solving complex problems. One such neural network that has garnered significant attention is the Multilayer Perceptron (MLP). After creating the dataset, we need to have a clear understanding of MLPs. But before we dive into the technical details, let’s first get acquainted with what an MLP truly is. A Multilayer Perceptron is a type of neural network composed of multiple layers, including an input layer, one or more hidden layers, and an output layer. Each of these layers contains neurons, the fundamental building blocks of the network. MLPs are renowned for their versatility and are widely used in tasks like forecasting models and image pattern recognition. [1] Exploring the Architecture and Functioning of a Multilaye...
Read more