Sign language is mostly used by deaf and dumb people as their mother tongue. Sign language uses the hand gestures. In general, deaf people have problem in communicating with other people, who unable understand sign language. The Smart Glove is a normal glove fitted with flex sensor and accelerometer. In general flex sensors are variable resistors and the resistance of flex sensors increases as sensor bends and flat flex sensor has an resistance 30KΩ at angle of 0 and changes to 70 KΩ at an angle of 90 degrees. The sensor output varies with degree of bend made on the sensor. They convert the change in bend to electrical resistance, the more the bend, more the resistance value. The output from the sensor is in analog in nature and is converted to digital and compared the output of sensors with prestored values in controller (Arduino UNO) and then using Bluetooth module the message can be transferred to the mobile using Arduino Bluetooth TTS app. In the same way Speech can be converted to Text using Arduino Voice Control and message is displayed on LCD. It has an greater mobility and low cost. In this project controller (Arduino UNO), LCD display (16×2), flex sensor-5, power supply, MPU 6050 Accelerometer and Bluetooth Module (HC-05) and Software Arduino IDE, Bluetooth TTS (Android Application) are used.
In General for converting Sign Language to speech conversion mainly there are two types.
- Image Processing Technique.
- Sensor based Technique.
In Image Processing the sign language from speech and hearing impaired persons can be detected by video based recognition system. The person will stand in front of camera try to communicate with other persons through their SL. And the algorithm present inside will convert this SL language to Speech. The time taken by this process is very less. And it gets accurate results. But the problem with this it involves a complex Algorithm and mainly with dealing video so the background must be so clear otherwise it may not detect the signs made by impaired people and unable to convert to text or speech.
In Sensor Based it use sensor based recognition that are mostly it can use a glove fitted with flex sensors and it can wear by impaired persons. And the algorithm present inside will convert SL language to speech. The time taken by this process is almost 0.7secwhich is higher than time taken by Image Processing Technique. The main advantage of this is it is a simple hardware structure and it can be carried to many places as usual. so in this way are mainly focused on Sensor Based Recognition System for Converting SL to Speech and Text conversion without effecting their respective languages.
In this proposed system, the approach is used with microcontroller (Arduino UNO) and flex sensor based data glove. LED indicates while the data is transmitted. The glove is internally equipped with flex sensors and accelerometer. Whereas flex sensors work on bending of fingers and Accelerometer works on orientation of palm. For every particular gesture, the flex detector produces a proportional modification in resistance and measures the orientation of hand. The process of those hand gesture is exit in controller. The gestures made are compared with in database and output is generated in the form off voice audio and display data on LCD.
The sign language elements includes Articulation Points(different joints), Hand Configurations(decides the bend at fingers, palm), Movements Type(forward, backward etc.), Hands Orientation(Palm Up, Palm Down) and Facial Expressions(Happy, Sad etc) plays an important role in SL. These factors decides what exactly the Impaired person is conveying. The Sign language Elements in detail is given in Fig 3.1.
Figure 3.1: Sign Language Elements
The overall functioning of the System is explained through the block diagram shown in figure 3.2. It represents the general order and hierarchy of various working blocks of the project. The person wears the glove which has flex sensors and accelerometer stitched to it and makes the gesture according to the Sign Language. Arduino Uno is used to gather signals from the flex sensor and accelerometer placed on the glove. Then the processed output is send over the LCD to display the text output and also via a Bluetooth link to an Android Smartphone or a Personal Computer consisting of test to speech software (application) and speech output is obtained.
- WORKING PRINCIPLE
In this project, Five flex sensors are used where flex sensors are the sensors that change in resistance depending on the amount of bend on the sensor since the resistance is directly proportional to the amount of bending and they convert the change in bend to electrical resistance, more the change more the value of resistance. So, it is often called a Flexible Potentiometer. The amount of resistance is changing when flex sensor is bending. Flex sensor acts as variable resistance R1 along with the static resistance R2 and works on the principle of Voltage divider. The variable output voltage V0 is calculated as
V0=Vin(R1/(R1+R2))
They are generally in the shape of strips and are maybe unidirectional or bidirectional. Each flex sensor requires a voltage of +5V. Each flex sensor receives a +5V. Each flex sensor receives a +5V supply when the power is turned on. When a user makes a letter gesture, the five signals from each flex sensor with combination of static resistance of 10KΩ gives analog resistance value and are sent to a microcontroller, where they are transformed from analogue to digital values and compare this digital data with prestored data
Figure 4.2: Flex sensor Connection with Arduino
The digital values are compared with already stored data in Arduino if values are matched with the database stored in Arduino then display the output on the LCD and the output can be shown on an Android device via Bluetooth Module HC-05. If there is no statement that matches current state of the hand, there is no output shown in the LCD. The HC-05 is an interesting module that allows you to add two-way (full-duplex) wireless capability to your projects. It may utilize this module to communicate between two microcontrollers, such as an Arduino, or with any Bluetooth-enabled device, such as a phone or laptop. Many Android applications are already available, making the procedure much easier. The module interacts with a microcontroller through USART at a rate of 9600 baud rate, making it simple to connect to any microcontroller that supports USART. The command mode can also be used to set the module's default settings. However this module doesn’t transfer multimedia like photos or songs. This compared data is transferred to the Mobile where in mobile they can hear the message. So Bluetooth module acts as trans-receiver.
The Android device must have Bluetooth Terminal app installed and in the same way it can convert speech to text by using Arduino Voice Control App. In this transmitter is inbuilt Bluetooth application in mobile and the receiver is Bluetooth Module (HC-05). When they try to speak through the phone using Arduino Voice Control App the Bluetooth Module receives and compares with data in Arduino and text displays on LCD display.
In this way it can convert from Sign language to Speech and from Speech to Text without effecting their respective language.
- CONNECTIONS:
The connections of our project are discussed below:
Arduino UNO which has 6 analog pins (A0-A5).
- Five flex sensors are to be taken and one terminal of each sensor is connected 5V of Arduino UNO along with static resistance of 10KΩ resistance and another terminal of sensors to the 5 analog pins of Arduino Uno (A0 - A4).
- Accelerometer pins like SDA is connected to A4 pin of UNO and SCL pin of Accelerometer is connected to A5 pin of UNO.
- GND, VCC pins of Accelerometer are connected to GND and VCC pins of UNO.
- Bluetooth Module (HC-05) pins like Tx, Rx are connected to Rx, Tx pins of Arduino UNO. And GND, VCC pins are connected to GND and VCC pins of UNO.
- LCD display pins like D4, D5, D6, D7 are connected to the 10,11,12,13 pins of Arduino UNO. RS, Enable pins are connected to 8, 9 pins of UNO.
- LCD display units like VDD and VSS are connected to 5v and Ground pin of Arduino UNO.
In this project sensor based sign language to speech recognition system is used. So for that is Flex sensors chosen. Using flex sensor it measure the change in resistance Motion Sensors (Flex Sensor) - Flex sensors are sensors that alter resistance based on the degree of bend they are subjected to. The electrical resistance is converted from the change in bend. They are available in thin strip from and can be unidirectional or bidirectional. Resistive carbon components are used in the Flex Sensor technology. The Flex Sensor achieves a fantastic form-factor on a tiny flexible substrate as a changeable printed resistor. When the substrate is bent, the sensor generates a resistance output that is proportional to the radius of the bend and the smaller the radius, the greater the resistance value. Flex sensors are available in lengths ranging from 1 to 5 inches, or around 73mm in length and 6.35mm in breadth. The flex sensor's resistance fluctuates between below or above 550. The major difference between a unidirectional flex sensor and a bidirectional flex sensor is that the resistance of a unidirectional flex sensor rises as it is bent, whereas the resistance of a bidirectional flex sensor reduces as it is bent. The unidirectional flex sensor has a resistance of 10K at rest or 00 bent. As it is bent more at 450 degrees, the resistance rises in proportion to the bend. The resistance of the unidirectional flex sensor range from 30K to 50K at 900 bend.
As sensors have an property of changing their resistance when they try to bend and are fitted with glove initially +5V is given to each flex Sensors and when speech and hearing impaired people wore that glove and try to communicate through sign language so automatically they try bend their fingers, so glove which are fitted with flex sensors bend then automatically there will be an change in voltage when there is an change voltage when there is an change in be resistance. The output of these flex sensors are Analog in nature. The output from these 5 flex sensors are send to Arduino where analog signals are converted to digital and these digital information is compared with the already stored data in Arduino if the output from flex sensors are matched with stored data then the data i.e., the string or alphabet are sent to the Bluetooth Module and from Bluetooth module it is transferred to Mobile through Arduino Bluetooth TTS App. In this transmitter is Bluetooth Module and the receiver is Bluetooth present in Mobile. In this project it can also convert Speech to text by using Arduino Voice Control. In this process transmitter is Bluetooth present in Mobile and the receiver is Bluetooth Module. So when they speak mobile Bluetooth is connected to Bluetooth Module (HS-05) and the text is displayed on LCD (16*2).In this it can convert both from sign to speec hand speech to text conversion.
