Movement of Rudra-

1. Rudra will recognize the face of the user through face recognition feature, and through ROS motors will run in the same direction.
2. Rudra can also recognize users voice, and act the way it is instructed to. Eg: user said start then Rudra will start moving in forward direction, if user says stop then it will stop moving.
3. We have also integrated it with a mobile application using flutter, the joystick in that app can also control Rudra. We used Firebase to connect raspberry pi to the mobile application.

Computer Vision- (Python)

1. As elderly people do not keep track of water they drank in a day, Rudra helps them do so using computer vision it recognizes when the user is drinking water and updates the information in firebase, which is connected to the mobile app.
2. It can also recognize if a person is upset, happy, anxious or angry and passes the result to the voice assistant via firebase, then the voice assistant starts sympathizing them, and tries to uplift their mood by playing song or jokes or if the elderly wants to do share something it will listen to them and give appropriate responses. 
3. It can be used as live streaming of the house if no one is at home. Family members also have the access to the live stream through mobile application, for quick emergency responses and remote monitoring.
4. It can also see if they are having food on time or not.

Voice Assistant- (Python & Dialogflow)

1. Responds according to the mood of the user.
2. Do not let them get bored, uplifts their mood, interacts with them, sympathize them
3. Recommends them food which will keep them healthy (according to their previous meal)- it stores the data of their previous meal and through machine learning algorithm recommends them food they can have next in order to stay healthy. (backend used- firebase- connects python program to voice assistant)
4. We connected the google calendar to the voice assistant which helps Rudra give reminders about what's next in their schedule.

Rudra is equipped with multiple sensors, control units and actuators. Starting from the head, on top of the base plate, there is a raspberry pi microprocessor which has a webcam, another camera module and a speaker connected to it. The webcam will be connected to the servo motors to set it according to the position of the face for this we have designed a camera case attached in front of Rudra, while the other camera looks straight ahead. The 11.1V rechargeable battery is attached below the base plate to power motor driver and a power bank is used to power raspberry pi. The L298n motor driver is connected to the raspberry pi along with a magnetometer.
The magnetometer tells the direction of the face detected and raspberry pi operates the motors accordingly.

So how does Rudra actually work? Let us start with the voice assistant. The voice input is given to Rudra through wireless Bluetooth earphones and it is then interpreted by the Raspberry Pi. the user can converse with Rudra like any other human being and they will never feel alone. It will regularly engage in conversations with the person, monitor their responses and reply empathetically.  Using the two cameras, Rudra has the ability to recognize faces, expressions, and different actions. Rudra will automatically log in the act of drinking water so that there is no need to do it manually on the app. Rudra will also recognize when the user is eating and ask them about their food so that it can give customized meal suggestions to the user for maintaining optimum health. Besides that, Rudra’s ability to recognize expressions allows it to give more sympathetic responses during conversations. 

As shown, the human and robot interfacing happens through the Raspberry pi. All the data is collected during interactions on the Firebase Realtime database and is used to trigger further actions as per the situation. 
We used AWS EC2 to integrate the computer vision to the raspberry pi, which will be connected to Voice assistant via firebase.

Next, the Rudra companion app allows users, family members or appointed medical professionals to control the robot and monitor the human in need remotely. The app is built using Flutter and data is transferred to raspberry pi module using Firebase Realtime Database. 

All these separate sensors and backend facilities work together to deliver the best performance possible for making Rudra the perfect companion for the elderly.