Speech driven app to assist people for assisted living

Residents in care homes find it difficult to carry out certain mental and communication tasks on their own without the help of carers or staff members due to mild impairments that they might be suffering from such as cognitive, mobility and spatial. This requires the availability of staff which comes as a premium. The latter has effects on the resident’s quality of life as without it they may feel isolated and demotivated.

The focus of this study is to use technological based approaches to facilitate the resident’s everyday tasks. As part of the study an application will be developed to help users in tasks such as: check reminders, meals, events, family visits, access websites and write emails through a portable mobile app controlled via a speech-driven interface. Such a system would be managed by the staff of the care home and additional remote carers such as relatives. This app helps to bridge the gap between the carers, residents and remote carers. One of the main approaches would be to use a speech driven approach in order to overcome some of the impairments typically offered by the target users.

The app is based on retrieving data from a database about the care facility and its events, meals, visits and reminders for various things, for example medication. Material.io components were used for a uniform look between screens. The technologies used include Flutter’s Framework which was used to develop scalable, multi-platform and user-friendly applications. A database to save the user’s data securely and update it in real-time is used. Finally, a voice assistant also comes into play, since it is used to interpret the user’s speech into actions that the app can execute such as, switch screens and provide voice feedback to the user when executing a command.

Figure 1. The User’s (Residents) Screens
Figure 2. The System’s Architecture

Student: Kyle Zammit
Course: B.Sc. IT (Hons.) Software Development
Supervisor: Dr Michel Camilleri