Practical Artificial Caregiver (PAC): a multimodal application to support the care of persons with impairments in care institutions

Disability is a public health issue and it is estimated that more than 1 billion persons globally live with some description of impairment. The number of persons with disabilities is increasing constantly, and many are more than likely to experience impairments at some point in their life. There is also a need to support and improve healthcare services for individuals with such conditions.

Combining different technologies, such as voice user interfaces (VUIs) and artificial intelligence (AI) assistants, to mimic human speech and behaviour is a promising and interesting concept. This concept could be applied on mobile devices, such as smartphones and smartwatches, to help tackle the issues of caregiver shortage, loneliness and/or physical disability.

Practical Artificial Caregiver (PAC) is an assistive solution developed with the objectives of supporting caregivers assisting persons with impairments and/or disabilities residing in care homes. The solution bridges together the interactions and communication that take place between a resident and their caregiver, without the need for a physical presence.

The architecture of this system was developed in such a way as to enable multiple mobile devices to access and make use of synchronised data from an online database. The system also involved utilising a simple but effective user experience (UX) for persons with impairments, using assistive technologies and others such as natural language processing, voice recognition and VUIs. This was achieved by deploying the use of mobile technologies, more specifically smartphones, tablets, and smartwatches. The system was split into two: an application for the caregivers called PAC Carer, and an application for the resident called PAC Assistant.

PAC Assistant was deployed through the use of an android smartwatch device. This was deemed to be the ideal choice, as it would allow the user to wear a non-intrusive device, thus eliminating the need to remember to carry a non-wearable device, and consequently feeling more at ease. Moreover, a watch device was chosen on the merit of being a familiar object to residents, having a simple interface. Therefore, explaining that it was practically a watch that ‘speaks’ and one they could ‘talk to’, helped encourage the residents to use the technology.

PAC Carer was deployed through the use of an android mobile device. The solution could be used on both a tablet and a mobile phone, depending on the caregiver’s preference. From this side of the solution, caregivers could view a resident’s journal of reminders and alarms – which is essentially a list of tasks or activities that a patient would need to do, and reminded of, on a daily basis. The caregiver could also create new entries into a resident’s journal, such as alarms (for example when someone would need to be notified about tea-time) and reminders (for instance, when one would need to be reminded to take their medication at a specific time during the day). The software would also enable caregivers to send messages to their residents, to which the latter could reply by using voice input from their watch application.