Wellbeing Track & Change: Usability and evaluation of Workplace Wellbeing Assist, a digital tool for the mental wellbeing of residential aged care workers

This aim of this project is to develop and evaluate a digital monitoring and decision support tool (Workplace Wellbeing Assist [WWA]) for use in residential aged care (RAC) facilities. WWA is designed to enhance the workplace culture and mental wellbeing of direct-care staff, however it is not a diagnostic tool. The project will be undertaken at a RAC facility located in the Barwon South West region of Victoria, Australia. The project involves 2 stages: (1) a 2-week unmoderated usability trial of WWA, and (2) a 24-week, in-situ evaluation trial of WWA. Usability trial Materials: WWA will be available to participants to use over a 2-week period via the internet on their mobile, tablet or computer. Procedures: Up to 10 participants will be recruited from a RAC facility participating in the project or the general public. Interested individuals will be emailed a Plain Language Information Statement (PLIS) and consent form (CF). Upon receipt of the CF, the participant will be given a username and password to access WWA for up to 2-weeks. They will be asked to test as many components of the digital tool as possible (i.e., the Resource Library), noting any problems encountered. For the duration of the usability trial, WWA will be pre-populated with dummy data to allow the participant to view and interact with the features of the digital tool. Following the 2-week usability trial period, a recorded semi-structured interview will be conducted with RAC participants. An interview will be conducted via the telephone, Skype or Zoom and will take approx. 30-45 minutes. General public participants will complete an online questionnaire via Survey Monkey which will take approx. 15 minutes, depending on how much feedback is provided. Participants will be nominally reimbursed with a $10 Woolworths gift card. Frequency of use: Participants will have up to 2-weeks to access WWA during the usability trial. They will be free to access and spend as much time using the digital tool as they choose during the usability trial period. It is expected that they will spend a maximum of 2 hours testing the digital tool. Data collected from the interviews and online questionnaires will be de-identified and reviewed for any problems encountered with WWA or the resources (i.e., videos not playing, broken links). These problems will then be corrected prior to the commencement of the evaluation trial. Evaluation trial Materials: WWA will be available to participants to use via the internet on their preferred device wherever they have internet connection. Procedures: For the evaluation to proceed at a facility, two conditions must be met: (1) a RAC manager must first be recruited to the study, and (2) a minimum of five direct-care workers must be recruited to the study. Managers and direct-care staff will be sent an invitation to participate in the evaluation to their RAC work email account. Upon completion of a set of mandatory pre-evaluation questionnaires, direct-care participants will have immediate access to WWA. Each time they login to WWA, they will have the option of completing a brief Workplace Wellbeing Survey (WWS). Participants will be encouraged to complete the survey on a fortnightly basis; however, they will be free to do so more or less often. The WWS consists of 10-items, six workplace-related items (e.g., job control) and four personal wellbeing items (e.g., ‘How satisfied do you feel with your life?’). Each item is answered on a 5-point Likert-style scale from ‘Not at all’ to ‘Completely’ and participants are asked to answer each item in relation to the past week. If a participant responds to a workplace-related item with ‘Not at all’ or ‘A little’, the question will expand to provide the participant with additional contextual responses to that item (e.g., I had little or no say over the way I do my job). These questions may take up to three minutes to complete, although it is expected that they will take less time across the evaluation period as participants become familiar with the questions. Direct-care staff will receive confidential and personalised feedback displayed on their WWA dashboard, based on a composite, aggregate score calculated from their responses to the four personal wellbeing survey items (in real-time). Based on participants aggregated score, they will be recommended three tailored resources (from a selection of 16 resources) via their WWA dashboard to assist their personal mental wellbeing. Resource topics include Effective Communication, Time Management, Strengths Building, Managing Worry, and Practising Gratitude. Each resource is structured in the same way with a topic-related image, three questions to help the participant decide if the resource maybe helpful to them, and four sections covering what the topic is, why it is important, practical advice and links to external credible external sources (i.e., Beyond Blue, Black Dog Institute) where the participant can learn more about the topic, or access practical tools (i.e., worksheets). All resources can be accessed at any time from a Resource Library located on the dashboard. In addition, participants will have access to two videos (each less than two minutes) on how to use and understand WWA, and two further text-based resources on the mental health continuum and mentally healthy workplace. Participants will also be able to monitor their personal wellbeing over time via a trendline graph displayed on their dashboard. The trendline is updated live upon each WWS completion. Each fortnight, managers participating in the evaluation will have access to a dashboard that will display aggregated and non-identifiable staff scores from the WWS, as well as the secondary responses selected in relation to the workplace-related items. To assist managers to take action, WWA includes a resource tailored for each of the six workplace-related items. The management resources contain an (up to 60 second) animated video summarising the topic and the same four sections as noted for the personal wellbeing resources above. In addition, each resource includes questions to help the manager consider the topic in relation to their workplace and where they may need to take action. Managers also have access to two videos (each less than two minutes) on how to use WWA and how to interpret direct-care staff wellbeing scores. In addition, managers will have access to the full Resource Library, including the 16 personal wellbeing resources and the two text-based resources (mental health continuum and mentally healthy workplace) made available to direct care staff. The manager dashboard and direct care staff dashboard are separate on WWA and neither has access to the other. Overall, accessing the resources, how often and how long, is up to the participant (direct care and manager). However, it is expected that the text-based resources will take approximately 15 minutes to read, plus additional time should the participant choose to work through any of the practical suggestions or access the external links. Following their first login to WWA, direct care staff will be presented with a PLIS, if upon reading the PLIS they decide to participate, they click on an ‘I agree with the above conditions’ button and will then be required to create an account by submitting a strong password. Direct care participants will then complete demographic items (at Week 0), and three sets of online questionnaires designed to measure changes in the workplace climate and staff mental wellbeing (at Week 0, Week 12, and Week 24). The questionnaires will take approximately 20-30 minutes to complete. Direct care participants will also complete a System Usability Scale two weeks into the evaluation. Managers interested in the evaluation will be sent a PLIS and CF to their RAC workplace email account. Upon receipt of their CF, the manager will be directed to complete demographic items online (via Survey Monkey). Managers will then be provided with a link to WWA and be required to create an account by submitting a strong password, following which they will have immediate access to WWA. They will not complete any further questionnaires. At Week 12, participants (direct care and managers) will be invited to participate in a recorded, one-to-one semi-structured interview via telephone, Zoom or Skype. Interviews will be conducted with the first participants to respond to an email invitation (up to N=15). The interviews will take up to 60 minutes and will allow participants to provide feedback on the acceptability, usability and usefulness or WWA. Interview participants will be nominally reimbursed with a $10 Woolworths gift card for their time. Frequency of use: The evaluation trial will be conducted across a 24-week period. It will be recommended that direct-care participants use WWA on a regular basis to monitor their mental wellbeing and to complete the Workplace Wellbeing Survey on at least a fortnightly basis. Managers will be advised to monitor the management dashboard on at least a fortnightly basis to review the updated, aggregated data. Usage data will be collected from WWA, such as number of logins, how many times the WWS was completed, and which and how many resources were accessed and for how long. All data collected from the evaluation trial will be de-identified prior to any analysis. Adherence: Regular use of WWA will be encouraged for direct-care staff and mangers to achieve maximum benefit in assisting to create a mentally healthy workplace culture. Push notifications and email reminders will be sent to direct-care staff if they have not completed the Workplace Wellbeing Survey for two weeks or more. Managers will be sent fortnightly reminders announcing that new aggregate data is available for their review and action where required.