Autism Spectrum Disorder
Conditions
Keywords
Data collection, autism, Behavioral Health Technician (BHT), behavioral economics, implementation, implementation strategy, implementation science, Therapeutic Support Staff (TSS)
Brief summary
In partnership with a digital health software company, the University research team created two versions of a mobile application to help behavioral health technicians (BHT's) who work with students with autism collect data. The first version comprises a basic electronic platform for data collection. The second version has the same basic electronic platform for data collection, plus additional features designed to increase motivation to collect data and ease the burden of data collection.
Interventions
Behavioral economics features designed to increase motivation to collect data and ease of data collection on the electronic platform.
Electronic platform to collect data
Sponsors
Study design
Intervention model description
After consenting to participate in the trial, participants will be randomized in pairs to one of the two study arms.
Eligibility
Inclusion criteria
* Participants must be working as a paraprofessionals or behavioral health technicians * Currently providing individual support to children with autism * Currently providing services in Philadelphia preschools, schools, day cares, or homes.
Exclusion criteria
* n/a
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| The Difference in Data Collection Consistency Between Groups Across Three Weeks | Three-week trial period | Consistency was the percentage of intervals in which aides entered data per session with children during each school day, across a three week period. These metrics were captured via web analytics in partnership with our digital health company. |
| The Difference in Data Collection Completion Between Groups Across Three Weeks | Three-week trial period | Consistency was the percentage of intervals in which aides entered data per session with children during each school day, across a three week period. These metrics were captured via web analytics in partnership with our digital health company. |
| The Difference in Data Collection Timeliness Between Groups Across Three Weeks | Three-week trial period | Consistency was the percentage of intervals in which aides entered data per session with children during each school day, across a three week period. These metrics were captured via web analytics in partnership with our digital health company. |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Perceived Norms for Implementors | Assessed at baseline and end of 3-week trial period | We adapted scales to measure mechanisms affecting data collection, including perceived norms, using social psychology methods (Armitage & Connor, 2001). The perceived norms measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger influence of norms on taking data. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time. |
| Intentions for Implementors | Assessed at baseline and end of 3-week trial period | We adapted scales to measure mechanisms affecting data collection, including intentions, using social psychology methods (Armitage & Connor, 2001). The intentions measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger intentions to take data. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time. |
| Self-Efficacy for Implementors | Assessed at baseline and end of 3-week trial period | We adapted scales to measure mechanisms affecting data collection, including self-efficacy, using social psychology methods (Armitage & Connor, 2001). The self-efficacy measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger self-efficacy about taking data. Self-Efficacy had 2 items. These items were averaged to create the scale score which could range from 1 to 7 for each measure. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time. |
| Descriptive Norms for Implementors | Assessed at baseline and end of 3-week trial period | We adapted scales to measure mechanisms affecting data collection, including descriptive norms, using social psychology methods (Armitage & Connor, 2001). The perceived norms measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger influence of norms on taking data. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time. |
| System Usability Scale | Three-week trial period | We used the System Usability Scale (SUS; Brooke, 1996) to measure aides' reactions to various statements regarding the app's usability with 10 items that use a 5-point scale from (1) strongly disagree to (5) strongly agree. The SUS has high internal consistent reliability (Cronbach's alpha = .91) and demonstrated sensitivity to change (Lewis, 2018). We adapted the original questionnaire to replace system with app across all items. The SUS is calculated as a proportion score out of 100, with higher scores indicating higher usability (≥85 = Excellent; ≥71 = Good; ≥51 = Okay (Bangor et al.). |
| Attitudes for Implementors | Assessed at baseline and end of 3-week trial period | We adapted scales to measure mechanisms affecting data collection, including attitudes, using social psychology methods (Armitage & Connor, 2001). The attitudes measure had one item, which was rated from 1=good to 7=bad (meaning that taking data would be bad). Attitudes had 2 items. These items were averaged to create the scale score which could range from 1 to 7 for each measure. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time. |
Countries
United States
Participant flow
Participants by arm
| Arm | Count |
|---|---|
| Enhanced Electronic Platform A basic electronic platform for data collection that contains additional features designed to increase motivation to collect data and ease of data collection.
Behavioral economics strategies: Behavioral economics features designed to increase motivation to collect data and ease of data collection on the electronic platform.
Electronic platform: Electronic platform to collect data | 14 |
| Electronic Platform Not Enhanced A basic electronic platform for data collection.
Electronic platform: Electronic platform to collect data | 13 |
| Total | 27 |
Withdrawals & dropouts
| Period | Reason | FG000 | FG001 |
|---|---|---|---|
| Overall Study | Lost to follow up | 1 | 2 |
Baseline characteristics
| Characteristic | Enhanced Electronic Platform | Total | Electronic Platform Not Enhanced |
|---|---|---|---|
| Age, Continuous | 35.5 Years STANDARD_DEVIATION 10.2 | 38.6 Years STANDARD_DEVIATION 12.4 | 41.6 Years STANDARD_DEVIATION 14.6 |
| Ethnicity (NIH/OMB) Hispanic or Latino | 1 Participants | 2 Participants | 1 Participants |
| Ethnicity (NIH/OMB) Not Hispanic or Latino | 13 Participants | 25 Participants | 12 Participants |
| Ethnicity (NIH/OMB) Unknown or Not Reported | 0 Participants | 0 Participants | 0 Participants |
| Race (NIH/OMB) American Indian or Alaska Native | 0 Participants | 0 Participants | 0 Participants |
| Race (NIH/OMB) Asian | 1 Participants | 1 Participants | 0 Participants |
| Race (NIH/OMB) Black or African American | 6 Participants | 12 Participants | 6 Participants |
| Race (NIH/OMB) More than one race | 1 Participants | 1 Participants | 0 Participants |
| Race (NIH/OMB) Native Hawaiian or Other Pacific Islander | 0 Participants | 0 Participants | 0 Participants |
| Race (NIH/OMB) Unknown or Not Reported | 1 Participants | 1 Participants | 0 Participants |
| Race (NIH/OMB) White | 5 Participants | 12 Participants | 7 Participants |
| Region of Enrollment United States | 14 participants | 27 participants | 13 participants |
| Sex/Gender, Customized Female | 7 Participants | 17 Participants | 10 Participants |
| Sex/Gender, Customized Male | 6 Participants | 9 Participants | 3 Participants |
| Sex/Gender, Customized Non-binary/ third gender | 0 Participants | 0 Participants | 0 Participants |
| Sex/Gender, Customized Prefer not to disclose | 1 Participants | 1 Participants | 0 Participants |
| Work experience, Years | 5.4 years STANDARD_DEVIATION 4.5 | 6.4 years STANDARD_DEVIATION 4.55 | 7.4 years STANDARD_DEVIATION 4.6 |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk |
|---|---|---|
| deaths Total, all-cause mortality | 0 / 14 | 0 / 13 |
| other Total, other adverse events | 0 / 14 | 0 / 13 |
| serious Total, serious adverse events | 0 / 14 | 0 / 13 |
Outcome results
The Difference in Data Collection Completion Between Groups Across Three Weeks
Consistency was the percentage of intervals in which aides entered data per session with children during each school day, across a three week period. These metrics were captured via web analytics in partnership with our digital health company.
Time frame: Three-week trial period
Population: 27 were enrolled (14 treatment, 13 control) and data was complete for 23
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Enhanced Electronic Platform | The Difference in Data Collection Completion Between Groups Across Three Weeks | .81 percentage of intervals |
| Electronic Platform Not Enhanced | The Difference in Data Collection Completion Between Groups Across Three Weeks | .98 percentage of intervals |
The Difference in Data Collection Consistency Between Groups Across Three Weeks
Consistency was the percentage of intervals in which aides entered data per session with children during each school day, across a three week period. These metrics were captured via web analytics in partnership with our digital health company.
Time frame: Three-week trial period
Population: 27 were enrolled (14 treatment, 13 control) and data complete for 23
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Enhanced Electronic Platform | The Difference in Data Collection Consistency Between Groups Across Three Weeks | .66 percentage of intervals |
| Electronic Platform Not Enhanced | The Difference in Data Collection Consistency Between Groups Across Three Weeks | .44 percentage of intervals |
The Difference in Data Collection Timeliness Between Groups Across Three Weeks
Consistency was the percentage of intervals in which aides entered data per session with children during each school day, across a three week period. These metrics were captured via web analytics in partnership with our digital health company.
Time frame: Three-week trial period
Population: 27 were enrolled (14 treatment, 13 control) and data was complete for 22
| Arm | Measure | Value (MEAN) |
|---|---|---|
| Enhanced Electronic Platform | The Difference in Data Collection Timeliness Between Groups Across Three Weeks | 4.69 percentage of intervals |
| Electronic Platform Not Enhanced | The Difference in Data Collection Timeliness Between Groups Across Three Weeks | 71.87 percentage of intervals |
Attitudes for Implementors
We adapted scales to measure mechanisms affecting data collection, including attitudes, using social psychology methods (Armitage & Connor, 2001). The attitudes measure had one item, which was rated from 1=good to 7=bad (meaning that taking data would be bad). Attitudes had 2 items. These items were averaged to create the scale score which could range from 1 to 7 for each measure. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time.
Time frame: Assessed at baseline and end of 3-week trial period
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Enhanced Electronic Platform | Attitudes for Implementors | 6.54 score on a scale | Standard Deviation 1.03 |
| Electronic Platform Not Enhanced | Attitudes for Implementors | 6.68 score on a scale | Standard Deviation 0.73 |
Descriptive Norms for Implementors
We adapted scales to measure mechanisms affecting data collection, including descriptive norms, using social psychology methods (Armitage & Connor, 2001). The perceived norms measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger influence of norms on taking data. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time.
Time frame: Assessed at baseline and end of 3-week trial period
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Enhanced Electronic Platform | Descriptive Norms for Implementors | 6.08 score on a scale | Standard Deviation 1.6 |
| Electronic Platform Not Enhanced | Descriptive Norms for Implementors | 6.27 score on a scale | Standard Deviation 1.27 |
Intentions for Implementors
We adapted scales to measure mechanisms affecting data collection, including intentions, using social psychology methods (Armitage & Connor, 2001). The intentions measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger intentions to take data. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time.
Time frame: Assessed at baseline and end of 3-week trial period
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Enhanced Electronic Platform | Intentions for Implementors | 5.31 score on a scale | Standard Deviation 2.59 |
| Electronic Platform Not Enhanced | Intentions for Implementors | 5.91 score on a scale | Standard Deviation 2.07 |
Perceived Norms for Implementors
We adapted scales to measure mechanisms affecting data collection, including perceived norms, using social psychology methods (Armitage & Connor, 2001). The perceived norms measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger influence of norms on taking data. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time.
Time frame: Assessed at baseline and end of 3-week trial period
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Enhanced Electronic Platform | Perceived Norms for Implementors | 5.62 score on a scale | Standard Deviation 2.14 |
| Electronic Platform Not Enhanced | Perceived Norms for Implementors | 6.18 score on a scale | Standard Deviation 1.83 |
Self-Efficacy for Implementors
We adapted scales to measure mechanisms affecting data collection, including self-efficacy, using social psychology methods (Armitage & Connor, 2001). The self-efficacy measure had one item, which was rated from 1=strongly disagree to 7=strongly agree. Strongly agree meant stronger self-efficacy about taking data. Self-Efficacy had 2 items. These items were averaged to create the scale score which could range from 1 to 7 for each measure. Aides completed measures at baseline and post-trial (3 weeks). Data reported are changes over time.
Time frame: Assessed at baseline and end of 3-week trial period
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Enhanced Electronic Platform | Self-Efficacy for Implementors | 5.5 score on a scale | Standard Deviation 1.72 |
| Electronic Platform Not Enhanced | Self-Efficacy for Implementors | 4.91 score on a scale | Standard Deviation 2.5 |
System Usability Scale
We used the System Usability Scale (SUS; Brooke, 1996) to measure aides' reactions to various statements regarding the app's usability with 10 items that use a 5-point scale from (1) strongly disagree to (5) strongly agree. The SUS has high internal consistent reliability (Cronbach's alpha = .91) and demonstrated sensitivity to change (Lewis, 2018). We adapted the original questionnaire to replace system with app across all items. The SUS is calculated as a proportion score out of 100, with higher scores indicating higher usability (≥85 = Excellent; ≥71 = Good; ≥51 = Okay (Bangor et al.).
Time frame: Three-week trial period
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Enhanced Electronic Platform | System Usability Scale | 76.4 score on a scale | Standard Deviation 15.2 |
| Electronic Platform Not Enhanced | System Usability Scale | 67.7 score on a scale | Standard Deviation 21.5 |