Cognitive Dysfunction, Cognitive Impairment, Cognitive Decline, Cognitive Change
Conditions
Brief summary
Cognitive impairment is a significant health problem in the United States, resulting in costs over $100 billion a year. We will provide an efficient, effective, and financially intelligent solution to Primary Care Physician's to identify cognitive impairment in the earliest stages, delay progression through appropriate treatment, and to afford patients the opportunity to make future plans at a time when symptoms are mild and patients are able to make informed decisions concerning financial and life activities. This has the potential to delay devastating effects of cognitive impairment, and to lessen the financial burden on the health care system in the United States.
Detailed description
Cognitive dysfunction in the elderly population, ranging from simple forgetfulness to a diagnosis of Alzheimer's disease, can impact one's quality of life and ability to function in daily activities. It is crucial that decline be detected as early as possible in order to evaluate whether the cause is treatable, and to employ appropriate treatment, if applicable. The majority of older patients rely on their primary care physician for the bulk of their healthcare needs, but there is a lack of sensitive tools available, and there is a lack of physician's time to use the tools, leading to a failure to provide therapeutic intervention at the earliest stages of loss to potentially slow the progression of disease. Psychology Software Tools, Inc. (PST) has developed the Computer Assessment of Memory and Cognitive Impairment (CAMCI), a computerized screening tool for detection of early signs of cognitive decline, which has been shown to be more effective in the identification of patients with subtle cognitive loss than the tools most frequently used within the primary care physician (PCP) office. CAMCI would provide an option for PCPs and clinicians to provide therapeutic intervention prior to a diagnosis of dementia. Recent additions to Current Procedural Terminology (CPT) codes permit insurance reimbursement for neuropsychological testing by a computer, including time for the physician's or clinical psychologist's interpretation and reporting. The introduction of this new revenue stream for PCPs and clinicians, coupled with the characteristics of being brief and self-administered make CAMCI an attractive option for improving early intervention, providing an intelligent business solution for healthcare professionals, and a useful and effective tool that allows physicians to better evaluate and serve their patients. The specific aims included in the current project focus on activities required to successfully move CAMCI to commercialization by extending support for late stage research and product development, including regulatory strategy and intellectual property development, data collection to replicate key studies, product extension through increasing minority representation, and development of a measure of meaningful change. The ultimate goal is to streamline commercialization of CAMCI, and to provide a useful and effective tool in the detection of cognitive dysfunction to physicians, the providers of the majority of healthcare to the elderly population, to improve efficiency and effectiveness of clinical practice.
Interventions
DEVICECAMCI
CAMCI is a computerized screening tool for the assessment of cognitive status. CAMCI accurately assesses cognitive performance using standard neuropsychological tests of memory, attention, and executive ability modified for computer administration, and an innovative Virtual Environment task, testing domains such as incidental memory, not easily assessed using paper-pencil tests. Computer-administered tasks ensure standard administration and scoring, avoiding inter-site and inter-examiner variability. The CAMCI battery consists of tasks testing multiple aspects of cognitive function, and a series of self-report questions administered via tablet computer. Using touchscreen technology for response input, CAMCI takes approximately 15-20 minutes to complete.
DIAGNOSTIC_TESTMoCA
The Montreal Cognitive Assessment (MoCA) is a rapid screening tool used to detect mild cognitive impairment (MCI), assessing cognitive domains (visuospatial skills, executive function, memory, attention, language, and orientation). Individual test scores are summed into a total score from 0 (worst) to 30 (best). Individuals may achieve any score within that range. Individual test scores are not reported. Scores: 26-30 considered Normal; 19-25 may suggest Mild Cognitive Impairment (MCI); 10-18 can suggest moderate impairment; \< 10 may indicate severe impairment
DIAGNOSTIC_TESTClinical Adjudication
Cognitive status assessed by traditional neuropsychological tests measuring visuospatial skills, executive function, memory, attention, language, orientation. Results compared to age-adjusted norms, reviewed by expert neuropsychologists for consensus classification, per Univ of Pittsburgh Alzheimers Disease Research Ctr and Ntl Alzheimers Coordinating Ctr, and Alzheimer's Assoc criteria. Final classification adjusted per clinical judgment. Impaired: ≥3 scores ≥2SD below norms Indeterminate: ≤2 scores \>1SD below norms Normal: Neither criteria met. Total and domain scores not reported.
Sponsors
National Institute on Aging (NIA)
Psychology Software Tools, Inc.
Study design
Allocation
NA
Intervention model
SINGLE_GROUP
Primary purpose
SCREENING
Masking
NONE
Eligibility
Sex/Gender
ALL
Age
60 Years to 95 Years
Healthy volunteers
Yes
Inclusion criteria
* Signed informed consent * Adequate visual and auditory acuity to allow neuropsychological testing * Able to read, write and understand study and test requirements * Within the age range of 60+
Exclusion criteria
* Significant neurologic disease, such as multi-infarct dementia, Parkinson's disease, epilepsy, stroke, multiple sclerosis or head trauma * History of major depression or other major psychiatric disorder, such as, schizophrenia and bipolar disorder * History of consuming 5 or more alcoholic drinks per day on a regular basis * Montreal Cognitive Assessment (MoCA) score \<10
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Agreement Analysis (Agreement to Reference Standard) - Test Group | baseline | Comparison (positive and negative percent agreement, confidence interval estimates, and quadratic weighted kappa) between CAMCI and clinical adjudication classifications for the Test group. Power calculations for agreement were assessed for Cohen Kappa's assuming at least 75% Normals with 80% power and .05 significance level, finding that we were sufficiently powered with at least 120 individuals in the test set, for a Kappa of at least 0.3. |
| Agreement Analysis (Agreement to Non-Reference Standard) - Test Group | baseline | Linear regression agreement analysis (linear regression equation and confidence intervals, and Pearson correlation) between CAMCI and Montreal Cognitive Assessment (MoCA) for the Test group. Comparison (positive and negative percent agreement, confidence interval estimates, and quadratic weighted kappa) between CAMCI and Montreal Cognitive Assessment (MoCA) for the Test group. CAMCI Scores ranged from 0-50, higher scores mean better outcome. MoCA scores ranged from 0-30, higher scores mean better outcome. Power calculations for agreement were assessed for Cohen Kappa's assuming at least 75% Normals with 80% power and .05 significance level, finding that we were sufficiently powered with at least 120 individuals in the test set, for a Kappa of at least 0.3. |
Other
| Measure | Time frame | Description |
|---|---|---|
| Change Metric (Measure of Significant Change) | baseline, 6 months, 12 months, 24 months post baseline | To derive a measure of reliable change that addressed unbalanced repeated measures and unequal timing of repeat measures, we built a two-stage hierarchical model for longitudinal data using Bayesian methods. Additionally, the standard deviation (SD) was allowed to change over time, which allowed for computation of an intraclass correlation coefficient (ICC), a measure of reliability. With an SD and ICC that can change over time, reliable change was computed for follow-up months from 3 to 24, in groups of 3. Reliable change was then adjusted for practice effects according to Duff, K. (2012). Computer Assessment of Memory and Cognitive Impairment (CAMCI) scores were calculated as the weighted sum of individual task scores converted to a total score from 0-worst to 50-best. As the intended use of CAMCI is to provide a single score of overall cognitive status, individual task scores were not reported. |
Countries
United States
Participant flow
Recruitment details
* Recruitment occurred at 4 participating sites representing a variety of regions across the US * Participants were recruited from both clinical and community settings. * Participants were required to be at least 60 years of age, have adequate visual and auditory acuity to allow neuropsychological tests * Exclusionary criteria included significant neurologic disease, history of major depression, psychiatric disorder, consuming 5 or more alcoholic drinks per day, MoCA score less than 10
Participants by arm
| Arm | Count |
|---|---|
| CAMCI Validation Validation of the Computer Assessment of Memory and Cognitive Impairment (CAMCI) against 1) clinical adjudication of traditional neuropsychological tests administered via paper and pencil and 2) the Montreal Cognitive Assessment (MoCA) tool for early detection of mild cognitive impairment (MCI) | 764 |
| Total | 764 |
Withdrawals & dropouts
| Period | Reason | FG000 |
|---|---|---|
| Overall Study | Age-ineligible | 1 |
| Overall Study | Did Not Qualify | 3 |
| Overall Study | Inadequate fluency with the English language | 3 |
| Overall Study | Protocol Violation | 2 |
Baseline characteristics
| Characteristic | CAMCI Validation |
|---|---|
| Age, Continuous | 71.0 years |
| Boston Naming Test | 52.20 points STANDARD_DEVIATION 7.36 |
| CAMCI Impaired (Score Range 0-15) | 72 Participants |
| CAMCI Indeterminate (Score Range 16-20) | 96 Participants |
| CAMCI Normal (Score Range 21-50) | 596 Participants |
| Ethnicity (NIH/OMB) Hispanic or Latino | 116 Participants |
| Ethnicity (NIH/OMB) Not Hispanic or Latino | 619 Participants |
| Ethnicity (NIH/OMB) Unknown or Not Reported | 29 Participants |
| Hopkins Verbal Learning Test - Revised (HVLT-R) - Recall Delayed Recall (20 minutes) | 7.25 points STANDARD_DEVIATION 3.21 |
| Hopkins Verbal Learning Test - Revised (HVLT-R) - Recall Learning Trial 1 Recall | 5.46 points STANDARD_DEVIATION 1.85 |
| Hopkins Verbal Learning Test - Revised (HVLT-R) - Recall Learning Trial 2 Recall | 7.68 points STANDARD_DEVIATION 2.16 |
| Hopkins Verbal Learning Test - Revised (HVLT-R) - Recall Learning Trial 3 Recall | 8.82 points STANDARD_DEVIATION 2.28 |
| Hopkins Verbal Learning Test - Revised (HVLT-R) - Recall Learning Trials Total Recall | 21.96 points STANDARD_DEVIATION 5.7 |
| Hopkins Verbal Learning Test - Revised (HVLT-R) - Recognition Recognition Discrimination Index | 9.53 count STANDARD_DEVIATION 2.3 |
| Hopkins Verbal Learning Test - Revised (HVLT-R) - Recognition Recognition - False Positives | 1.42 count STANDARD_DEVIATION 1.61 |
| Hopkins Verbal Learning Test - Revised (HVLT-R) - Recognition Recognition - True Positives | 10.94 count STANDARD_DEVIATION 1.45 |
| Hopkins Verbal Learning Test - Revised (HVLT-R) - Retention % | 77.77 percentage STANDARD_DEVIATION 28.47 |
| Letter Fluency (F-A-S) Test Fluency - A | 11.17 points STANDARD_DEVIATION 4.67 |
| Letter Fluency (F-A-S) Test Fluency - F | 12.52 points STANDARD_DEVIATION 4.49 |
| Letter Fluency (F-A-S) Test Fluency - S | 13.61 points STANDARD_DEVIATION 4.92 |
| Letter Fluency (F-A-S) Test Fluency Total | 37.29 points STANDARD_DEVIATION 12.7 |
| Montreal Cognitive Assessment (MoCA) Impaired (Score Range 10-18) | 39 Participants |
| Montreal Cognitive Assessment (MoCA) Indeterminate (Score Range 19-25) | 388 Participants |
| Montreal Cognitive Assessment (MoCA) Normal (Score Range 26-30) | 337 Participants |
| Neuropsychological classification Impaired | 67 Participants |
| Neuropsychological classification Indeterminate | 176 Participants |
| Neuropsychological classification Normal | 521 Participants |
| Race (NIH/OMB) American Indian or Alaska Native | 0 Participants |
| Race (NIH/OMB) Asian | 21 Participants |
| Race (NIH/OMB) Black or African American | 266 Participants |
| Race (NIH/OMB) More than one race | 43 Participants |
| Race (NIH/OMB) Native Hawaiian or Other Pacific Islander | 0 Participants |
| Race (NIH/OMB) Unknown or Not Reported | 0 Participants |
| Race (NIH/OMB) White | 434 Participants |
| Region of Enrollment United States | 764 participants |
| Rey-Osterrieth Complex Figure Test (ROCF) Delayed Recall - Long Delay (30 minutes) | 12.99 points STANDARD_DEVIATION 5.98 |
| Rey-Osterrieth Complex Figure Test (ROCF) Figure Copy | 28.31 points STANDARD_DEVIATION 5.61 |
| Rey-Osterrieth Complex Figure Test (ROCF) Immediate Recall - Short Delay (3 minutes) | 13.24 points STANDARD_DEVIATION 5.99 |
| Semantic Fluency Test (Animals) | 18.65 points STANDARD_DEVIATION 5.15 |
| Sex: Female, Male Female | 550 Participants |
| Sex: Female, Male Male | 214 Participants |
| Trail Making Test - Completion Time Part A Time (150 seconds max) | 37.28 time in seconds STANDARD_DEVIATION 18.79 |
| Trail Making Test - Completion Time Part B Time (300 seconds max) | 106.16 time in seconds STANDARD_DEVIATION 63.65 |
| Trail Making Test - Score Part A Score | 24.00 points STANDARD_DEVIATION 0.05 |
| Trail Making Test - Score Part B Score | 23.54 points STANDARD_DEVIATION 2.45 |
| WAIS-IV Digit Span Backward Max Span (0-8) - number of digits recalled on the final correct trial | 4.40 points STANDARD_DEVIATION 1.34 |
| WAIS-IV Digit Span Backward Score (0-16) - total of points obtained on each trial/span (0,1,2) | 7.77 points STANDARD_DEVIATION 2.36 |
| WAIS-IV Digit Span Forward Max Span (0-9) - number of digits recalled on the final correct trial | 6.23 points STANDARD_DEVIATION 1.3 |
| WAIS-IV Digit Span Forward Score (0-16) - total of points obtained on each trial/span (0,1,2) | 9.42 points STANDARD_DEVIATION 2.23 |
| WAIS-IV Digit Symbol Substitution Test (DSST) | 52.36 points STANDARD_DEVIATION 15.45 |
| WMS-IV Logical Memory II - Delayed Recall Delayed Recall Total (adult), Score Range (0-50) | 21.93 points STANDARD_DEVIATION 8.27 |
| WMS-IV Logical Memory II - Delayed Recall Delayed Recall Total (older adult), Score Range (0-39) | 17.93 points STANDARD_DEVIATION 8.1 |
| WMS-IV Logical Memory II - Delayed Recall Story A Delayed Recall (older adult), Score Range (0-14) | 7.94 points STANDARD_DEVIATION 3.72 |
| WMS-IV Logical Memory II - Delayed Recall Story B Delayed Recall (adult), Score Range (0-25) | 11.57 points STANDARD_DEVIATION 4.69 |
| WMS-IV Logical Memory II - Delayed Recall Story B Delayed Recall (older adult), Score Range (0-25) | 9.99 points STANDARD_DEVIATION 5.09 |
| WMS-IV Logical Memory II - Delayed Recall Story C Delayed Recall (adult), Score Range (0-25) | 10.37 points STANDARD_DEVIATION 4.28 |
| WMS-IV Logical Memory I - Immediate Recall Immediate Recall Total (adult), Score Range (0-50) | 25.32 points STANDARD_DEVIATION 7.36 |
| WMS-IV Logical Memory I - Immediate Recall Immediate Recall Total (older adult), Score Range (0-53) | 31.27 points STANDARD_DEVIATION 8.69 |
| WMS-IV Logical Memory I - Immediate Recall Story A 1st Immediate Recall (older adult), Score Range (0-14) | 8.59 points STANDARD_DEVIATION 2.72 |
| WMS-IV Logical Memory I - Immediate Recall Story A 2nd Immediate Recall (older adult), Score Range (0-14) | 11.16 points STANDARD_DEVIATION 2.94 |
| WMS-IV Logical Memory I - Immediate Recall Story B Immediate Recall (adult), Score Range (0-25) | 13.48 points STANDARD_DEVIATION 4.17 |
| WMS-IV Logical Memory I - Immediate Recall Story B Immediate Recall (older adult), Score Range (0-25) | 11.52 points STANDARD_DEVIATION 4.68 |
| WMS-IV Logical Memory I - Immediate Recall Story C Immediate Recall (adult), Score Range (0-25) | 11.84 points STANDARD_DEVIATION 3.86 |
| WRAT5 - Word Reading | 59.59 points STANDARD_DEVIATION 8.33 |
Adverse events
| Event type | EG000 affected / at risk |
|---|---|
| deaths Total, all-cause mortality | 5 / 773 |
| other Total, other adverse events | 1 / 773 |
| serious Total, serious adverse events | 6 / 773 |
Outcome results
Primary
Agreement Analysis (Agreement to Non-Reference Standard) - Test Group
Linear regression agreement analysis (linear regression equation and confidence intervals, and Pearson correlation) between CAMCI and Montreal Cognitive Assessment (MoCA) for the Test group. Comparison (positive and negative percent agreement, confidence interval estimates, and quadratic weighted kappa) between CAMCI and Montreal Cognitive Assessment (MoCA) for the Test group. CAMCI Scores ranged from 0-50, higher scores mean better outcome. MoCA scores ranged from 0-30, higher scores mean better outcome. Power calculations for agreement were assessed for Cohen Kappa's assuming at least 75% Normals with 80% power and .05 significance level, finding that we were sufficiently powered with at least 120 individuals in the test set, for a Kappa of at least 0.3.
Time frame: baseline
Population: Test group. Of 773 participants enrolled, 9 were excluded, leaving 764 for analysis. Participants who completed CAMCI version A at baseline (N=680) were assigned through stratified random sampling to Training (75%) or Test (25%) groups, balanced by age, education, race, ethnicity, gender, and cognitive classification. The groups were further adjusted to match the racial and ethnic distribution in the U.S., resulting in a Training group (N=374) and a Test group (N=126).
| Arm | Measure | Group | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|---|
| Clinical Adjudication - Normal | Agreement Analysis (Agreement to Non-Reference Standard) - Test Group | CAMCI - Indeterminate | 1 Participants |
| Clinical Adjudication - Normal | Agreement Analysis (Agreement to Non-Reference Standard) - Test Group | CAMCI - Normal | 54 Participants |
| Clinical Adjudication - Normal | Agreement Analysis (Agreement to Non-Reference Standard) - Test Group | CAMCI - Impaired | 2 Participants |
| Clinical Adjudication - Indeterminate | Agreement Analysis (Agreement to Non-Reference Standard) - Test Group | CAMCI - Indeterminate | 12 Participants |
| Clinical Adjudication - Indeterminate | Agreement Analysis (Agreement to Non-Reference Standard) - Test Group | CAMCI - Normal | 40 Participants |
| Clinical Adjudication - Indeterminate | Agreement Analysis (Agreement to Non-Reference Standard) - Test Group | CAMCI - Impaired | 11 Participants |
| Clinical Adjudication - Impaired | Agreement Analysis (Agreement to Non-Reference Standard) - Test Group | CAMCI - Normal | 2 Participants |
| Clinical Adjudication - Impaired | Agreement Analysis (Agreement to Non-Reference Standard) - Test Group | CAMCI - Impaired | 4 Participants |
| Clinical Adjudication - Impaired | Agreement Analysis (Agreement to Non-Reference Standard) - Test Group | CAMCI - Indeterminate | 0 Participants |
95% CI: [0.3892, 0.6091]
95% CI: [0.4644, 0.644]
95% CI: [0.2401, 0.5461]
Primary
Agreement Analysis (Agreement to Reference Standard) - Test Group
Comparison (positive and negative percent agreement, confidence interval estimates, and quadratic weighted kappa) between CAMCI and clinical adjudication classifications for the Test group. Power calculations for agreement were assessed for Cohen Kappa's assuming at least 75% Normals with 80% power and .05 significance level, finding that we were sufficiently powered with at least 120 individuals in the test set, for a Kappa of at least 0.3.
Time frame: baseline
Population: Test group. Of 773 participants enrolled, 9 were excluded, leaving 764 for analysis. Participants who completed CAMCI version A at baseline (N=680) were assigned through stratified random sampling to Training (75%) or Test (25%) groups, balanced by age, education, race, ethnicity, gender, and cognitive classification. The groups were further adjusted to match the racial and ethnic distribution in the U.S., resulting in a Training group (N=374) and a Test group (N=126).
| Arm | Measure | Group | Value (COUNT_OF_PARTICIPANTS) |
|---|---|---|---|
| Clinical Adjudication - Normal | Agreement Analysis (Agreement to Reference Standard) - Test Group | CAMCI - Indeterminate (Sores 16-20) | 8 Participants |
| Clinical Adjudication - Normal | Agreement Analysis (Agreement to Reference Standard) - Test Group | CAMCI - Normal (Scores 21-50) | 76 Participants |
| Clinical Adjudication - Normal | Agreement Analysis (Agreement to Reference Standard) - Test Group | CAMCI - Impaired (Scores 0-15) | 3 Participants |
| Clinical Adjudication - Indeterminate | Agreement Analysis (Agreement to Reference Standard) - Test Group | CAMCI - Indeterminate (Sores 16-20) | 4 Participants |
| Clinical Adjudication - Indeterminate | Agreement Analysis (Agreement to Reference Standard) - Test Group | CAMCI - Normal (Scores 21-50) | 18 Participants |
| Clinical Adjudication - Indeterminate | Agreement Analysis (Agreement to Reference Standard) - Test Group | CAMCI - Impaired (Scores 0-15) | 11 Participants |
| Clinical Adjudication - Impaired | Agreement Analysis (Agreement to Reference Standard) - Test Group | CAMCI - Normal (Scores 21-50) | 2 Participants |
| Clinical Adjudication - Impaired | Agreement Analysis (Agreement to Reference Standard) - Test Group | CAMCI - Impaired (Scores 0-15) | 3 Participants |
| Clinical Adjudication - Impaired | Agreement Analysis (Agreement to Reference Standard) - Test Group | CAMCI - Indeterminate (Sores 16-20) | 1 Participants |
95% CI: [0.569, 0.7408]
95% CI: [0.2791, 0.6101]
Other Pre-specified
Change Metric (Measure of Significant Change)
To derive a measure of reliable change that addressed unbalanced repeated measures and unequal timing of repeat measures, we built a two-stage hierarchical model for longitudinal data using Bayesian methods. Additionally, the standard deviation (SD) was allowed to change over time, which allowed for computation of an intraclass correlation coefficient (ICC), a measure of reliability. With an SD and ICC that can change over time, reliable change was computed for follow-up months from 3 to 24, in groups of 3. Reliable change was then adjusted for practice effects according to Duff, K. (2012). Computer Assessment of Memory and Cognitive Impairment (CAMCI) scores were calculated as the weighted sum of individual task scores converted to a total score from 0-worst to 50-best. As the intended use of CAMCI is to provide a single score of overall cognitive status, individual task scores were not reported.
Time frame: baseline, 6 months, 12 months, 24 months post baseline
Population: Individuals who received a consistent version of CAMCI across all visits (i.e., CAMCI-A), and were classified as Normal through clinical adjudication at all visits.
| Arm | Measure | Group | Value (MEAN) |
|---|---|---|---|
| Clinical Adjudication - Normal | Change Metric (Measure of Significant Change) | 0-6 Months | 11.33 score on a scale |
| Clinical Adjudication - Normal | Change Metric (Measure of Significant Change) | 0-12 Months | 11.32 score on a scale |
| Clinical Adjudication - Normal | Change Metric (Measure of Significant Change) | 6-24 Months | 11.38 score on a scale |
| Clinical Adjudication - Normal | Change Metric (Measure of Significant Change) | 12-24 Months | 11.42 score on a scale |