Systematic Review
Conditions
Keywords
abstract, meta-analysis, systematic review, interpretation, results, quality, bias, spin, limits, limitation, risk of bias
Brief summary
Objective: The investigators aim to evaluate the impact of a adding a limitations section in systematic review's abstract on result's interpretation by readers Design: randomized controlled trial with two parallel arms. Participants will be invited to participate in an online survey. Participants: Eligible participants are corresponding authors of clinical trials published between 2010 and 2012 and referenced in Medline. Intervention: The investigators will evaluate the impact of the presence of a limitations section in abstract of systematic review with meta- analysis. The investigators selected abstracts of meta-analysis from a sample. Selected abstract will be standardised and the treatment's name hidden. Two groups of abstract will be presented as follow: 1) abstract without limitations section 2) abstract with limitations section. The limitations section will be written by authors and will briefly describe the risk of bias of included studies. Selected participant are invited by e-mail to answer the survey. After reading one abstract from his/her group of randomisation they are invited to answer 5 questions about their interpretation of the results.
Interventions
BEHAVIORALinterpretation of the abstract of the systematic review
Sponsors
Assistance Publique - Hôpitaux de Paris
Study design
Allocation
RANDOMIZED
Intervention model
PARALLEL
Primary purpose
OTHER
Masking
SINGLE (Subject)
Eligibility
Sex/Gender
ALL
Healthy volunteers
No
Inclusion criteria
* corresponding authors of clinical trail indexed in pubmed with email address available
Design outcomes
Primary
| Measure | Time frame | Description |
|---|---|---|
| Confidence in the Results of the Systematic Review | 1 month | we ask participants: how confident are you in the results of this study?, with the choice of answer based on a 10 points Likert scale (score from 0, not at all confident, to 10, very confident ) |
Secondary
| Measure | Time frame | Description |
|---|---|---|
| Confidence With the Validity of the Conclusions | 1 month | We ask participants: \- How confident are you in the validity of the conclusions of this study? with the choice of answer based on a 10 points Likert scale from 0, not at all confident, to 10, very confident |
| Beneficial Effect of the Experimental Intervention | 1 month | We ask participants: \- How confident are you that the intervention A could be of benefit to patients? with the choice of answer based on a 10 points Likert scale from 0, not at all confident, to 10, very confident |
| Influence of Results on Clinical Practice | 1 month | We ask participants: How confident are you that the results of this study could influence your clinical practice? with the choice of answer based on a 10 points Likert scale from 0, not at all confident, to 10, very confident |
| The Rigor of the Systematic Review | 1 month | We ask participants: Do you think that this systematic review was conducted rigorously? with the choice of answer based on a 10 points Likert scale from 0, not at all confident, to 10, very confident |
Countries
France
Participant flow
Participants by arm
| Arm | Count |
|---|---|
| Abstract With Limitation Section Added we add a limitation section in each selected abstract, the limitation section will focus on the quality of included studies.
interpretation of the abstract of the systematic review | 150 |
| Abstract Without Limitation Section We selected 30 abstracts with a conclusion in favour the experimental treatment from a sample of systematic reviews that evaluate the effect of health care intervention.
interpretation of the abstract of the systematic review | 150 |
| Total | 300 |
Baseline characteristics
| Characteristic | Abstract With Limitation Section Added | Abstract Without Limitation Section | Total |
|---|---|---|---|
| Age, Continuous | 45.2 years STANDARD_DEVIATION 10.3 | 46.7 years STANDARD_DEVIATION 10.9 | 46.0 years STANDARD_DEVIATION 10.7 |
| Authored at least one systematic review | 77 Participants | 82 Participants | 159 Participants |
| Clinical experience >10 years | 110 Participants | 114 Participants | 224 Participants |
| Clinical experience 5 to 10 years | 34 Participants | 28 Participants | 62 Participants |
| Clinical experience <5 years | 6 Participants | 8 Participants | 14 Participants |
| Education qualification MD | 78 Participants | 78 Participants | 156 Participants |
| Education qualification Other | 14 Participants | 8 Participants | 22 Participants |
| Education qualification PhD | 58 Participants | 64 Participants | 122 Participants |
| No. of randomized trials involved in 0 trial | 16 Participants | 11 Participants | 27 Participants |
| No. of randomized trials involved in >10 trials | 41 Participants | 37 Participants | 78 Participants |
| No. of randomized trials involved in 1-3 trials | 48 Participants | 48 Participants | 96 Participants |
| No. of randomized trials involved in 4-9 trials | 45 Participants | 54 Participants | 99 Participants |
| No. of systematic reviews peer-reviewed 0 review | 75 Participants | 64 Participants | 139 Participants |
| No. of systematic reviews peer-reviewed 1-3 reviews | 55 Participants | 69 Participants | 124 Participants |
| No. of systematic reviews peer-reviewed >3 reviews | 20 Participants | 16 Participants | 36 Participants |
| Reading reports of systematic reviews Rarely or sometimes | 61 Participants | 62 Participants | 123 Participants |
| Reading reports of systematic reviews Regularly | 89 Participants | 87 Participants | 176 Participants |
| Sex: Female, Male Female | 41 Participants | 43 Participants | 84 Participants |
| Sex: Female, Male Male | 109 Participants | 107 Participants | 216 Participants |
| Training in clinical epidemiology | 101 Participants | 76 Participants | 177 Participants |
| Training in methods of randomized trials | 86 Participants | 71 Participants | 157 Participants |
Adverse events
| Event type | EG000 affected / at risk | EG001 affected / at risk |
|---|---|---|
| deaths Total, all-cause mortality | 0 / 150 | 0 / 150 |
| other Total, other adverse events | 0 / 150 | 0 / 150 |
| serious Total, serious adverse events | 0 / 150 | 0 / 150 |
Outcome results
Primary
Confidence in the Results of the Systematic Review
we ask participants: how confident are you in the results of this study?, with the choice of answer based on a 10 points Likert scale (score from 0, not at all confident, to 10, very confident )
Time frame: 1 month
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Abstract With Limitation Section Added | Confidence in the Results of the Systematic Review | 4.6 score on a scale | Standard Deviation 2.5 |
| Abstract Without Limitation Section | Confidence in the Results of the Systematic Review | 4.4 score on a scale | Standard Deviation 2.3 |
p-value: 0.595% CI: [-0.4, 0.7]Mixed Models Analysis
Secondary
Beneficial Effect of the Experimental Intervention
We ask participants: \- How confident are you that the intervention A could be of benefit to patients? with the choice of answer based on a 10 points Likert scale from 0, not at all confident, to 10, very confident
Time frame: 1 month
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Abstract With Limitation Section Added | Beneficial Effect of the Experimental Intervention | 4.4 score on a scale | Standard Deviation 2.6 |
| Abstract Without Limitation Section | Beneficial Effect of the Experimental Intervention | 4.3 score on a scale | Standard Deviation 2.3 |
p-value: 0.695% CI: [-0.4, 0.7]Mixed Models Analysis
Secondary
Confidence With the Validity of the Conclusions
We ask participants: \- How confident are you in the validity of the conclusions of this study? with the choice of answer based on a 10 points Likert scale from 0, not at all confident, to 10, very confident
Time frame: 1 month
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Abstract With Limitation Section Added | Confidence With the Validity of the Conclusions | 4.1 score on a scale | Standard Deviation 2.5 |
| Abstract Without Limitation Section | Confidence With the Validity of the Conclusions | 4.0 score on a scale | Standard Deviation 2.3 |
p-value: 0.895% CI: [-0.5, 0.6]Mixed Models Analysis
Secondary
Influence of Results on Clinical Practice
We ask participants: How confident are you that the results of this study could influence your clinical practice? with the choice of answer based on a 10 points Likert scale from 0, not at all confident, to 10, very confident
Time frame: 1 month
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Abstract With Limitation Section Added | Influence of Results on Clinical Practice | 3.8 score on a scale | Standard Deviation 2.6 |
| Abstract Without Limitation Section | Influence of Results on Clinical Practice | 3.8 score on a scale | Standard Deviation 2.3 |
p-value: 0.895% CI: [-0.6, 0.5]Mixed Models Analysis
Secondary
The Rigor of the Systematic Review
We ask participants: Do you think that this systematic review was conducted rigorously? with the choice of answer based on a 10 points Likert scale from 0, not at all confident, to 10, very confident
Time frame: 1 month
| Arm | Measure | Value (MEAN) | Dispersion |
|---|---|---|---|
| Abstract With Limitation Section Added | The Rigor of the Systematic Review | 4.1 score on a scale | Standard Deviation 2.7 |
| Abstract Without Limitation Section | The Rigor of the Systematic Review | 4.4 score on a scale | Standard Deviation 2.6 |
p-value: 0.295% CI: [-1, 0.2]Mixed Models Analysis