Logo-ija
Int J Aging. 2024;2:e20. doi: 10.34172/ija.2024.e20

Review Article

Quality Evaluation of Included Randomized Controlled Trials in Cochrane’s Urinary Incontinence Systematic Reviews Group

Sakineh Hajebrahimi 1, 2 ORCID logo, Amirreza Mosayebzade 1, Morteza Atayi 1, Nasim Mahdavi 1, Amin Bidar 3, Hanieh Salehi-Pourmehr 1, * ORCID logo

Author information:
1Research Center for Evidence-Based Medicine, Iranian EBM Centre: A JBI Centre of Excellence, Faculty of Medicine, Tabriz University of Medical Sciences, Tabriz, Iran
2Department of Urology, Tabriz University of Medical Sciences, Tabriz, Iran
3Nursing Student, Khalkhal University of Medical Science, Ardabil, Iran

*Corresponding Author: Hanieh Salehi-Pourmehr, Emails: poormehrh@yahoo.com; salehih@tbzmed.ac.ir

Abstract

Objectives:

To evaluate the quality of systematic reviews and meta-analyses conducted by the Cochrane Urology Group on urinary incontinence (UI).

Design:

A systematic review.

Setting(s):

Cochrane Urology Group on UI.

Participants:

37 systematic reviews, which included a total of 611 randomized controlled trials (RCTs) based on searches until July 2023.

Outcome measures:

Quality of systematic reviews and meta-analyses.

Results:

The most common risk of bias in the included RCTs was related to the blinding of participants and personnel, also known as performance bias. The findings also highlighted the prevalence and impact of UI, a condition that is often underreported due to social stigma. Our results emphasize the importance of maintaining high-quality studies in the Cochrane Library, which is pivotal in enhancing medical knowledge and facilitating improved clinical decision-making. Our findings underscore the need for the rigorous evaluation of the methodological quality of studies, a crucial step in selecting the superior clinical literature.

Conclusions:

Despite significant enhancements in the quality of studies, there is still a considerable distance from achieving an ideal RCT.

Keywords: Quality assessment, Risk of bias, Urinary incontinence, Cochrane urology group

Copyright and License Information

© 2024 The Author(s).
This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Funding Statement

None.

Please cite this article as follows: Hajebrahimi S, Mosayebzade A, Atayi M, Mahdavi N, Bidar A, Salehi-Pourmehr H. Quality evaluation of included randomized controlled trials in cochrane’s urinary incontinence systematic reviews group. Int J Aging. 2024;2: e20. doi: 10.34172/ija.2024.e20


Introduction

Over the past decade, there has been a significant surge in the number of medical papers and journals. This proliferation has inundated us with a vast amount of data and information, making it nearly impossible to stay current. Furthermore, the escalating quantity of studies could potentially compromise the standards and methods of reporting. It is unequivocally evident that reviews derived from low-quality studies can adversely affect decision-making for patient care, both nationally and globally. According to the hierarchy of evidence, systematic reviews and meta-analyses are considered the most reliable sources of information. The Cochrane Collaboration is a renowned organization that maintains a comprehensive library of high-quality systematic reviews, organized into 53 distinct review groups, each focusing on a specific topic. These meticulously conducted studies aim to enhance medical knowledge and facilitate optimal medical decision-making. The Cochrane urology group, in particular, has undertaken numerous systematic reviews on our disease of interest, namely, urinary incontinence (UI).1 UI, defined by a joint report from the International Urogynecological Association and the International Continence Society as the involuntary loss of urine, is a prevalent medical condition affecting individuals of all ages and across diverse racial backgrounds.1-3 Despite its prevalence, UI is often underreported due to associated embarrassment and social stigma. This condition is able to impact an individual’s quality of life significantly, but the effects can be substantially mitigated with proper evaluation, treatment, and management.4 It is noteworthy that UI is more common in women.5 The primary types of UI include stress incontinence, which involves any leakage of urine following an increase in abdominal pressure due to actions such as coughing, laughing, sneezing, or physical activity. Urge incontinence is characterized by the immediate loss of urine following a sudden urge to urinate, and mixed incontinence demonstrates symptoms of both urge incontinence and stress incontinence simultaneously.6 Given the prevalence and significance of UI, a vast number of studies have been conducted on this topic by the Cochrane urology group. The current study seeks to assess the quality of studies incorporated in the Cochrane systematic reviews.


Methods

The current systematic review study included 37 systematic reviews that have been conducted by the Cochrane urology group on UI until July 2023. Considering that the collaborated randomized controlled trials (RCTs) in systematic reviews have already been performed by informed consent, it was unnecessary to do the same. At the time of searching, the Cochrane Library consisted of 197 Cochrane reviews and 37 protocols in the urology subgroup. The database was searched for UI-related articles, and the inclusion and exclusion criteria were checked as well. The protocols and trials were excluded from all results regarding UI, and 37 systematic reviews were included in our study. Related data, including the title, publication year, author, and study setting of each study, were extracted and organized. The assessment of risk of bias for each systematic review was handled by two independent researchers according to the Joanna Briggs Institute (JBI) risk of bias assessment tool for systematic reviews. Disagreements were solved by a third party. After all meta-analyses and systematic reviews were appraised, RCTs from each systematic review were assessed for different potential biases based on the JBI risk of bias assessment tool for RCTs. The collected data were imported into Excel and reported with descriptive results. The current study has reported the data based on the Preferred Reporting Items for Systematic Review and Meta-analysis (PRISMA) guideline, which is a standard for reporting systematic reviews conducted on RCTs to avoid missing information and settings that are necessary to mention. In addition, this guideline can be used as a cornerstone of systematic review reporting.


Results

Our study incorporated 37 systematic reviews, encompassing a total of 611 RCTs. Based on the quality of systematic reviews and meta-analyses and their understudied RCTs utilizing the JBI risk of bias assessment tool for systematic reviews, this study primarily aimed to ascertain whether the studies posed a well-defined research question (Table 1). The responses to these inquiries were categorized as “Yes”, “No”, “Unclear”, or “Not applicable”. Subsequently, the studies were classified into one of three “Included”, “Excluded”, or “Seek further info” categories. Upon evaluation, it was found that all the included systematic reviews posed appropriate research questions (Table 2). The specifics of these systematic reviews are delineated in Table 3. In addition, PRISMA guidelines have been used in most of the included systematic reviews. Our analysis indicated that the domains of allocation concealment (selection bias) and blinding of outcome assessor (detection bias) most frequently exhibited unclear results, thereby posing the most common risk of bias. The domain of blinding of participants and personnel (performance bias) had the highest risk of bias, while the least risk of bias was attributed to the random sequence generation (selection bias) domain.


Table 1. Assessing the Quality of Studies Using the JBI Checklist
No. Author – Year Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11
1 Temtanakitpaisan, 20227 Yes Yes Yes Yes Yes Yes Yes N/A Yes Yes Yes
2 Saraswat, 20208 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
3 Freites, 20199 Yes Yes Yes Yes Yes Yes Yes N/A Yes Yes Yes
4 Bakali, 201910 Yes Yes Yes Yes Yes Yes Yes N/A N/A Yes Yes
5 Buckley, 201911 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
6 Wieland, 201912 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
7 Thomas, 201913 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
8 Baessler, 201814 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
9 Dumoulin, 201815 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
10 Nambiar, 201716 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
11 Glazener, 201717 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
12 Stewart, 201718 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
13 Lapitan, 201719 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
14 Kang, 201520 Yes Yes Yes Yes Yes Yes Yes N/A N/A Yes Yes
15 Ayeleke, 201521 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
16 Anderson, 201522 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
17 Imamura, 201523 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
18 Silva, 201424 Yes Yes Yes Yes Yes Yes Yes N/A N/A Yes Yes
19 Utomo, 201425 Yes Yes Yes Yes Yes Yes Yes N/A Yes Yes Yes
20 Lipp, 201426 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
21 Herbison, 201327 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
22 Wang, 201328 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
23 Berghamns, 201329 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
24 Clement, 201330 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
25 Rai, 201231 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
26 Cody, 2012 (1)32 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
27 Cody, 2012 (2)33 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
28 Hay-smith, 201134 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
29 Herderschee, 201135 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
30 Fader, 200836 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
31 Fader, 200737 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
32 Mariappan, 200538 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes
33 Alhasso, 200539 Yes Yes Yes Yes Yes Yes Yes Yes N/A Yes Yes
34 Ostaszkiewicz, 2004 (1)40 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes
35 Ostaszkiewicz, 2004 (2)41 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes
36 Wallace, 200442 Yes Yes Yes Yes Yes Yes Yes Yes No Yes Yes
37 Eustice, 200043 Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes Yes

Note. N/A: Not applicable. JBI: Joanna Briggs Institute. 1. Is the review question clearly and explicitly stated? 2. Were the inclusion criteria appropriate for the review question? 3. Was the search strategy appropriate? 4. Were the sources and resources used to search for studies adequate? 5. Were the criteria for appraising studies appropriate? 6. Was critical appraisal conducted by two or more reviewers independently? 7. Were there methods to minimize errors in data extraction? 8. Were the methods used to combine studies appropriate? 9. Was the likelihood of publication bias assessed? 10. Were recommendations for policy and/or practice supported by the reported data? 11. Were the specific directives for new research appropriate?


Table 2. Objectives and Clinical Questions of Cochrane Systematic Review Studies
No. Study Aims
1 Temtanakitpaisan, 20227 Prescribing prophylactic antibiotics for preventing infection after continence surgery in women with stress urinary incontinence (SUI)
2 Saraswat, 2020)8 Assessing the efficacy of traditional sub-urethral slings in treating SUI in women, with a summary of related economic evaluation findings
3 Freites, 20199 Investigating the impact of laparoscopic colposuspension on UI in women and summarizing the key findings from associated economic evaluations
4 Bakali, 201910 Examining the effects of interventions for recurrent SUI following unsuccessful minimally invasive surgery using artificial midurethral tape in women, along with a summary of principal findings from economic evaluations
5 Buckley, 201911 Analyzing the impact of conservative interventions on daytime functional UI in children
6 Wieland, 201912 Studying the effects of yoga on the treatment of UI in women
7 Thomas, 201913 Evaluating the impact of interventions on UI in adults at least one month post-stroke
8 Baessler, 201814 Determining the influence on bladder function post-surgery for symptomatic pelvic organ prolapse, with or without concurrent or delayed two-stage measures for treating or preventing SUI
9 Dumoulin, 201815 Assessing the effects of pelvic floor muscle training for women with UI in comparison to no treatment, placebo, sham treatments, or other inactive control treatments, along with a summary of related economic evaluation findings
10 Nambiar, 201716 Evaluating the efficacy of mini-sling methods in women with clinical urodynamic SUI or mixed UI (MUI) in terms of enhancing urinary control status, quality of life, and side effects
11 Glazener, 201717 Determining the impact of needle suspension on SUI or MUI compared to other treatment alternatives
12 Stewart, 201718 Assessing the effects of electrical stimulation with non-implanted devices, either alone or combined with other treatments, for managing SUI or stress-inducing MUI in women, including cost-effectiveness review results
13 Lapitan, 201719 Investigating the effects of open retropubic colposuspension on treating UI in women. A secondary objective was to assess the safety of open retropubic colposuspension in terms of resultant side effects
14 Kang, 201520 Evaluating the effectiveness of collagen denaturation with transurethral radiofrequency fork compared to other interventions in treating women with UI
15 Ayeleke, 201521 Comparing the effects of pelvic floor muscle exercise in conjunction with another active treatment versus the same active treatment alone in managing women with UI
16 Anderson, 201522 Determining the effectiveness of conservative management for UI up to 12 months post-prostatectomy via urethra, suprapubic, laparoscopic, radical retropubic, or perineal, including any individual conservative treatment or any combination of conservative treatments
17 Imamura, 201523 Examining the effectiveness of specific lifestyle interventions (e.g., weight loss, dietary changes, fluid intake, reduction of caffeinated, carbonated, and alcoholic beverages, avoidance of constipation, smoking cessation, and physical activity) in managing UI in adults
18 Silva, 201424 Determining the effects of surgical treatment on UI potentially due to sphincter inefficiency post-prostate surgery
19 Utomo, 201425 Evaluating the effectiveness of various surgical treatments for the functional obstruction of the bladder outlet in adults with neurogenic bladder dysfunction
20 Lipp, 201426 Determining the utility of mechanical devices in managing UI in adult women
21 Herbison, 201327 Investigating the effectiveness of vaginal cones in managing SUI in women
22 Wang, 201328 Assessing the efficacy and side effects of acupuncture in treating SUI in adults
23 Berghamns, 201329 Evaluating the effectiveness of electrical stimulation using non-implanted devices for men with stress incontinence, urgency, or MUI compared to no treatment, placebo treatment, or any other single treatment
24 Clement, 201330 Examining whether a treatment approach based on urodynamic diagnosis, as opposed to one based on history and examination, results in more effective clinical care and improved outcomes for individuals with UI
25 Rai, 201231 Comparing the impact of anticholinergic drugs with various non-drug treatments for non-neurogenic overactive bladder syndrome in adults
26 Cody, 2012 (1)32 Evaluating the effects of both topical and systemic estrogens used in the treatment of UI
27 Cody, 2012 (2)33 Determining the optimal method to enhance or replace the function of the lower urinary device using parts of the intestine when the bladder needs to be removed or has become nonfunctional or hazardous due to illness
28 Hay-smith, 201134 Comparing the effects of different pelvic floor muscle exercise approaches on women with UI
29 Herderschee, 201135 Investigating whether feedback or biofeedback enhances the benefits of pelvic floor muscle training for women with UI and comparing the effectiveness of different forms of feedback or biofeedback
30 Fader, 200836 Evaluating the effectiveness of various types of absorbent products designed for managing moderate to severe incontinence
31 Fader, 200737 Assessing the effectiveness of different designs of absorbent products for women with light UI
32 Mariappan, 200538 Determining whether serotonin-norepinephrine reuptake inhibitors in the treatment of women with SUI or MUI, which includes stress incontinence, are more effective than placebo (or no treatment, other drug and non-drug treatments, or surgery), and identifying the optimal dosage to be used
33 Alhasso, 200539 Determining the effectiveness of adrenergic agonists in the treatment of UI in adults
34 Ostaszkiewicz, 2004 (1)40 Evaluating the impact of habit retraining on managing UI in adults
35 Ostaszkiewicz, 2004 (2)41 Assessing the effects of timed voiding on managing UI in adults who are unable to independently use the toilet
36 Wallace, 200442 Evaluating the impact of bladder training on the treatment of UI
37 Eustice, 200043 Evaluating the effects of prompted voiding in managing UI in adults

Table 3. Number of Different Biases in the Articles Included in This Study
No. Study No of
Included
RCTs
(Sample
Size)
Random Sequence Generation (Selection Bias) Allocation
Concealment
(Selection
Bias)
Blinding of Participants and Personnel
(Performance Bias)
Blinding of
Outcome
Assessor
(Detection
Bias)
Incomplete
Outcome
Data
(Attrition
Bias)
Selective
Reporting
(Reporting
Data)
Blinding
(Performance and Detection
Bias)
Other Bias
Low High Unclear Low High Unclear Low High Unclear Low High Unclear Low High Unclear Low High Unclear Low High Unclear Low High Unclear
1 Temtanakitpaisan, 20227 3 (144) 2 1 1 1 1 1 2 3 3 2 1 3
2 Saraswat, 20208 34 (3244) 11 3 20 7 2 25 27 3 4 2 3 29
3 Freites, 20199 26 (2271) 12 1 13 11 1 14 1 25 3 1 22 19 7 18 1 7 17 9
4 Bakali, 201910 1 (46) 1 1 1 1 1 1 1
5 Buckley, 201911 27 (1803) 14 13 7 20 5 16 6 7 9 11 20 5 2 19 4 4 23 4
6 Wieland, 201912 2 (49) 2 1 1 2 1 1 1 1 2 2
7 Thomas, 201913 19 (1338) 10 11 4 17 9 12 7 14 9 2 10 1 20 12 2 7
8 Baessler, 201814 31 (1817) 17 1 13 10 1 20 30 12 4 15 12 2 17 28 2 1 23 4 4
9 Dumoulin, 201815 19 (2717) 16 3 7 1 11 8 4 7 7 8 4 16 3 10 1 8 4 2 13
10 Nambiar, 201716 31 (3290) 14 3 14 10 1 20 4 12 15 6 4 21 14 9 8
11 Glazener, 201717 10 (846) 4 2 4 1 3 6 2 5 48 2 3 50 11 1 43 10
12 Stewart, 201718 56 (3781) 18 2 36 13 2 42 6 3 47 15 3 38 8 6 42 29 1 26 34 13 9
13 Lapitan, 201719 55 (5417) 24 6 25 8 6 41 3 6 46 2 3 50 10 1 44 55
14 Kang, 201520 1 (173) 1 1 1 1 1 1
15 Ayeleke, 201521 13 (585) 4 9 3 10 12 1 1 12 1 4 8 1 12
16 Anderson, 201522 50 (4717) 24 26 20 30 1 43 6 9 17 34 16 10 34 14 4 32
17 Imamura, 201523 11 (5974) 4 1 6 2 1 8 10 1 3 8 4 1 6 11 4 7
18 Silva, 201424 1 1 1 1 1 1
19 Utomo, 201425 5 2 3 3 2 4 1 4 1 2 1 2 1 1 3 1 3 1
20 Lipp, 201426 8 (787) 4 1 3 5 1 2 3 4 1 5 3 1 1 6
21 Herbison, 201327 23 (1806) 9 1 13 6 1 16 12 11 4 2 17 6 9 8 11 12 19 4
22 Wang, 201328 1 (120) 1 1 1 1 1 1 1
23 Berghamns, 201329 6 (544) 3 3 2 4 2 4 2 4 1 1 4 6 1 2 3
24 Clement, 201330 8 (1100) 7 1 6 2 3 1 4 4 3 1 5 3 3 4 1
25 Rai, 201231 23 (3865) 8 15 4 19 8 9 5 4 5 14
26 Cody, 2012 (1)32 34 (19676) 8 26 13 1 20 10 1 23 19 9 16
27 Cody, 2012 (2)33 5 (355) 1 1 3 1 4 1 4 1 4 4
28 Hay-smith, 201134 21 (1490) 10 4 6 6 3 12 2 19 20 1 2 4 15 11 1 9
29 Herderschee, 201135 24 (1583) 12 4 9 6 4 15 8 2 16 2 15 8 18 7 19
30 Fader, 200836 2 (185) 2
31 Fader, 200737 1 (85) 1
32 Mariappan, 200538 10 (3944) 8 2
33 Alhasso, 200539 22 (1099) 18 4
34 Ostaszkiewicz, 2004 (1)40 4 (378) 4
35 Ostaszkiewicz, 2004 (2)41 2 (298) 1 1
36 Wallace, 200442 12 (1473) 2 1 9
37 Eustice, 200043 9 (674) 9
All 611 243 31 276 188 32 393 46 130 265 88 65 307 199 83 295 195 20 163 36 26 108 157 37 146

Note. RCT: Randomized controlled trial.

Risk of bias at different time points

Furthermore, an analysis of the risk of bias domains was conducted over two distinct periods (up until 2015 and from 2016 to 2022). In the initial period, allocation concealment (selection bias) and random sequence generation (selection bias) were the most common domains with unclear results. However, in the recent period, the domains of blinding of participants and personnel (performance bias) and blinding of the outcome assessor (detection bias) emerged as the most common risk of bias domains. Upon rigorous evaluation, it was revealed that the predominant risk of bias across all domains pertains to ambiguous results. This observation is comprehensively illustrated in Figures 1-6.

ija-2-e20-g001
Figure 1.

Evaluating the Extent of Selection Bias in Trials Incorporated Into the Systematic Reviews of the Cochrane Urinary Incontinence Group


ija-2-e20-g002
Figure 2.

Evaluating the Extent of Selection Bias in Trials Incorporated Into the Systematic Reviews of the Cochrane Urinary Incontinence Group


ija-2-e20-g003
Figure 3.

Evaluating the Extent of Performance Bias in Trials Incorporated Into the Systematic Reviews of the Cochrane Urinary Incontinence Group


ija-2-e20-g004
Figure 4.

Evaluating the Extent of Detection Bias in Trials Incorporated Into the Systematic Reviews of the Cochrane Urinary Incontinence Group


ija-2-e20-g005
Figure 5.

Evaluating the Extent of Attrition Bias in Trials Incorporated Into the Systematic Reviews of the Cochrane Urinary Incontinence Group


ija-2-e20-g006
Figure 6.

Evaluating the Extent of Reporting Bias in Trials Incorporated Into the Systematic Reviews of the Cochrane Urinary Incontinence Group



Discussion

The results of the current study indicated that, according to the JBI risk of bias assessment tool for systematic reviews, all Cochrane systematic reviews regarding UI have been conducted in high quality, and most of them have been reported according to PRISMA guidelines. On the other hand, the most common risk of bias in the included RCTs for our subjects, systematic reviews, is the blinding of participants and personnel (performance bias).

Interpretation of Findings

UI, a prevalent issue, often remains unreported due to the embarrassment and social stigma associated with it. This condition, despite its substantial social and economic implications, is capable of significantly affecting an individual’s standard of living. However, the effects can be markedly mitigated with appropriate evaluation, intervention, and management. As the majority of incontinence instances are either treatable or manageable, possessing comprehensive knowledge on this subject is of paramount significance.4 As such, erroneous reporting of clinical outcomes can have far-reaching implications on healthcare, influencing everything from individual patient well-being to the formulation of public health strategies.44 This underscores the importance of assessing the methodology of studies, a pivotal step in the selection of superior clinical literature. This assessment should hinge on both internal and external validity, encompassing elements such as study design, guidance, and data analysis.45 Within the hierarchy of evidence, the pinnacle is occupied by meta-analyses, systematic reviews, and randomized clinical trials, as they offer the most robust level of evidence.46 These research methodologies are pivotal in generating superior clinical evidence, thereby enhancing the effectiveness of treatments. Review articles, particularly those that collate data from consistent and uniform clinical trials, encompass the most potent form of clinical evidence. Such studies significantly influence the development of guidelines and clinical decision-making processes. However, it is crucial to note that improper execution of these studies or a high degree of bias can lead to the generation of inaccurate evidence. This could potentially inflict harm on patients and the entire healthcare system in a multitude of ways. The Cochrane Library has established itself as a robust and trustworthy database, playing a pivotal role in augmenting medical knowledge and facilitating improved clinical decision-making and judgment. Consequently, it is of paramount importance to uphold the quality of these types of studies, which are instrumental in the formulation of guidelines. In the past, several studies spanning various medical disciplines have been undertaken to assess the quality of systematic reviews. A study was conducted on 42 systematic reviews within the field of internal medicine. The results showed that, on average, 4.6 out of the 11 items from the Assessment of Multiple Systematic Reviews guideline received a full score.47 The findings of a study by Salehi-Pourmehr et al,48 which examined the quality of systematic reviews on urologic cancers, demonstrated that the most common sources of bias risk were the unclear results of allocation concealment and random sequence generation domains, both of which are related to selection bias. Moreover, the highest risk of bias originated from the domain of blinding of participants and personnel (performance bias), while the lowest risk of bias was observed in incomplete outcome data (attrition bias) and selective reporting domains. The risk of bias was also evaluated across different periods, revealing that the indications of some domains had increased, while others represented a decrease. Partially similar to the findings of Salehi-Pourmehr et al, Hajebrahimi et al49 discovered that the most prevalent biases in RCTs featured in systematic reviews concerning gynecologic cancers were the unclear results of the allocation concealment domain (selection bias) and the blinding of participants and personnel domain (performance bias). Additionally, the highest risk of bias was associated with the blinding of participants and personnel (performance bias) and the incomplete outcome data domains (attrition bias). Furthermore, the domains with the lowest risk of bias were the incomplete outcome data (attrition bias) and the random sequence generation (selection bias). In this study, the most frequently observed biases stemmed from ambiguous results in the allocation concealment domain and the blinding of the outcome assessor domain, leading to selection bias and detection bias, respectively. Further, the blinding of participants and personnel domain exhibited the highest risk of performance bias. Conversely, the domain associated with the lowest risk of bias was the random sequence generation, which is linked to selection bias.


Conclusions

Taking into account all these factors, it is evident that the risk of bias in certain domains has decreased, while in others it has been on the rise. This observation, coupled with the consensus that the most prevalent risk of bias is tied to ambiguous results, suggests that despite significant enhancements in the quality of studies –from execution to documentation– there is still a considerable distance from achieving an ideal RCT.

Accumulating health sciences evidence requires de-biasing potential errors in datasets and systematic reviews to ensure the safety and effectiveness of medication or therapeutic interventions. Publication bias and systematic error can create context-specific inequities, and safety concerns and distort risk predictions. To de-bias and reduce systematic errors, healthcare providers should be held accountable for providing reassurance on risks associated with therapeutics or procedures. To de-bias the outcome, the strategy of the approach should begin by stating the predefined hypothesis and a valid scientific rationale(s) before evaluating evidence-based outcome measures. The most common bias in systematic reviews may be rooted in bio/medical sciences dataset processing, which can include a single cause or a multitude of random/systematic errors based on reasonable assumptions and variables such as differences in the target population, inter-observer variation, and inherent bias. The likelihood of publication bias can generate and perpetuate uncertainties, particularly when missing data are purposefully excluded. To reduce bias, it is beneficial to first create a checklist(s) based on consensus and standard guideline(s) for reading, assignment, data collection/extraction, analysis, and data extrapolation and interpretation. Creating specific checklists with a set or series of questions can help reduce common pitfalls in publication bias, particularly when a large number of variables are involved or outcomes are suboptimal or invalid. Predefined criteria for data review/assessment can reduce the likelihood of bias and obviate the need to include all confounding variables in context and outcome measures, study endpoints, and/or dataset interpretations.50


Implications for Practice

The findings of our study underscore several critical implications for practice in conducting and evaluating systematic reviews and RCTs in UI:

  1. Emphasizing Research Question Clarity: The consistent identification of well-defined research questions across included systematic reviews highlights the importance of clarity in framing research objectives. Practitioners and researchers should prioritize the formulation of precise and relevant research questions to enhance the focus and applicability of future studies.

  2. Enhancing Methodological Rigor: Given that allocation concealment and binding of outcome assessors were frequently categorized as having unclear risk of bias, there is an urgent need for researchers to adopt more stringent methodological practices. This includes implementing robust allocation concealment strategies and ensuring blinding where feasible to minimize selection and detection biases.

  3. Adhering to Reporting Guidelines: The prevalent use of PRISMA guidelines among the included systematic reviews demonstrates a positive trend toward transparency and quality in reporting. Continued adherence to these guidelines should be encouraged, as they provide a structured approach to presenting systematic reviews, which can aid in the reproducibility and reliability of findings.

  4. Focusing on Risk of Bias Assessment: The identification of ambiguous results across various risk of bias domains suggests that systematic reviews should incorporate comprehensive assessments of bias. Practitioners should advocate for the use of validated tools, such as the JBI risk of bias assessment tool, to evaluate and report bias transparently, thereby improving the overall quality of evidence.

  5. Providing Continuous Education and Training: There is a clear need for ongoing education and training for researchers in the principles of study design, particularly regarding blinding and allocation concealment techniques. Workshops and resources aimed at improving methodological skills can help address common biases identified in our analysis.

  6. Monitoring Trends Over Time: The shift in predominant risk of bias domains from 2015 to 2022 indicates evolving practices in RCTs. Researchers should remain vigilant about emerging trends in bias and adapt their methodologies, thus ensuring that new challenges are addressed proactively.

  7. Encouraging Collaborative Research Efforts: Collaborations between clinical researchers, methodologists, and statisticians can facilitate the design of more rigorous trials. By fostering interdisciplinary partnerships, it is possible to enhance the robustness of research methodologies and improve the quality of evidence generated.


Recommendations for Future Research

Future studies in UI should prioritize the development of clear research questions using frameworks such as Population, Intervention, Comparison(s), and Outcome, adhere to high methodological standards to minimize biases, and consistently employ standardized risk of bias assessment tools for transparency and comparability. Researchers are encouraged to follow established reporting guidelines, explore innovative blinding techniques, and consider longitudinal designs to track trends over time. Collaborative efforts across multiple centers can enhance sample sizes and generalizability, while training programs focusing on research methodology can build capacity among emerging researchers. Additionally, utilizing adaptive trial designs can improve efficiency, and engaging stakeholders early in the research process ensures that studies address relevant clinical questions and patient needs. Implementing these recommendations will enhance methodological rigor and contribute to better clinical outcomes in UI.


Author contributions

Conceptualization: Sakineh Hajebrahimi, Hanieh Salehi-Pourmehr.

Data curation: Amirreza Mosayyebzadeh.

Formal analysis: Hanieh Salehi-Pourmehr.

Investigation: Hanieh Salehi-Pourmehr, Sakineh Hajebrahimi.

Methodology: Hanieh Salehi-Pourmehr, Sakineh Hajebrahimi.

Project administration: Sakineh Hajebrahimi.

Resources: Amirreza Mosayyebzadeh, Morteza Atayi, Nasim Mahdavi.

Software: Amirreza Mosayyebzadeh.

Supervision: Sakineh Hajebrahimi, Hanieh Salehi-Pourmehr.

Validation: Sakineh Hajebrahimi, Hanieh Salehi-Pourmehr.

Visualization: Sakineh Hajebrahimi, Hanieh Salehi-Pourmehr.

Writing–original draft: Amirreza Mosayyebzadeh, Morteza Atayi, Nasim Mahdavi.

Writing–review & editing: Sakineh Hajebrahimi, Hanieh Salehi-Pourmehr.


Ethical approval

This study was approved by the Regional Ethics Committee of Tabriz University of Medical Sciences (IR.TBZMED.REC.1400.705).


Conflict of interests

None declared.


References

  1. Rahn DD, Roshanravan SM. Pathophysiology of urinary incontinence, voiding dysfunction, and overactive bladder. Obstet Gynecol Clin North Am 2009; 36(3):463-74. doi: 10.1016/j.ogc.2009.08.012 [Crossref] [ Google Scholar]
  2. Haylen BT, de Ridder D, Freeman RM, Swift SE, Berghmans B, Lee J. An International Urogynecological Association (IUGA)/International Continence Society (ICS) joint report on the terminology for female pelvic floor dysfunction. Neurourol Urodyn 2010; 29(1):4-20. doi: 10.1002/nau.20798 [Crossref] [ Google Scholar]
  3. Townsend MK, Curhan GC, Resnick NM, Grodstein F. The incidence of urinary incontinence across Asian, Black, and White women in the United States. Am J Obstet Gynecol 2010;202(4):378.e1-378.e7. doi: 10.1016/j.ajog.2009.11.021.
  4. Bardsley A. An overview of urinary incontinence. Br J Nurs 2016; 25(18):S14-21. doi: 10.12968/bjon.2016.25.18.S14 [Crossref] [ Google Scholar]
  5. Norton P, Brubaker L. Urinary incontinence in women. Lancet 2006; 367(9504):57-67. doi: 10.1016/s0140-6736(06)67925-7 [Crossref] [ Google Scholar]
  6. Choo MS, Ku JH, Oh SJ, Lee KS, Paick JS, Seo JT. Prevalence of urinary incontinence in Korean women: an epidemiologic survey. Int Urogynecol J Pelvic Floor Dysfunct 2007; 18(11):1309-15. doi: 10.1007/s00192-007-0322-z [Crossref] [ Google Scholar]
  7. Temtanakitpaisan T, Buppasiri P, Lumbiganon P, Laopaiboon M, Rattanakanokchai S. Prophylactic antibiotics for preventing infection after continence surgery in women with stress urinary incontinence. Cochrane Database Syst Rev 2022; 3(3):CD012457. doi: 10.1002/14651858.CD012457.pub2 [Crossref] [ Google Scholar]
  8. Saraswat L, Rehman H, Omar MI, Cody JD, Aluko P, Glazener CM. Traditional suburethral sling operations for urinary incontinence in women. Cochrane Database Syst Rev 2020; 1(1):CD001754. doi: 10.1002/14651858.CD001754.pub5 [Crossref] [ Google Scholar]
  9. Freites J, Stewart F, Omar MI, Mashayekhi A, Agur WI. Laparoscopic colposuspension for urinary incontinence in women. Cochrane Database Syst Rev 2019; 12(12):CD002239. doi: 10.1002/14651858.CD002239.pub4 [Crossref] [ Google Scholar]
  10. Bakali E, Johnson E, Buckley BS, Hilton P, Walker B, Tincello DG. Interventions for treating recurrent stress urinary incontinence after failed minimally invasive synthetic midurethral tape surgery in women. Cochrane Database Syst Rev 2019; 9(9):CD009407. doi: 10.1002/14651858.CD009407.pub3 [Crossref] [ Google Scholar]
  11. Buckley BS, Sanders CD, Spineli L, Deng Q, Kwong JS. Conservative interventions for treating functional daytime urinary incontinence in children. Cochrane Database Syst Rev 2019; 9(9):CD012367. doi: 10.1002/14651858.CD012367.pub2 [Crossref] [ Google Scholar]
  12. Wieland LS, Shrestha N, Lassi ZS, Panda S, Chiaramonte D, Skoetz N. Yoga for treating urinary incontinence in women. Cochrane Database Syst Rev 2019; 2(2):CD012668. doi: 10.1002/14651858.CD012668.pub2 [Crossref] [ Google Scholar]
  13. Thomas LH, Coupe J, Cross LD, Tan AL, Watkins CL. Interventions for treating urinary incontinence after stroke in adults. Cochrane Database Syst Rev 2019; 2(2):CD004462. doi: 10.1002/14651858.CD004462.pub4 [Crossref] [ Google Scholar]
  14. Baessler K, Christmann-Schmid C, Maher C, Haya N, Crawford TJ, Brown J. Surgery for women with pelvic organ prolapse with or without stress urinary incontinence. Cochrane Database Syst Rev 2018; 8(8):CD013108. doi: 10.1002/14651858.Cd013108 [Crossref] [ Google Scholar]
  15. Dumoulin C, Cacciari LP, Hay-Smith EJ. Pelvic floor muscle training versus no treatment, or inactive control treatments, for urinary incontinence in women. Cochrane Database Syst Rev 2018; 10(10):CD005654. doi: 10.1002/14651858.CD005654.pub4 [Crossref] [ Google Scholar]
  16. Nambiar A, Cody JD, Jeffery ST, Aluko P. Single-incision sling operations for urinary incontinence in women. Cochrane Database Syst Rev 2017; 7(7):CD008709. doi: 10.1002/14651858.CD008709.pub3 [Crossref] [ Google Scholar]
  17. Glazener CM, Cooper K, Mashayekhi A. Bladder neck needle suspension for urinary incontinence in women. Cochrane Database Syst Rev 2017; 7(7):CD003636. doi: 10.1002/14651858.CD003636.pub4 [Crossref] [ Google Scholar]
  18. Stewart F, Berghmans B, Bø K, Glazener CM. Electrical stimulation with non-implanted devices for stress urinary incontinence in women. Cochrane Database Syst Rev 2017; 12(12):CD012390. doi: 10.1002/14651858.CD012390.pub2 [Crossref] [ Google Scholar]
  19. Lapitan MC, Cody JD, Mashayekhi A. Open retropubic colposuspension for urinary incontinence in women. Cochrane Database Syst Rev 2017; 7(7):CD002912. doi: 10.1002/14651858.CD002912.pub7 [Crossref] [ Google Scholar]
  20. Kang D, Han J, Neuberger MM, Moy ML, Wallace SA, Alonso-Coello P. Transurethral radiofrequency collagen denaturation for the treatment of women with urinary incontinence. Cochrane Database Syst Rev 2015; 2015(3):CD010217. doi: 10.1002/14651858.CD010217.pub2 [Crossref] [ Google Scholar]
  21. Ayeleke RO, Hay-Smith EJ, Omar MI. Pelvic floor muscle training added to another active treatment versus the same active treatment alone for urinary incontinence in women. Cochrane Database Syst Rev 2015; 2015(11):CD010551. doi: 10.1002/14651858.CD010551.pub3 [Crossref] [ Google Scholar]
  22. Anderson CA, Omar MI, Campbell SE, Hunter KF, Cody JD, Glazener CM. Conservative management for postprostatectomy urinary incontinence. Cochrane Database Syst Rev 2015; 1(1):CD001843. doi: 10.1002/14651858.CD001843.pub5 [Crossref] [ Google Scholar]
  23. Imamura M, Williams K, Wells M, McGrother C. Lifestyle interventions for the treatment of urinary incontinence in adults. Cochrane Database Syst Rev 2015; 2015(12):CD003505. doi: 10.1002/14651858.CD003505.pub5 [Crossref] [ Google Scholar]
  24. Silva LA, Andriolo RB, Atallah ÁN, da Silva EM. Surgery for stress urinary incontinence due to presumed sphincter deficiency after prostate surgery. Cochrane Database Syst Rev 2014; 2014(9):CD008306. doi: 10.1002/14651858.CD008306.pub3 [Crossref] [ Google Scholar]
  25. Utomo E, Groen J, Blok BF. Surgical management of functional bladder outlet obstruction in adults with neurogenic bladder dysfunction. Cochrane Database Syst Rev 2014; 2014(5):CD004927. doi: 10.1002/14651858.CD004927.pub4 [Crossref] [ Google Scholar]
  26. Lipp A, Shaw C, Glavind K. Mechanical devices for urinary incontinence in women. Cochrane Database Syst Rev 2014; 2014(12):CD001756. doi: 10.1002/14651858.CD001756.pub6 [Crossref] [ Google Scholar]
  27. Herbison GP, Dean N. Weighted vaginal cones for urinary incontinence. Cochrane Database Syst Rev 2013; 2013(7):CD002114. doi: 10.1002/14651858.CD002114.pub2 [Crossref] [ Google Scholar]
  28. Wang Y, Zhishun L, Peng W, Zhao J, Liu B. Acupuncture for stress urinary incontinence in adults. Cochrane Database Syst Rev 2013; 2013(7):CD009408. doi: 10.1002/14651858.CD009408.pub2 [Crossref] [ Google Scholar]
  29. Berghmans B, Hendriks E, Bernards A, de Bie R, Omar MI. Electrical stimulation with non-implanted electrodes for urinary incontinence in men. Cochrane Database Syst Rev 2013; 2013(6):CD001202. doi: 10.1002/14651858.CD001202.pub5 [Crossref] [ Google Scholar]
  30. Clement KD, Lapitan MC, Omar MI, Glazener CM. Urodynamic studies for management of urinary incontinence in children and adults. Cochrane Database Syst Rev 2013; 2013(10):CD003195. doi: 10.1002/14651858.CD003195.pub3 [Crossref] [ Google Scholar]
  31. Rai BP, Cody JD, Alhasso A, Stewart L. Anticholinergic drugs versus non-drug active therapies for non-neurogenic overactive bladder syndrome in adults. Cochrane Database Syst Rev 2012; 12(12):CD003193. doi: 10.1002/14651858.CD003193.pub4 [Crossref] [ Google Scholar]
  32. Cody JD, Jacobs ML, Richardson K, Moehrer B, Hextall A. Oestrogen therapy for urinary incontinence in post-menopausal women. Cochrane Database Syst Rev 2012; 10(10):CD001405. doi: 10.1002/14651858.CD001405.pub3 [Crossref] [ Google Scholar]
  33. Cody JD, Nabi G, Dublin N, McClinton S, Neal DE, Pickard R. Urinary diversion and bladder reconstruction/replacement using intestinal segments for intractable incontinence or following cystectomy. Cochrane Database Syst Rev 2012; 2012(2):CD003306. doi: 10.1002/14651858.CD003306.pub2 [Crossref] [ Google Scholar]
  34. Hay-Smith EJ, Herderschee R, Dumoulin C, Herbison GP. Comparisons of approaches to pelvic floor muscle training for urinary incontinence in women. Cochrane Database Syst Rev. 2011(12):CD009508. doi: 10.1002/14651858.Cd009508.
  35. Herderschee R, Hay-Smith EJ, Herbison GP, Roovers JP, Heineman MJ. Feedback or biofeedback to augment pelvic floor muscle training for urinary incontinence in women. Cochrane Database Syst Rev. 2011(7):CD009252. doi: 10.1002/14651858.Cd009252.
  36. Fader M, Cottenden AM, Getliffe K. Absorbent products for moderate-heavy urinary and/or faecal incontinence in women and men. Cochrane Database Syst Rev. 2008(4):CD007408. doi: 10.1002/14651858.Cd007408.
  37. Fader M, Cottenden AM, Getliffe K. Absorbent products for light urinary incontinence in women. Cochrane Database Syst Rev 2007; 2007(2):CD001406. doi: 10.1002/14651858.CD001406.pub2 [Crossref] [ Google Scholar]
  38. Mariappan P, Ballantyne Z, N’Dow JM, Alhasso AA. Serotonin and noradrenaline reuptake inhibitors (SNRI) for stress urinary incontinence in adults. Cochrane Database Syst Rev. 2005(3):CD004742. doi: 10.1002/14651858.CD004742.pub2.
  39. Alhasso A, Glazener CM, Pickard R, N’Dow J. Adrenergic drugs for urinary incontinence in adults. Cochrane Database Syst Rev 2005; 2005(3):CD001842. doi: 10.1002/14651858.CD001842.pub2 [Crossref] [ Google Scholar]
  40. Ostaszkiewicz J, Johnston L, Roe B. Habit retraining for the management of urinary incontinence in adults. Cochrane Database Syst Rev 2004; 2004(2):CD002801. doi: 10.1002/14651858.CD002801.pub2 [Crossref] [ Google Scholar]
  41. Ostaszkiewicz J, Johnston L, Roe B. Timed voiding for the management of urinary incontinence in adults. Cochrane Database Syst Rev 2004; 2004(1):CD002802. doi: 10.1002/14651858.CD002802.pub2 [Crossref] [ Google Scholar]
  42. Wallace SA, Roe B, Williams K, Palmer M. Bladder training for urinary incontinence in adults. Cochrane Database Syst Rev 2004; 2004(1):CD001308. doi: 10.1002/14651858.CD001308.pub2 [Crossref] [ Google Scholar]
  43. Eustice S, Roe B, Paterson J. Prompted voiding for the management of urinary incontinence in adults. Cochrane Database Syst Rev 2000; 2000(2):CD002113. doi: 10.1002/14651858.Cd002113 [Crossref] [ Google Scholar]
  44. Rosenstein AH, O’Daniel M. Disruptive behavior and clinical outcomes: perceptions of nurses and physicians. Am J Nurs 2005; 105(1):54-64. doi: 10.1097/00000446-200501000-00025 [Crossref] [ Google Scholar]
  45. Khorsan R, Crawford C. How to assess the external validity and model validity of therapeutic trials: a conceptual approach to systematic review methodology. Evid Based Complement Alternat Med 2014; 2014:694804. doi: 10.1155/2014/694804 [Crossref] [ Google Scholar]
  46. Rosner AL. Evidence-based medicine: revisiting the pyramid of priorities. J Bodyw Mov Ther 2012; 16(1):42-9. doi: 10.1016/j.jbmt.2011.05.003 [Crossref] [ Google Scholar]
  47. Shea BJ, Bouter LM, Peterson J, Boers M, Andersson N, Ortiz Z. External validation of a measurement tool to assess systematic reviews (AMSTAR). PLoS One 2007; 2(12):e1350. doi: 10.1371/journal.pone.0001350 [Crossref] [ Google Scholar]
  48. Salehi-Pourmehr H, Naseri A, Mostafaei A, Vahedi L, Sajjadi S, Tayebi S. Misconduct in research integrity: assessment the quality of systematic reviews in Cochrane urological cancer review group. Turk J Urol 2021; 47(5):392-419. doi: 10.5152/tud.2021.21038 [Crossref] [ Google Scholar]
  49. Hajebrahimi S, Dalir Akbari N, Haji Kamanaj A, Hassannezhad S, Aminizadeh S, Darvishi F. Quality of the systematic reviews in Cochrane gynecological cancer group and their understudied RCTs. J Obstet Gynaecol India 2022; 72(Suppl 1):346-51. doi: 10.1007/s13224-022-01655-6 [Crossref] [ Google Scholar]
  50. Sadaie MR. Publication bias and systematic error: how to review health sciences evidence. Int J Drug Res Clin 2024; 2:e8. doi: 10.34172/ijdrc.2024.e8 [Crossref] [ Google Scholar]

Submitted: 09 Sep 2024
Accepted: 28 Oct 2024
First published online: 01 Dec 2024
EndNote EndNote

(Enw Format - Win & Mac)

BibTeX BibTeX

(Bib Format - Win & Mac)

Bookends Bookends

(Ris Format - Mac only)

EasyBib EasyBib

(Ris Format - Win & Mac)

Medlars Medlars

(Txt Format - Win & Mac)

Mendeley Web Mendeley Web
Mendeley Mendeley

(Ris Format - Win & Mac)

Papers Papers

(Ris Format - Win & Mac)

ProCite ProCite

(Ris Format - Win & Mac)

Reference Manager Reference Manager

(Ris Format - Win only)

Refworks Refworks

(Refworks Format - Win & Mac)

Zotero Zotero

(Ris Format - FireFox Plugin)

Abstract View: 5
PDF Download: 9
Full Text View: 1