Dr. Johan Hartshorne considers some pressing questions in his critical appraisal of a systematic review by Monje, et al.
Educational aims and objectives
This article aims to discuss Monje, et al.’s (2016) systematic review and meta-analysis of the impact of Maintenance therapy on the prevention of peri-implant diseases
Implant Practice US subscribers can answer the CE questions to earn 2 hours of CE from reading this article. Take the quiz by clicking here. Correctly answering the questions will demonstrate the reader can:
- Realize the outcomes and conclusions of this systematic review.
- Recognize the clinical implications for patients considering implant therapy.
- Identify some biological complications of peri-implant mucositis and peri-implantitis.
- Realize the importance of implementation of peri-implant maintenance therapy (PIMT) to prevent potential biologic complications and increase the long-term success rate.
- Identify some patient-, clinical- (surgical and restorative), and implant-related risk factors that should be considered to increase the long-term success rate of implants.
This article aims to provide a critical appraisal of a systematic review: Monje A, Aranda L, Diaz KT, Alarcón MA, Bagramian RA, Wang HL, Catena A. Impact of maintenance therapy for the prevention of peri-implant diseases: a systematic review and meta-analysis. J Dent Res. 2016;95(4):372-379.
The research originated from the Department of Periodontics and Oral Medicine at the University of Michigan’s School of Dentistry, based in Ann Arbor, Michigan.
This systematic review concluded, within its limitations, that implant therapy must not be limited to the placement and restoration of dental implants, but also the implementation of peri-implant maintenance therapy (PIMT) to potentially prevent biologic complications and hence to increase the long-term success rate.
Although it must be tailored to a patient’s risk profiling, the findings suggest reason to claim a minimum recall PIMT interval of 5 to 6 months. Additionally, it must be stressed that even with the establishment of PIMT, biologic complications might still occur. Thus, patient-, clinical-, and implant-related factors must be thoroughly explored.
Critical appraisal conclusion
Within the limitations of the present systematic review, the results suggest that regular PIMT reduces the rate of peri-implant mucositis and peri-implantitis, and that 5 to 6 months should be considered a minimum recall interval for PIMT to prevent peri-implant disease. The findings also show that a history of periodontitis significantly increased the incidence of peri-implant disease and suggested more stringent follow-up maintenance at shorter intervals.
In addition, patient-, clinical- (surgical and restorative), and implant-related risk factors must be thoroughly considered to potentially prevent biologic complications and hence increase the long-term success rate of implants.
Although the results presented in this review are considered as low-level evidence, it has given us a better understanding of the importance of regular preventive care in preventing biological complications and increasing the longevity of implant restorations.
Implications for clinical practice
This review holds a number of implications for clinical practice. It suggests that patients considering implant therapy should be adequately informed of the following:
- Risks for peri-implant disease
- Importance of maintaining proper oral hygiene
- Need for long-term regular follow-up and professional maintenance care
- Need to identify early signs of the disease and to develop preventive treatment strategies, particularly for those at high risk.
The review makes a number of other points, going on to state that preventing peri-implant disease and that maintaining the peri-implant tissues in health is the responsibility of both the patient and the dental team. Preventive care should be aimed at eliminating bacterial plaque through meticulous oral hygiene practices and professional mechanical debridement.
Modifiable risk factors, such as perio-dontitis, uncontrolled diabetes, and smoking should be managed before implant placement. Patients with a high-risk profile should be followed up at more frequent intervals to monitor and maintain peri-implant tissue health.
Implant position, design, and the placement of prosthetic superstructures should be planned so that they will facilitate proper personal cleaning and professional monitoring and maintenance therapy. Clinicians placing implants should ensure that there is adequate attached and unmovable keratinized soft tissue surrounding the implant to facilitate peri-implant health. Establishing a radiographic baseline at the time of implant placement is key to monitoring peri-implant bone stability.
Prosthetic components and superstructures should fit properly to avoid microgaps for biofilm adherence and excess cement at submucosal restorative margins. Implant position, design, and placement of prosthetic superstructures should be planned so that they will facilitate proper personal cleaning and professional monitoring and maintenance therapy.
Establishing a clinical (periodontal probing) and radiographic baseline at final prosthesis placement is key to early diagnosis and proper intervention of peri-implant inflammation. Debridement of the implant-supported restoration must be directed at three components: the prosthesis, the abutment, and the implant fixture, should its surface become exposed to the oral cavity.
What is the impact of PIMT upon the incidence of biologic complications (such as mucositis and peri-implantitis)?
This review methodology was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines (Moher, et al., 2009). The Newcastle-Ottawa Scale (NOS) (Wells, et al., 2011) was used to evaluate the methodological quality of non-randomized trials included studies.
Electronic and manual literature searches were conducted by three independent reviewers in several databases, including Medline, Embase, the Cochrane Central Register of Controlled Trials, and Cochrane Oral Health Group databases for articles published up to June 2015 without language restriction. Electronic screening of the gray literature was conducted according to the Assessment of Multiple Systematic Reviews (AMSTAR) guidelines (Shea, et al., 2009). Additionally, a manual search of periodontics-related journals was done.
Randomized or non-randomized clinical trials, prospective, or retrospective cohort studies or case series studies involving humans with a sample size of at least 10 subjects and a minimum follow-up time of 6 months were eligible.
Implants had to meet the criteria of rough surface, with or without a smooth surface collar.
Studies included in the meta-analysis had to give a clear description of PIMT intervals and biologic complications. Participants in the studies were patients with mandibular and/or maxillary complete or partially edentulous healthy subjects in need of dental implants to restore oral function.
The test group were subjects receiving PIMT at regular recall intervals after implant placement for treatment of peri-implant disease. The control group received no regular PIMT or a longer interval compared to the test group.
The primary outcomes measured were the incidence of peri-implant mucositis and peri-implantitis at implant and patient level. Secondary outcomes measured were implant survival and implant failure rates. Three reviewers independently screened and extracted the data from studies. In case of disagreement, consensus was reached by discussion with a fourth reviewer. In case of unclear data, authors were contacted to provide the data. A multivariate negative binomial regression was used to examine potential effects of PIMT on the incidence of peri-implant disease, implant success and failure rates.
Thirteen clinical trials were included in the qualitative analysis and 10 clinical trials in the quantitative analysis.
For mucositis, a history of periodontal disease showed negative effects (z = -8.12, P < 0.001; lower mucositis with larger number of patients with history of periodontitis). Moreover, PIMT interval was shown to significantly influence the incidence of mucositis at this level (z = 8.64, P < 0.001).
Significant effects of treatment (z = -19.04, P < 0.001), history of periodontitis (z = -14.64, P < 0.001; increased peri-implantitis with larger number of patients with history of periodontitis), and mean PIMT (z = -29.31, P < 0.001) were obtained for peri-implantitis. Additionally, a significant effect was found for the intervals of PIMT on the incidence of peri-implantitis at this level (z = -5.26, P < 0.001).
For mucositis, there were significant effects of treatment (z = -14.36, P < 0.001), history of periodontitis (z = -5.83, P < 0.001; lower mucositis with larger number of periodontal disease patients), and mean PIMT interval (z = -21.07, P < 0.001; lower peri-implantitis with larger interval). PIMT range did show an influence on mucositis at this level (z = -3.07, P = 0.002). For peri-implantitis, the same negative binomial model was applied.
Significant effects of treatment (z = -16.63, P < 0.001), history of periodontal disease (z = 3.79, P < 0.001; increased peri-implantitis with larger number of patients with history of periodontal disease), and mean PIMT (z = -3.94, P < 0.001) were observed.
In addition, PIMT interval demonstrated a significant effect on the incidence of peri-implantitis at this level (z = -26.51, P < 0.001). Mean PIMT had a statistically significant effect on implant survival rate (z = -7.88, P < 0.001) and implant failure rate (z = -30.59, P = 0.001). PIMT interval significantly influenced the incidence of mucositis (at implant level only) (z = 8.64, P < 0.001) and peri-implantitis (z = -5.26, P < 0.001). History of periodontitis also showed a statistically significant effect on implant failure rate (z = 38.03, P < 0.001).
Results suggest that 5 to 6 months is a reasonable interval for PIMT.
Within the limitations of the present systematic review, it can be concluded that implant therapy must not be limited to the placement and restoration of dental implants, but to the implementation of PIMT as well in order to potentially prevent biologic complications and hence heighten the long-term success rate.
Although implant treatment must be tailored to a patient’s risk profiling, the findings suggest reason to claim a minimum recall PIMT interval of 5 to 6 months. Additionally, it must be stressed that even in the establishment of PIMT, biologic complications might occur. Hence, patient-, clinical-, and implant-related factors must be thoroughly explored.
The authors declared that no funding, either directly or indirectly, was provided for the elaboration of this study nor was there potential conflict of interest with respect to the authorship and/or publication of this review.
Background and importance
Peri-implant soft tissue healing and health differs significantly from that of natural teeth.
Key characteristics of peri-implant soft tissues are deeper probing depth, weaker connective tissue attachment, reduced vascular supply, and more pronounced inflammatory tissue response, making the implant more vulnerable to bacterial accumulations and other external stimulations (Belibasakis, 2014; Wang, et al., 2015).
Biological complications of peri-implant tissues — namely, peri-implant mucositis and peri-implantitis — are a frequently encountered problem, characterized by the inflammatory destruction of the tissues surrounding the implant as a result of biofilm formation on the prosthesis, the abutment, and the implant fixture, should its surface become exposed to the oral cavity (Belibasakis 2014). Microbial colonization in the form of dental plaque biofilms and the host inflammatory response in the peri-implant tissues have been considered as the critical etiological factors for the development of peri-implant diseases (Mombelli and Lang, 1998).
However, several other etiological factors and risk indicators, such as a history of periodontitis, excess cement, smoking, diabetes, absence of keratinized mucosa, implant characteristics, and occlusal overload could also increase the incidence of peri-implant diseases (Figuero, et al., 2014). The diagnostic criteria for peri-implant diseases are mainly clinical and radiographic (Mombelli and Lang, 1998).
Peri-implant mucositis is a reversible inflammatory process characterized by inflamed or erythematous peri-implant mucosa and bleeding with probing with no evidence of loss of the supporting bone (Mombelli and Lang, 1998; Heitz-Mayfield and Lang, 2010).
Peri-implantitis, on the other hand, is characterized by both soft tissue inflammation and progressive loss of supporting bone leading to the formation of a peri-implant pocket (of greater than 5 mm), bleeding or suppuration on probing, and, radiographically, a characteristic symmetrical “saucer-shaped” bone destruction (or “crater”) around the implant (Mombelli and Lang, 1998; Heitz-Mayfield and Lang, 2010).
Peri-implant diseases are common following implant therapy (Atieh, et al., 2013). Recent systematic reviews estimate that the frequency of peri-implant mucositis was 63.4% of participants and 30.7% of implants, and that of peri-implantitis were 18.8% of participants and 9.6% of implants. A higher frequency (36.3%) of occurrence of peri-implant diseases was recorded for smokers (Atieh, et al., 2013).
Another meta-analysis estimated a weighted mean prevalence of peri-implant mucositis and peri-implantitis of 43% (CI: 32%-54%) and 22% (CI: 14%-30%), respectively (Derks and Tomasi, 2015).
Emerging evidence shows that there is an increased risk for conversion of peri-implant mucositis to peri-implantitis in undiagnosed patients or in patients who are not receiving supportive therapy (Costa, et al., 2012).
A recent study suggested that peri-implantitis is a common condition and that several patient- and implant-related factors influence the risk for moderate/severe peri-implantitis (Derks, et al, 2016).
Peri-implantitis is a progressive chronic inflammatory process that, if not diagnosed and/or treated appropriately and promptly, may lead to gradual destruction of the supporting bone, and potentially implant loss (Carcuac and Berglundh, 2014). With an increasing number of dental implants being placed annually and the fact that peri-implant diseases’ prevalence increases with the number of years an implant is in function (Roos-Jansaker, 2007), one can expect that peri-implant disease will increasingly become a significant challenge to clinicians, threatening the success and longevity of dental implant therapy.
Tarnow (2016) places this nicely in perspective with the following quote from his article: “In addition, because at least 10% of all implants being placed this year will have peri-implant disease after 10 years, it makes us realize how important this is.
“The estimate that one million implants were placed last year worldwide should serve as an eye-opener. If, conservatively, 10% of all implants will have this problem, then 100,000 per year will need our attention. It should also be mentioned that life in the mouth is a hostile environment for both teeth and implants. We therefore will see more peri-implantitis on implants as they spend more time in the mouth. This is just like teeth.”
Therefore, as the number of implants placed per year continues to increase, it is becoming increasingly important for the clinician to ensure that patients are properly informed and that they are receiving appropriate, regular long-term supportive treatment to ensure that their implants remain disease-free.
Recent evidence indicates that without a regular maintenance program, including clinical reevaluation, plaque control, oral hygiene instruction, and other professional preventive measures, the benefits provided by dental implants cannot be maintained on a long-term basis, and biological complications such as peri-implant mucositis and/or peri-implantitis may result (Smeets, et al., 2014; Tonetti, et al., 2015).
To date, current approaches to implant maintenance are empirical with no evidence-based guidelines or protocol for prevention of peri-implant diseases. Additionally, there is currently no reliable evidence to identify the most effective intervention for treating peri-implantitis (Esposito, et al., 2012). If the clinician is better able to identify and understand the etiology, pathogenesis, and characteristics of peri-implant disease, prevention and management can be performed more effectively. The present systematic review aims at assessing the impact of maintenance therapy on the incidence of peri-implant diseases.
Are the results valid?
The present systematic review had major limitations affecting the quality and validity of the evidence presented.
Overall, the primary studies included in the study were based upon convenience samples of limited sample size, questionable study design producing low-level evidence, an unclear or high risk for bias, and therefore may not provide a true reflection of the effect of PIMT on peri-implant disease. Furthermore, follow-up time and PIMT interval range analyzed varied considerably and may not accurately demonstrate its true impact on the incidence of peri-implant disease.
The number of studies and sample sizes were too small to analyze the effect of extraneous variables (such as smoking, systemic conditions, bone quality, implant location, peri-implant mucosal dimensions, implant design and surface characteristics, number of implants, placement protocols, loading protocols, follow-up period, and type of prosthesis) on the incidence of peri-implant disease.
These confounding variables could affect the reliability of the evidence presented. In view of the previously mentioned limitations, caution should therefore be exercised when interpreting the data and extrapolating directly from the findings to daily implant practice.
What are the key findings?
An incidence rate ratio (IRR) is a relative measure of the effect (peri-implant disease) of a given exposure (PIMT) or the approximation of the relative risk of a non-exposure to PIMT. IRR is calculated by dividing the incidence of peri-implant disease among the exposed population (PIMT) by the incidence among the non-exposed population (no-PIMT).
Subjects receiving PIMT had 0.9369 times the rate (risk) of having mucositis and 0.7056 times the rate of having peri- implantitis at implant level compared to subjects not receiving PIMT. (Rate ratios are often interpreted as if they were risk ratios — for example, subjects receiving PIMT had 0.9369 times the risk of getting mucositis compared to subjects not receiving PIMT, but it is more precise to refer to the ratio of rates rather than risk.)
PIMT intervals had a significant effect on the incidence of mucositis at implant level and peri-implantitis at implant level and patient level. The findings of this review suggest 5 to 6 months to be a reasonable interval for PIMT. History of periodontitis significantly increased the incidence of mucositis and peri-implantitis at both implant and patient level. Mean PIMT and periodontitis showed significant effects on implant failure rate. The systematic review showed that PIMT has a positive impact on peri-implant tissue health as well as implant survival rate.
The results of this study lend further support to findings of previous studies that long-term maintenance care and frequency of maintenance visits are essential to reduce the risk of peri-implantitis, and to enhance implant survival (Atieh, et al., 2013; Gay, et al., 2015). The finding that history of periodontitis increases the incidence of peri-implantitis is also in agreement with a recent study suggesting that susceptibility confers higher risk of implantitis (Derks, et al., 2016).
How are the results of this review applicable in clinical practice?
Are these interventions feasible?
The findings of this study, although very low-level evidence, remain very clinically relevant to daily implant practice.
Early diagnosis and identifying risk factors during treatment planning and throughout therapy, together with regular long-term follow-up are fundamental principles in preventing disease and selecting appropriate treatment protocols.
Long-term maintenance care at more frequent intervals for high-risk groups is essential to reduce the risk of peri-implant disease. Patients receiving implant treatment must be duly informed on the risk for peri-implant diseases and the need for regular preventive care (Atieh, et al., 2013; Jepsen, et al., 2015). It is important that implant placement and prosthetic reconstructions need to allow proper personal cleaning, diagnosis by probing, and professional plaque removal (Jepsen, et al., 2015).
Peri-implant mucositis is a reversible condition; therefore, early diagnosis is imperative for initiating preventive care. Patient-administered mechanical plaque control alone should be considered the current standard of care for preventing and managing peri-implant mucositis (Jepsen, et al., 2015). Treatment of peri-implant mucositis usually includes mechanical debridement of biofilm and calculus either by professional intervention and/or by home-use oral hygiene techniques, with or without the adjunctive use of antimicrobials.
Based on the available data, it seems that the nonsurgical therapy of peri-implantitis is not effective in disease resolution because only limited improvements in the main clinical parameters have been reported, and there is a clear tendency for disease recurrence. It is therefore recommended to consider advanced therapies, such as surgical intervention, when nonsurgical peri-implant surgery is unable to achieve significant improvements in the clinical parameters (Figuero, et al., 2014; Heitz-Mayfield, et al., 2014; Robertson, et al., 2015).
Treatment of peri-implantitis should be tailored to the severity of the lesion, which ranges from mechanical debridement to implant removal. Several nonsurgical and surgical treatment alternatives exist, and there is little consensus on superior treatment methods.
For more detailed nonsurgical and surgical treatment protocols, readers can refer to the following clinical literature: Figuero, et al., 2014; Heitz-Mayfield, et al., 2014; Robertson, et al., 2015; Heitz-Mayfield and Mombelli, 2014; Lang, et al., 2004.
Do benefits outweigh the potential harms and costs?
“Peri-implant disease is not an easy, predictable disease to treat once it is past the mucositis level into the bone. Therefore, the key is prevention based on proper implant design, proper placement, and correct contours for ease of hygiene. This, along with meticulous maintenance care by both the dentist and the patient, will mean that we won’t ‘revisit’ these implants.” (Tarnow, 2016)
Furthermore, treatment of peri-implantitis can be inconvenient and uncomfortable for the patient, and appropriate treatment is demanding in terms of resources and costs.
Thus, as the global number of individuals who undergo implant-retained restorative therapy increases, peri-implantitis will increasingly become a major and growing problem in dentistry (Derks, et al., 2016). It is best to prevent peri-mucositis, which is the precursor of peri-implantitis. Long-term regular follow-up and maintenance care is key to preventing peri-implant disease.
Early diagnosis will ensure that appropriate therapy is initiated to reverse peri-implant mucositis. Additionally, long-term maintenance care, especially for high-risk groups, is essential to reduce the risk of peri-implantitis and subsequent implant failure.
To summarize, prevention incurs less risk of peri-implant disease, discomfort, emotional stress, and costs associated with surgical treatment protocols associated with managing peri-implantitis or ultimate removal of a failed implant.
Within the limitations of the present systematic review, the results support the fundamental principle that dental implants necessitate regular long-term attention to peri-implant maintenance through rigorously employed regular professional follow-up and patient self-management. This review has highlighted the importance of proper oral hygiene, and more stringent follow-up care is suggested, with recall intervals every 5 to 6 months.
The findings show that a history of periodontitis significantly increases the incidence of peri-implant disease — meaning clinicians should pay particular attention to patients with periodontal disease, with regard to periodontal therapy and adherence to regular periodontal maintenance. In addition, patient-, clinical- (surgical and restorative), and implant-related factors must be thoroughly considered to potentially prevent biologic complications and hence increase the long-term success rate of dental implants.
Although the results presented in this review are low-level evidence, it has given us a better understanding of the importance of regular preventive care in preventing biological complications and increasing the longevity of implant restorations. Future research should focus on quality study design and adherence to standardized guidelines for diagnostic criteria for measuring outcome parameters, and especially patient-reported outcomes.
Well-designed studies with attention to inclusion criteria and larger sample sizes will allow for analysis of treatment outcomes at multivariable level to identify sources of heterogeneity. Further investigations should also focus on optimal ways for treating peri-implant mucositis and peri-implantitis that can provide evidence-based therapeutic protocols for peri-implant disease.
- Atieh MA, Alsabeeha NH, Faggion CM Jr, Duncan WJ. The frequency of peri-implant diseases: a systematic review and meta-analysis. J Periodontol. 2013;84(11):586-1598.
- Belibasakis GN. Microbiological and immuno-pathological aspects of peri-implant diseases. Arch Oral Biol. 2014;59(1):66-72.
- Carcuac O, Berglundh T. Composition of human peri-implantitis and periodontitis lesions. J Dent Res. 2014;93(11):1083–1088.
- Costa FO, Takenaka-Martinez S, Cota LO, Ferreira SD, Silva GL, Costa JE. Peri-implant disease in subjects with and without preventive maintenance: a 5-year follow-up. J Clin Periodontol. 2012;39(2):173-183.
- Derks J, Schaller D, Håkansson J, Wennström JL, Tomasi C, Berglundh T. Effectiveness of implant therapy analyzed in a Swedish population: prevalence of peri-implantitis. J Dent Res. 2015;95(1):43-49.
- Derks J, Tomasi C. Peri-implant health and disease. A systematic review of current epidemiology. J Clin Periodontol. 2016;42(suppl 16):158-171.
- Esposito M, Grusovin MG, Worthington HV. Interventions for replacing missing teeth: treatment of peri-implantitis: Cochrane Database Syst Rev. 2012;CD004970.
- Figuero E, Graziani F, Sanz I, Herrera D, Sanz M. Management of peri-implant mucositis and peri-implantitis. Periodontol 2000. 2014;66(1):255-273.
- Gay IC, Tran DT, Weltman R, et al. Role of supportive maintenance therapy on implant survival: a university-based 17 years retrospective analysis. Int J Dent Hyg. 2016;14(4):267-271.
- Heitz-Mayfield LJ, Lang NP. Comparative biology of chronic and aggressive periodontitis vs. peri-implantitis. Periodontol 2000. 2010;53:167-181.
- Heitz-Mayfield LJA, Mombelli A. The therapy of peri-implantitis: a systematic review. Int J Oral Maxillofac Implants. 2014;29(suppl):325-345.
- Heitz-Mayfield LJ, Needleman I, Salvi GE, Pjetursson BE. Consensus statements and clinical recommendations for prevention and management of biologic and technical implant complications. Int J Oral Maxillofac Implants. 2014;29(suppl):346-350.
- Jepsen S, Berglundh T, Genco R, et al. Primary prevention of peri-implantitis: managing peri-implant mucositis. J Clin Periodontol. 2015;42(suppl 16):152–157.
- Lang NP, Berglundh T, Heitz-Mayfield LJ, Pjetursson BE, Salvi GE, Sanz M. Consensus statements and recommended clinical procedures regarding implant survival and complications. Int J Oral Maxillofac Implants. 2004;19(suppl):150-154 .
- Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann Int Med. 2009;151(4): 264-269.
- Mombelli A, Lang NP. The diagnosis and treatment of peri-implantitis. Periodontol 2000. 1998;17:63-76.
- Robertson K, Shahbazian T, MacLeod S. Treatment of peri-implantitis and the failing implant. Dent Clin N Am. 2015;59(2):329-343.
- Roos-Jansaker AM. Long time follow-up of implant therapy and treatment of peri-implantitis. Swed Dent J Suppl. 2007;(188)7-66.
- Shea BJ, Hamel C, Wells GA, et al. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62(10):1013-1020.
- Smeets R, Henningsen A, Jung O, Heiland M, Hammächer C, Stein JM. Definition, etiology, prevention and treatment of peri-implantitis—a review. Head Face Med. 2014;10:34.
- Tarnow DP. Increasing prevalence of peri-implantitis: How will we manage? J Dent Res. 2016;95(1):7-8.
- Tonetti MS, Chapple IL, Jepsen S, Sanz M. Primary and secondary prevention of periodontal and peri-implant diseases: Introduction to, and objectives of the 11th European Workshop on Periodontology consensus conference. J Clin Periodontol. 2015; 42(suppl 16):1–4.
- Wang Y, Zhang Y, Miron RJ. Health, maintenance, and recovery of soft tissues around implants. Clin Implant Dent Rel Res. 2015; 18(3):618-634.
- Wells GA, Shea B, O’Connell D, Peterson J, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of non-randomized studies in meta-analyses. Ottawa Hospital Research Institute. 2011;http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp. Accessed November 9, 2017.