Simulators for endoscopic retrograde cholangiopancreatography training: systematic review and meta-analysis

INTRODUCTION

Endoscopic retrograde cholangiopancreatography (ERCP) is a valuable technique for evaluating pancreatic and biliary ductal anatomy in a wide variety of clinical situations¹. The first report of successful cannulation of the major duodenal papilla, which is the main stage of the procedure, was conducted in the United States in 1968 by McCune et al.² The first endoscopic biliary sphincterotomies were reported almost simultaneously in 1974 by Kawai from Japan and Classen from Germany³. ERCP has evolved substantially: it has gone from a diagnostic procedure to a therapeutic tool thanks to technological innovations in endoscopes and accessories⁴. The learning process is long, and the main and essential step of the procedure is the cannulation of the major duodenal papilla⁵. ERCP can present severe complications (i.e., pancreatitis, hemorrhage, cholangitis, and perforation), especially in the hands of inexperienced endoscopists. The use of a simulator could be a valid tool for training and performing ERCP more effectively and safely⁶.

In this study, the systematic review and the meta-analysis were performed to clarify whether the use of simulators in ERCP training increases the cannulation rate of the duodenal papilla more than the traditional “master-student” teaching method.

METHODS

Source of data and research

This systematic review followed the recommendations of the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA)⁷. No language or year of publication restrictions was applied. The following databases were searched: MEDLINE, EMBASE, CENTRAL, Web of Science, and LILACS. The gray literature included ProQuest Dissertations & Theses Global and clinical trial records (clinicaltrials.gov). The date of the search was July 20, 2020. The search used the following keywords combined with Boolean logical operators, appropriate for each database: “ERCP,” “simulator(s),’ ‘training(s),” and “model(s).”

Selection of studies

Only randomized clinical trials (RCTs) were included in which the intervention group was trained with ERCP simulators before the practice of ERCP in supervised patients. The control group practiced ERCP directly on supervised patients (“master-student”) without previous training on simulators. Two authors independently evaluated all studies identified by the survey using Review Management Website Covidence (http://www.covidence.org). A third review author resolved any disagreements.

Data extraction and quality assessment

Information on the study design, description of the participants, type of simulator^8-11, type of training, description of the control group, and each outcome explored in the studies were extracted.

All included studies were evaluated for their methodological quality using the Cochrane Collaboration Risk of Bias tool (RevMan 5)¹². The tool measures nine bias categories (i.e., selection, allocation, masking of data and statistical collectors, performance, detection, attrition, selective reporting of outcomes, and other biases). The items were scored as positive (low risk of bias), negative (high risk of bias), or insufficient information (uncertain risk of bias). The GRADE method was used to classify the level of evidence of the outcomes (i.e., high, moderate, low, and very low) using GRADEproGDT (https://gradepro.org/)^13-17.

Statistical analysis

RevMan version 5.3 was used for the statistical analyses.

RESULTS

The search identified 3,310 studies. After duplicate studies were excluded, 2,386 studies were evaluated based on the title and abstract, of which 2,335 were excluded. Full-text studies were retrieved for 51 titles, of which 46 were excluded.

Four studies were included in the review^18-21, although five publications were identified. It is noteworthy that two publications by Meng et al.²¹ and Meng et al.²² have the same method and same authors, and the beginning of the study of the 2019 publication²¹ was in 2016²², according to the information obtained at clinicaltrails.gov. We believed that one of the publications, Meng et al.²², served as a pilot study or initial publication and was incorporated into the publication referenced as Meng et al.²¹, so only the latter was enrolled in this review.

The findings of the studies^18-21 are summarized in Table 1.

Table 1
Characteristics of the included studies.

	Lim et al.18	Liao et al.19	Hritz et al.20	Meng et al.21
Type of publication	ArticleCompleted	ArticleCompleted	Abstract	Abstract
Centers	Multicenter	Multicenter	Multicenter	Not reported
Location	USA	Taiwan	Hungary	China
Number of participants	16 (8 control, 8 intervention)	16 (8 control, 8 intervention)	15 (9 controls, 6 intervention)	12 (6 controls, 6 intervention)
Characteristics of participants	Maximum 30 ERCPs	1,000 EGDs done, 0 ERCPs	Experience in EGD, 0 ERCPs	No previous endoscopic experience
Length of follow-up after training	16 weeks	12 weeks	Not specified	Not reported
Type of simulator	Mechanical (EMS)	Mechanical (EMS)	Computational (AccuTouch®)	Mechanical (EMS)
Number of ERCPs	265 (139 intervention and 126 control)	190 (98 intervention and 92 control)	59 (25 intervention and 34 control)	300 (150 intervention and 150 control)
Mean ERCPs per student	17±10	11.875	3.93	25
Description of interventions	Theoretical and simulator training, repeated after 8 weeks. In the ERCP, if in 10 min the trainee failed after two attempts, he/she received manual help from the supervisor	Theoretical classes on ERCP. The group in 2008 received only one 6-h practical class in the simulator. In 2009, the group received biweekly practical classes in the simulator for 12 weeks. In ERCP, unlimited verbal instructions and manual help after 5 min of attempts. After a second failure with manual help, the supervisor took over	Theoretical of 2 h and training in the simulator. There was no report on the supervision of the ERCP	Training with verbal instructions for 20 h in the simulator. In ERCP, verbal instructions and manual help, if necessary. If failure after 20 min of attempts, the supervisor took over
Description of the group control	Theoretical training without training in the simulator	Theoretical classes on ERCP only	A 2-h seminar on ERCP, without simulator training	Only training with verbal instructions and manual assistance, if necessary, during ERCP
Outcome main	Cannulation success rate	Cannulation success rate	Note successful cannulation	Cannulation success rate
Definition of success in cannulation	Deep biliary cannulation in 10 min of attempts with fewer than three manual aids from the supervisor	Deep biliary cannulation with up to two manual aids from the supervisor for 10 min	Deep biliary cannulation	Selective biliary cannulation in 20 min

• ERCP: Endoscopic retrograde cholangiopancreatography; Endoscopic retrograde cholangiopancreatography: mechanical simulator.

Methodological quality of the included studies

Figure 1 shows the evaluation of the risk of bias of the four included studies^18-21.

Figure 1
Summary of risk of bias of the included studies.

Meta-analysis

Using a meta-analysis with the random-effects model, a significant difference was found between ERCP simulator training and traditional teaching in the outcome cannulation success rate (three studies with sufficient data were included^18,19,21). A relative risk of 1.40 (95%CI 1.24–1.58) was found in favor of the simulator group (heterogeneity I²=0% and p<0.00001) (Figure 2).

Figure 2
Meta-analysis of the outcome success rate at cannulation.

DISCUSSION

No other systematic review on ERCP simulators was found in the literature.

Considering the data on the cannulation success rate of the three studies included in the meta-analysis¹⁸,¹⁹,²¹, there was a significant increase in the training group with simulators compared with the traditional teaching method. This result, which is favorable to the use of the simulator, may have obtained from the better knowledge of how to manipulate the endoscope (duodenoscope) and accessories for the ERCP before practice in patients^23,24.

A systematic review by Ekkelenkamp et al.²⁵ evaluated simulators for endoscopy in general. They found 14 studies related to students who were trained in ERCP simulators and cited an RCT¹⁸ without analyzing it. Considering the quality of the studies in this meta-analysis¹⁸-²¹, more full-text studies (not just abstracts) and improvements are needed to perform and describe randomization and allocation concealment. The masking of data collectors and statisticians was uncertain in all studies. Due to the nature of the intervention, masking of the participants was not possible.

A low risk of bias was identified regarding the blinding of outcome evaluators and the risk of incomplete results. The selective outcome report was considered high risk due to the absence of ERCP complication rates in patients. A high risk of bias from the conflict of interest was considered in three studies¹⁸,¹⁹,²¹, as one of the authors was the developer of the simulator used, a fact that could influence the study design.

Regarding the evaluation of the quality of the evidence for the success rate of cannulation using the GRADE system, it can be concluded that our findings have moderate confidence, taking into account the risk of bias, inconsistency (heterogeneity), indirect evidence, imprecision, and publication bias²⁶. The evidence found in this systematic review may be altered by new studies, modifying the confidence in the estimation of the effect, and may even modify the estimate²⁶.

The limitations of this study are based on the small number of RCTs that met the selection criteria, which may have influenced the calculation of the results. The strategy to get around this limitation was a broad and exhaustive search so that no relevant RCTs were excluded.

CONCLUSIONS

Evidence of moderate confidence suggests that the training of physicians with ERCP simulators, when compared to the traditional teacher-student method, improves the success rate of cannulation of the greater duodenal papilla. Future studies may present such evidence in a more incisive and reliable way.

• Funding: none.

REFERENCES

• ^1. Falkenstein DB, Abrams RM, Kessler RE, Jones B, Johnson G, Zimmon DS. Endoscopic retrograde cholangiopancreatography in the dog: a model for training and research. Gastrointest Endosc. 1974;21(1):25-6. https://doi.org/10.1016/s0016-5107(74)73775-0
  » https://doi.org/10.1016/s0016-5107(74)73775-0
• ^2. McCune WS, Shorb PE, Moscovitz H. Endoscopic cannulation of the ampulla of vater: a preliminary report. Ann Surg. 1968;167(5):752-6. https://doi.org/10.1097/00000658-196805000-00013
  » https://doi.org/10.1097/00000658-196805000-00013
• ^3. Baillie J. Biliary sphincterotomy: less benign than once thought? Curr Gastroenterol Rep. 1999;1(2):102-6. https://doi.org/10.1007/s11894-996-0007-7
  » https://doi.org/10.1007/s11894-996-0007-7
• ^4. Baillie J. Endoscopic retrograde cholangiopancreatography simulation. Gastrointest Endosc Clin N Am. 2006;16(3):529-42, viii. https://doi.org/10.1016/j.giec.2006.03.017
  » https://doi.org/10.1016/j.giec.2006.03.017
• ^5. Leung JW, Yen D. ERCP training – the potential role of simulation practice. J Interv Gastroenterol. 2011;1(1):14-8. https://doi.org/10.4161/jig.1.1.14594
  » https://doi.org/10.4161/jig.1.1.14594
• ^6. Leung J, Lim B, Ngo C, Lao WC, Wing LY, Hung I, et al. Head-to-head comparison of practice with endoscopic retrograde cholangiopancreatography computer and mechanical simulators by experienced endoscopists and trainees. Dig Endosc. 2012;24(3):175-81. https://doi.org/10.1111/j.1443-1661.2011.01209.x
  » https://doi.org/10.1111/j.1443-1661.2011.01209.x
• ^7. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097
  » https://doi.org/10.1371/journal.pmed.1000097
• ^8. Itoi T, Gotoda T, Baron TH, Sofuni A, Itokawa F, Tsuji S, et al. Creation of simulated papillae for endoscopic sphincterotomy and papillectomy training by using in vivo and ex vivo pig model (with videos). Gastrointest Endosc. 2013;77(5):793-800. https://doi.org/10.1016/j.gie.2012.12.015
  » https://doi.org/10.1016/j.gie.2012.12.015
• ^9. Neumann M, Mayer G, Ell C, Felzmann T, Reingruber B, Horbach T, et al. The Erlangen Endo-Trainer: life-like simulation for diagnostic and interventional endoscopic retrograde cholangiography. Endoscopy. 2000;32(11):906-10. https://doi.org/10.1055/s-2000-8090
  » https://doi.org/10.1055/s-2000-8090
• ^10. Leung JW, Lee JG, Rojany M, Wilson R, Leung FW. Development of a novel ERCP mechanical simulator. Gastrointest Endosc. 2007;65(7):1056-62. https://doi.org/10.1016/j.gie.2006.11.018
  » https://doi.org/10.1016/j.gie.2006.11.018
• ^11. Bittner 4th JG, Mellinger JD, Imam T, Schade RR, Macfadyen Jr BV. Face and construct validity of a computer-based virtual reality simulator for ERCP. Gastrointest Endosc. 2010;71(2):357-64. https://doi.org/10.1016/j.gie.2009.08.033
  » https://doi.org/10.1016/j.gie.2009.08.033
• ^12. Higgins JPT, Altman DG. Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. Available from: https://handbook-5-1.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm
  » https://handbook-5-1.cochrane.org/chapter_8/8_assessing_risk_of_bias_in_included_studies.htm
• ^13. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-6. https://doi.org/10.1136/bmj.39489.470347.AD
  » https://doi.org/10.1136/bmj.39489.470347.AD
• ^14. Guyatt GH, Oxman AD, Vist G, Kunz R, Brozek J, Alonso-Coello P, et al. GRADE guidelines: 4. Rating the quality of evidence -- study limitations (risk of bias). J Clin Epidemiol. 2011;64(4):407-15. https://doi.org/10.1016/j.jclinepi.2010.07.017
  » https://doi.org/10.1016/j.jclinepi.2010.07.017
• ^15. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 7. Rating the quality of evidence -- inconsistency. J Clin Epidemiol. 2011;64(12):1294-302. https://doi.org/10.1016/j.jclinepi.2011.03.017
  » https://doi.org/10.1016/j.jclinepi.2011.03.017
• ^16. Guyatt GH, Oxman AD, Kunz R, Brozek J, Alonso-Coello P, Rind D, et al. GRADE guidelines 6. Rating the quality of evidence -- imprecision. J Clin Epidemiol. 2011;64(12):1283-93. https://doi.org/10.1016/j.jclinepi.2011.01.012
  » https://doi.org/10.1016/j.jclinepi.2011.01.012
• ^17. Guyatt GH, Oxman AD, Kunz R, Woodcock J, Brozek J, Helfand M, et al. GRADE guidelines: 8. Rating the quality of evidence -- indirectness. J Clin Epidemiol. 2011;64(12):1303-10. https://doi.org/10.1016/j.jclinepi.2011.04.014
  » https://doi.org/10.1016/j.jclinepi.2011.04.014
• ^18. Lim BS, Leung JW, Lee J, Yen D, Beckett L, Tancredi D, et al. Effect of ERCP mechanical simulator (EMS) practice on trainees’ ERCP performance in the early learning period: US multicenter randomized controlled trial. Am J Gastroenterol. 2011;106(2):300-6. https://doi.org/10.1038/ajg.2010.411
  » https://doi.org/10.1038/ajg.2010.411
• ^19. Liao WC, Leung JW, Wang HP, Chang WH, Chu CH, Lin JT, et al. Coached practice using ERCP mechanical simulator improves trainees’ ERCP performance: a randomized controlled trial. Endoscopy. 2013;45(10):799-805. https://doi.org/10.1055/s-0033-1344224
  » https://doi.org/10.1055/s-0033-1344224
• ^20. Hritz I, Dubravcsik Z, Szepes AZ, Szepes ZG, Kruglikova I, Jensen PF, et al. Assessment of the effectiveness of ERCP mechanical simulator (EMS) exercise on trainees’ ERCP performance in the initial learning period: multicenter randomized controlled trial. United European Gastroenterology Journal. 2013;1(1 Supplement 1):A333.
• ^21. Meng W, Leung JW, Yue P, Wang Z, Wang X, Wang H, et al. Impact of simulation practice on clinical ERCP performance of novice endoscopists – a RCT using ERCP mechanical simulator (EMS). Gastrointestinal Endoscopy. 2019;89(6 Suppl.):AB255-6. https://doi.org/10.1016/j.gie.2019.03.280
  » https://doi.org/10.1016/j.gie.2019.03.280
• ^22. Meng W, Leung JW, Yue P, Wang Z, Wang X, Wang H, et al. Practice with ERCP mechanical simulator (EMS) improves basic ERCP skills of novice surgical trainees. Gastrointestinal Endoscopy. 2016;83(5 Suppl.):AB267-8. https://doi.org/10.1016/j.gie.2016.03.397
  » https://doi.org/10.1016/j.gie.2016.03.397
• ^23. D’Assunção MA, Fry LC, Monkemuller K. Development and testing of a new, simple and inexpensive ex-vivo, ERCP training model for basic and intermediate ERCP skills. Gastrointestinal Endoscopy. 2016;83(5 Suppl.):AB608. https://doi.org/10.1016/j.gie.2016.03.1268
  » https://doi.org/10.1016/j.gie.2016.03.1268
• ^24. Hashiba K, Armellini ST, Leite GD, Marson FP, Siqueira PR, Ratin RF, et al. Evaluation of a new model for ERCP training. Gastrointestinal Endoscopy. 2017;85(5 Suppl.):AB220-1. https://doi.org/10.1016/j.gie.2017.03.491
  » https://doi.org/10.1016/j.gie.2017.03.491
• ^25. Ekkelenkamp VE, Koch AD, de Man RA, Kuipers EJ. Training and competence assessment in GI endoscopy: a systematic review. Gut. 2016;65(4):607-15. https://doi.org/10.1136/gutjnl-2014-307173
  » https://doi.org/10.1136/gutjnl-2014-307173
• ^26. Schünemann H, Brożek J, Guyatt G, Oxman A. GRADE Handbook, 2013. Available from: https://gdt.gradepro.org/app/handbook/handbook.html
  » https://gdt.gradepro.org/app/handbook/handbook.html

Publication Dates

• Publication in this collection
  18 Oct 2021
• Date of issue
  Aug 2021

History

• Received
  06 Apr 2021
• Accepted
  12 June 2021