National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients

Transurethral resection of bladder tumours (TURBT) is the initial diagnostic treatment for patients with bladder cancer. TURBT is not an easy procedure to master and simulator training may play a role in improving the learning curve.

From June 2019 to March 2021, 31 doctors at urology departments in Denmark performed two pretraining TURBT procedures on patients, followed by proficiency-based mastery learning on a virtual reality simulator and then two post-training TURBTs on patients.

Operating theatre performances were video-recorded and assessed by two independent, blinded raters using the Objective Structured Assessment for Transurethral Resection of Bladder Tumours Skills (OSATURBS) assessment tool. Paired-sample tests were used to compare pretraining and post-training analyses and independent tests for between-group comparisons. This trial is registered at ClinicalTrials.gov as NCT03864302.

Before training, novices had significantly lower performance scores in comparison to those with intermediate experience ( = 0.017) and experienced doctors ( 0.001). After training, novices significantly improved their clinical performance score (from 11.4 to 17.1; = 0.049, = 10). Those with intermediate experience and experienced doctors did not benefit significantly from simulator training ( = 0.9 and = 0.8, respectively).

We trained surgeons in an operation to remove bladder tumours using a virtual reality simulator. Novice doctors improved their performance significantly after the training, but the training effects for more experienced doctors were minimal. Therefore, we suggest the introduction of mandatory simulator training in the residency programme for urologists.

Transurethral resection of bladder tumours (TURBT) is the initial diagnostic treatment for patients with bladder cancer (BC). BC is common, with 570 000 new patients diagnosed worldwide annually, and more than half of these patients will experience recurrence of BC []. BC is two-faced: non–muscle-invasive bladder cancer (NMIBC) is recurrent but has minimal malignant potential, while muscle-invasive bladder cancer (MIBC) is lethal if left untreated and has a significant mortality rate even after radical cystectomy []. The three objectives with TURBT are cancer clearance, correct histopathological diagnosis, and avoidance of adverse events []. Complete tumour eradication and adequate tissue sampling for histopathological diagnosis are paramount for accurate risk stratification in BC []. Although the anatomy and technical armamentarium seem simple, TURBT is a highly complex procedure [].

Detrusor muscle in the first, apparently complete transurethral resection of bladder tumour specimen is a surrogate marker of resection quality, predicts risk of early recurrence, and is dependent on operator experience.

Impact of surgical experience on recurrence and progression after transurethral resection of bladder tumour in non-muscle-invasive bladder cancer.

The learning curve for transurethral resection of bladder tumour: how many is enough to be independent, safe and effective surgeon?.

Inexperience has a negative impact on the quality of TURBT. Detrusor muscle (DM) presence and overall survival are lower for patients operated on by residents in comparison to consultants []. In addition, recurrence rates and readmission rates are higher for surgical residents than for consultants []. These findings underscore that TURBT is not an easy procedure to master and that residents in their initial learning phase need substantial support to ensure that patient safety is not endangered [].

Surgical curricula should strive to ensure trainees are proficient in surgical skills before they progress to clinical performance, and a complete curriculum should be developed and evaluated for intended (and unintended) effects []. Kern’s six-step approach [] is a model that facilitates complete curriculum development. It includes six distinct steps that are closely interrelated although not in a strictly linear fashion:

Detrusor muscle in the first, apparently complete transurethral resection of bladder tumour specimen is a surrogate marker of resection quality, predicts risk of early recurrence, and is dependent on operator experience.

Impact of surgical experience on recurrence and progression after transurethral resection of bladder tumour in non-muscle-invasive bladder cancer.

Step 1 is already covered regarding TURBT, as adverse effects have been identified as a problem stemming from classical apprenticeship training [], and a national assessment of general needs included TURBT on the final list of urology procedures that should be practised in a simulation-based environment []. Furthermore, it has been established that simulation-based training (SBT) is an excellent educational strategy for skills training in TURBT (step 4) [], and goals and specific measurable objectives of SBT have been defined (step 3) []. However, scientific evidence is lacking regarding who to train (step 2) and when the SBT curriculum should be implemented (step 5) and the proposed effects evaluated (step 6).

Bube SH, Kingo PS, Madsen MG, et al. Validation of a novel assessment tool identifying proficiency in transurethral bladder tumour resection: the OSATURBS assessment tool. J Endourol. In press. https://doi.org/10.1089/end.2021.0768.

Optimally, this approach should be applied to patient-related procedures using an assessment tool with validity evidence [].

The aim of our study was to implement a national SBT TURBT training programme and explore which surgeons improved their clinical patient performance after training to a predefined proficiency level on a simulator.

The ethics committee of the Zealand Region deemed this study to be exempt from approval (REG-008-2018).

Bube SH, Kingo PS, Madsen MG, et al. Validation of a novel assessment tool identifying proficiency in transurethral bladder tumour resection: the OSATURBS assessment tool. J Endourol. In press. https://doi.org/10.1089/end.2021.0768.

Doctors at urology departments in three major hospitals in Denmark were invited to participate. Surgeons of all experience levels were invited, ranging from first-year residents to consultants. Participants were volunteers and gave informed consent before inclusion. Pretraining data for the cohort have been reported previously [].

Before simulator training, the participants performed two TURBT on patients (with tumours ≤3 cm). An expert investigator (urologist) was present. The investigator could act as a supervisor on request by the participant or if the investigator deemed it necessary. The two performances were video-recorded.

Participants were restricted to performing only TURBT related to the study during their inclusion period.

All participants filled out a questionnaire on demographics, including age, sex, title, and the number of previous TURBT operations performed. The TURB Mentor virtual reality (VR) simulator (Simbionix/Surgical Science, Gothenburg, Sweden) was used in this study. A standardised introduction to the simulator interface and equipment was followed by a warm-up session on the simulator limited to 15 min to ensure familiarity.

After the warm-up session, the participants performed simulator training as mastery learning, whereby individual training continues until each trainee reaches a predefined level of proficiency []. The trainees received computerised feedback and formative feedback from the urologist. We previously gathered validity evidence for the simulator test according to the contemporary framework for validity and defined the proficiency level using an established standard-setting method []. Each participant continued training until they reached the computerised proficiency level on three consecutive performances in each of the three TURBT cases on the simulator. Training sessions were limited to a maximum of 2 h per session, or less if the participant requested it because of fatigue. Participants repeated training sessions (minimum 1 d apart) until they reached proficiency.

After simulator training, the participants performed two TURBT (tumours ≤3 cm) on patients with the investigator present. The two post-training performances were video-recorded. The final TURBT was performed within 30 d of the simulator training.

Bube SH, Kingo PS, Madsen MG, et al. Validation of a novel assessment tool identifying proficiency in transurethral bladder tumour resection: the OSATURBS assessment tool. J Endourol. In press. https://doi.org/10.1089/end.2021.0768.

Video recordings of pretraining and post-training TURBT procedures were assessed by two blinded video raters using the Objective Structured Assessment for Transurethral Resection of Bladder Tumours Skills (OSATURBS) assessment tool []. The participating doctors were aware of the scoring items in the OSATURBS tool (insertion of scope, bladder filling, diagnostic cystoscopy, interpretation and strategy, instrumentation, finalisation, haemostasis, progression, and overall impression). Videos were presented to the raters in a randomised order to mask experience levels and procedure order (ie, raters were blinded to operator experience and whether it was a pretraining or post-training procedure). Paired-sample tests were used for pretraining and post-training analyses and independent tests for between-group comparisons. Correlations are reported as Pearson’s . Significance was defined as 0.05. SPSS version 28.0 (IBM, Armonk, NY, USA) was used for statistical analysis.

In total, 32 doctors were enrolled; one participant in the novice group did not perform post-training TURBTs and was excluded. Ten novices (10 TURBT), nine individuals with intermediate experience (11–49 TURBT), and 12 experienced doctors (≥50 TURBT) were included (). The data collection period was, on average, 17 d per participant (range 2–66). Data were collected from June 2019 to March 2021 at three hospitals in Denmark.

The Objective Structured Assessment for Transurethral Resection of Bladder Tumours Skills (OSATURBS) tool for assessment of surgical skills in transurethral resection of bladder tumour. The scale ranges from a minimum of 0 to a maximum of 36, as scores are recoded as 1 → 0, 2 → 1, 3 → 2, 4 → 3, and 5 → 4 for all nine items.

There was good correlation between OSATURBS scores () in the two pretraining (Pearson’s = 0.71; 0.001) and the two post-training procedures (Pearson’s = 0.66; 0.001). Thus, the performance scores were calculated as the mean for the pretraining TURBT and the post-training TURBT.

The novices had significantly lower pretraining performance scores in comparison to individuals with intermediate experience ( = 0.017) and experienced doctors ( 0.001).

The clinical performance of novices significantly improved after simulator training (from 11.4 to 17.1; = 0.049, = 10). However, neither the intermediates nor experienced groups improved significantly after simulator training ().

Effect of simulator training. Change in performance score for transurethral resection of bladder tumour (TURBT) after simulator training according to previous clinical experience.

illustrates the effects of previous TURBT procedures and sensitivity to simulator training. Again, most novices had positive training effects, and some individuals had considerable improvements after simulator training (+22.5 points).

The post-training scores did not differ significantly between the novice and intermediate groups (mean difference 3.76, 95% confidence interval [CI] −2.9 to 10.0; = 0.2), but was significantly inferior for the novice group in comparison to the experienced group (mean difference 8.3, 95% CI 3.3–13.3; = 0.003; ).

[16] A high-fidelity, virtual reality, transurethral resection of bladder tumor simulator: validation as a tool for training. [17] Our study provides unique evidence for the higher levels of the Kirkpatrick hierarchy. Results for Kirkpatrick level 1 have been reported for VR SBT in TURBT, with good overall trainee satisfaction and improved self-confidence [], and Kirkpatrick level 2 evaluation revealed improved simulator scores after SBT training in TURBT []. Following on to the next level, our findings provide essential documentation of skills transfer from the simulation environment to clinical performance.

[19] [20] In this study we implemented a national standardised programme of mastery learning for TURBT (Kern’s step 4, educational strategy and step 5, implementation). Mastery learning has several implications for training effects. Overall, mastery learning ensures that all trainees reach the same minimum standard. By contrast, training defined according to quantitative measures (such as training time or the number of procedures performed) ensures that only the amount of training and not a defined proficiency level—or quality—is accomplished, with a risk of large variations in actual proficiency. According to the Danish national specialist programme in urology, TURBT competence is achieved after approximately ten TURBT procedures in the first year, followed by 15 procedures in the following 5 yr of residency []. There is an association between surgical volume and outcome; however, there are substantial variations in the volume needed to reach proficiency, and a curriculum based on the number of procedures would be unnecessary for some surgeons and, most importantly, would not be sufficient for some surgeons to reach proficiency []. Agreement on standards across institutions is crucial for the reliability and generalisability of a training programme, and one way to account for variations in learning is to implement proficiency-based training programmes [].

[21] The Simbla TURBT simulator in urological residency training: from needs analysis to validation. [21] The Simbla TURBT simulator in urological residency training: from needs analysis to validation. [22] How to teach the teacher to teach the TUR-B. [23] Dedicated teaching programs can improve the quality of TUR of non-muscle-invasive bladder tumours (NMIBT): experience of a single institution. 6 Impact of resident involvement in endoscopic bladder cancer surgery on pathological outcomes. 7 Impact of surgical experience on recurrence and progression after transurethral resection of bladder tumour in non-muscle-invasive bladder cancer. 8 The effect of resident involvement on perioperative outcomes in transurethral urologic surgeries. Few standardised TURBT programmes have been reported. de Vries et al [] conducted a validation study on a physical TURBT simulator and assessed performance in terms of procedure time, resection completeness, bladder perforations, and a global rating by video raters, and found that this approach could discriminate between different experience levels. The authors suggested the introduction of a time-defined training programme in urological residency but did not report data to support their statement []. Notably, the authors contributed with necessary knowledge via a needs analysis on the procedural steps and technical and nontechnical pitfalls in TURBT, which can support the design of a proficiency-based curriculum in TURBT (Kern’s step 2). In contrast to our patient-free SBT approach, two studies reported on changes to the classical apprenticeship model. Pycha and Palermo [] suggested a step-by-step progression programme for clinical procedures and emphasised the importance of undisturbed communication between the trainee and supervisor. The authors found that complication rates decreased on inclusion of a supervisor with better performance and concluded that teaching TURBT in the clinical setting is challenging. Brausi et al [] described an organisational intervention comprising routine flexible cystoscopy and a bladder diagram for diagnosis, video recordings of all TURBTs under the supervision of a senior urologist, monthly teaching meetings, and pathology assessed by a urologist for shared decision-making and surgeon feedback. They found improvements in DM presence and recurrence rates for specialists and trainees. Whereas both of these groups focused on improving teaching in the patient-related setting, our training programme focused on enhancing surgical skills in a patient-free environment, acknowledging that classical apprenticeship in TURBT compromises patient safety [].

[14] Bube SH, Kingo PS, Madsen MG, et al. Validation of a novel assessment tool identifying proficiency in transurethral bladder tumour resection: the OSATURBS assessment tool. J Endourol. In press. https://doi.org/10.1089/end.2021.0768. [24] Pietersen PI, Bjerrum F, Tolsgaard MG, et al. Standard setting in simulation-based training of surgical procedures. Ann Surg. In press. https://doi.org/10.1097/sla.0000000000005209. [20] [26] Validation of educational assessments: a primer for simulation and beyond. We assessed clinical performance using the OSATURBS tool with validity evidence reported according to Messick’s validity framework []. An established standard-setting method was used to determine the pass/fail score necessary for proficiency-based training, which is unfortunately often overlooked in research on surgical simulation []. Robust validity evidence should be explored using contemporary frameworks as recommended by the American Psychological Association in 1999 and endorsed internationally []. Despite these recommendations, a recent systematic review on surgical simulation and surgical skills found that only 6.6% of 498 studies reported contemporary validity evidence. The reporting quality was even poorer in urological studies (only 1.2% used the modern framework for validity evidence) []. As the urological community embraces competency-based medical education, we must report gold-standard evidence to support our assessments of future surgeons [].

[28] [17] At the beginning of the learning curve, the effect of each case performed is high, but the gain decreases with subsequent cases. Accordingly, we found that the inexperienced doctors improved after the VR training. As illustrates, there was a positive training effect in the novice and intermediate groups, with a breaking point at approximately 20 procedures. SBT did not change the performance for the most experienced doctors in the intermediate group or for the experienced group, which may be explained by the level of complexity of the simulator cases. This provides important knowledge on who to train (Kern’s step 2) when developing a complete TURBT curriculum. Our findings are in line with those of Thomsen and colleagues [], who investigated VR simulation training in cataract surgery and found positive effects on clinical procedures performed by novice and intermediate surgeons who had performed up to 75 cataract procedures on patients. Other studies on transfer have primarily assessed inexperienced residents and demonstrated improvements in clinical performance for different surgical procedures []. Neumann et al [] conducted a randomised trial comparing VR SBT (intervention) to an instructional video session and found that VR training had a greater effect on simulator scores. However, the study participants were medical students, and therefore the results are difficult to extrapolate to the suggested target for a TURBT VR SBT programme, namely residents in urological apprenticeship.

[29] Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer. [29] Reliable and valid assessment of competence in endoscopic ultrasonography and fine-needle aspiration for mediastinal staging of non-small cell lung cancer. [30] Our study has some limitations that need to be taken into account. Clinical assessments are prone to several biases, including rater bias. Therefore, we included video rater assessments to diminish the risk of social biases. Konge et al [] explored the assessment of endoscopic ultrasonography and fine-needle aspiration in mediastinal staging of non–small-cell lung cancer. They found that direct observation resulted in 10% lower scores for residents and 10% higher scores for consultants when compared to blinded video-based assessments []. In a study on assessment of flexible cystoscopy, Dagnaes-Hansen et al [] also found significant differences between direct and video-based assessments of urologist performance. These findings are in concurrence with our results and emphasise that even in a controlled research set-up and conducted in good faith, direct observation assessments are prone to bias.

[4] Consensus definition and prediction of complexity in transurethral resection or bladder endoscopic dissection of bladder tumours. The sample size in our study could result in type II errors. However, despite the low number of participants, we found significant positive training effects for novices. Variations in complexity for the procedures may confound our results. To account for TURBT case variation, we used strict inclusion criteria for TURBT cases and used mean performance scores for two cases before training and two cases after training. However, differences in case complexity could still influence the scores, and future studies should consider including assessment of each case, for example, by using the novel Bladder Complexity Checklist developed by a Delphi panel of international specialists and stakeholders in TURBT and bladder cancer treatment []. However, the strong correlations for the two pretraining procedures and the two post-training procedures indicate that the variation in complexity did not affect the effect sizes. Finally, the cost of the VR simulator could be a prohibitive factor for implementing a mastery learning programme in countries with limited financial resources.

Images Powered by Shutterstock