Patient safety and efficacy within a surgical innovation framework

Introduction

Innovation drives the advancement of surgical practice, ultimately improving the quality of healthcare for most patients. However, at points during the development, implementation, and dissemination of innovative technologies and techniques, patients may experience unintended, yet significant, harm (1). While some complications exist outside of the surgeon’s control such as previous undiagnosed faults in the technology/device itself or unknown side effects from utilizing the technology/device, many complications occur due to surgeon or operating team inexperience with the technology or technique (2,3). Until recently, the extent and severity of these complications were unknown due to the lack of a formal evaluative framework for surgical innovation.

Historically, surgical innovation enjoyed a privileged status outside of the scope of traditional investigation, driven, in part, by desperate patients seeking novel therapies to cure their challenging ailments (4). To this day, patients still equate new with best, a sentiment often times perpetuated by industry and hospitals promoting such novel devices or procedures (5). However, the main driver for the persistence of mainly trial and error processes to evaluate surgical innovation is that surgical therapies are complex with many potentially confounding variables. Unlike pharmacological development and evaluation where drug dosage can be adjusted, drug administration is standardized, and experimental arms can be blinded, surgical procedures are all-or-nothing interventions, experience variation in surgeon skill (e.g., learning curve) and non-surgical factors (e.g., anesthesia and perioperative care), and are difficult to blind. These challenges make assessment with randomized control trials impractical in many situations (6,7). Furthermore, insistence on adhering to such unrealistic evaluative methods may hinder surgical innovation. Alternative evidence-based methods are needed for practical and pragmatic evaluation of surgical innovation.

Surgical innovation is foundational to improving the human condition. Pragmatic and practical evaluation methods can simultaneously adhere to patient safety standards and stimulate the development of innovative technologies and techniques.

In 1999, the Institute of Medicine’s To Err Is Human publication brought patient safety concerns to the forefront and fundamentally altered perceptions of medical safety. This report hastened the maturation of the disciplines of health services and outcomes research and quality improvement as well as the development of new research methodologies and metrics to adequately address patient safety and healthcare quality concerns. Recently, these methods and metrics were adopted into frameworks to address the evaluation of surgical innovation more formally. The most well-cited framework, the IDEAL (Idea, Development, Exploration, Assessment, Long-term study) framework, consists of multiple stages describing surgical innovation and proposes methods, interventions, and outcomes that should be utilized at each stage to assure patient safety and innovation efficacy (Table 1) (8,9). This descriptive model characterizes the planning, evaluation, and reporting needed at each stage of the innovation process.

Surgical innovation lacks a gold standard definition (10,11). This has led to a regulatory and oversight grey area, especially regarding innovative surgical techniques or approaches as opposed to clearer standards for new surgical devices (1). The adoption and expansion of robotic-assisted surgical devices exemplifies this well. Robotic-assisted surgery combines the minimally invasive benefits of laparoscopic surgery including less postoperative pain, faster recovery time, and shorter hospital stays with high-definition visualization, enhanced dexterity with more degrees of freedom, tremor reduction, and improved ergonomics (12). The United States Food and Drug Administration (FDA) has approved robotic-assisted surgical devices for use in hysterectomy, prostatectomy, and colectomy based on short-term (30 days) patient follow up. However, the FDA has not evaluated the safety or effectiveness of robotic-assisted surgical devices for the prevention or treatment of cancer (13). Yet, these devices are currently used to treat patients with cancer every day.

With these considerations in mind, we will discuss the innovation life cycle of a robotic distal pancreatectomy using the IDEAL framework to highlight issues of patient safety and efficacy (9).

Pre-clinical stage

In general, surgeons address pathology by removing or repairing organs and tissues while striving to avoid disrupting non-pathologic areas. In many instances, the desire to achieve these ends more efficiently and safely drives surgical innovation. Furthermore, the ease of adoption and steeper learning curve has contributed to a rapid adoption of robotic-assisted surgery to treat many diseases.

In the pre-clinical stage, investigators determine the feasibility of using the device or performing the procedure. When considering implementation of a robotic-assisted approach for distal pancreatectomy, common study designs include simulation, animal, and cadaveric studies. The surgeon must first be comfortable performing a distal pancreatectomy with a standard laparoscopic or open approach prior to considering the robotic-assisted approach in case the surgeon is unable to proceed with the robotic approach (e.g., uncontrollable hemorrhage, unclear anatomy, technical issue) and needs to convert to an alternative approach to continue the procedure. If the surgeon does not have experience on the robotic platform, they should start training with online didactics, virtual reality simulation, dry lab simulation, and, finally, an available certification course which often includes an animal lab. After obtaining sufficient proficiency in the general use of the robotic platform, a human cadaveric study should be performed when possible. This will allow the surgeon to fully describe the procedure steps and the procedure efficacy (i.e., did the procedure achieve its intended outcome). Ideally, the same operating room team that will participate in the first live human procedure should participate in the cadaveric study. This way team members can present and discuss issues and risks not immediately apparent to the surgeon.

While this stage does not include any specific safety concerns as there is no patient interaction, surgeons need to consider possible safety risks that may arise throughout the procedure and investigate ways to mitigate these risks. For example, is the patient optimally positioned on the operating room table to avoid potential complications (e.g., neuropathy, pressure wounds, joint dislocation, etc.). In addition, teams need to have a clear protocol for emergent situations that includes rapid conversion to open surgery. Furthermore, a structured safety review of innovative procedures or devices by an independent and dedicated team of trained surgical quality officers and experienced operative personnel may improve patient safety and decrease time for broader institutional approval (14). Results from animal or cadaveric studies should be published in peer-reviewed journals to include a detailed description of the procedure, documentation of any risks and safety concerns that arose and how they were addressed, and the procedure efficacy. In addition, many countries require registration with a governing body when attempting to study a device that poses a significant risk to patients prior to first-in-human application. For example, an investigational device exemption (IDE) is required in the United States when attempting to use the robotic platform for a procedure that it has not already been approved for. Finally, modeling studies evaluating the overall impact on healthcare costs should be performed when considering adoption of a new device or surgical approach such as a robotic-assisted distal pancreatectomy. Costs remain a primary barrier to broader adoption of innovative procedures and devices including robotic platforms (15). For robotic distal pancreatectomy, cost-effectiveness studies should evaluate value compared to laparoscopic and open approaches and include both direct and indirect costs for patients and institutions especially in light of potential insurance coverage and reimbursement concerns.

Stage 1: Idea—first in human use

This stage describes the first application of a new device or procedure in a patient. An innovative procedure needs to be conducted in compliance with institutional requirements for such interventions (i.e., institutional review board approval). Many times, financial support for trials evaluating innovative devices or surgical approaches is provided by industry or the surgeons performing the procedures are supported by industry in other ways that may influence outcomes. For example, Intuitive Surgical supports many recent and current trials evaluating the Da Vinci Surgical System^® (Intuitive Surgical Inc., Sunnyvale, CA, USA). Transparent reporting of potential conflicts of interests is necessary and foundational to assure appropriate ethical conduct. In addition, these patients are often highly selected, and the selection criteria should be described in detail. Patients either not selected for the procedure or that refused study participation should also be documented and explained with reason. Investigators need to obtain patient consent that includes discussion of the innovative nature of the procedure, surgeon experience, and disclosure of conflicts of interests. For surgeons with little clinical experience on the robotic platform, a trained robotic preceptor present at the operation can provide guidance in troubleshooting issues related to robotic performance.

The primary objective is to generate preliminary data on the technical success of the procedure, relevant clinical measures, and any complications or adverse events. Prospectively maintained databases should be created to collect these data. For robotic distal pancreatectomy, relevant data points include intraoperative blood loss, operative time, post-operative pancreatic fistula as well as indication-specific outcomes (e.g., cancer) such as margin positive rate and number of lymph nodes harvested. The surgeon and operative team should debrief immediately after the procedure and at a future time point to discuss any technical, procedural, or safety concerns. Adverse events should be documented and closely monitored including discussion before an independent review board. Procedures performed on a laparoscopic or robotic platform can be recorded. Video review can demonstrate efficacy and help identify technical issues, complications, or areas needing improvement. Furthermore, video recordings can be used in future publications or presentations. A case report submitted for peer review may help identify areas of potential risk and offer suggestions on improving surgical technique.

Stage 2a: Development—toward stabilization of the technique

This stage is characterized by rapid iterative modification of surgical technique and operative indications. The procedure is refined based on experience and evaluation. Methods of evaluation are predominately single center prospective cohort evaluations with cases presented sequentially along with relevant clinical outcomes, complications, and adverse events. Sequentially plotted outcomes and quality metrics can demonstrate significant trends regarding efficacy and safety. Data collection should continue prospectively. In addition to a discussion about the experimental nature of the procedure and limited surgeon experience, patient consent should include known outcomes and risks from stage 1. Case series publications should include a clear technical description of the initial procedure followed by details about how, when, and why technical or procedural modifications were made. Specific points of interest for distal pancreatectomy include, but are not limited to, location of port placement, use and location of an assistant port(s), specific robotic instruments used, medial to lateral or lateral to medial dissection, technique of splenic vessel ligation (e.g., clips or staple) and technique of specimen extraction. Changes to the surgical indication or patient selection criteria need to be highlighted and explained. At the completion of this stage, intentional iterative change should be complete.

Stage 2b: Exploration—bridge to a pivotal trial

During this stage, the innovation spreads to a larger group of surgeons and patients. The goal is to demonstrate reproducibility while working towards a definitive comparison trial versus current best practice. Inclusion criteria expands as surgeons become more skilled with the procedure to include all patients that may benefit, not just the highly selected patients. For distal pancreatectomy, initial inclusion criteria may have been limited to healthy patient with easily resectable benign conditions. Expanded criteria may include patients with comorbidities and malignant conditions such as pancreatic adenocarcinoma. Learning curve assessment can evaluate surgeon efficiency and procedural skill. While the innovative procedure is now well defined, small inter-surgeon differences in technique need to be documented and evaluated with preplanned subgroup analysis. Patient characteristics and outcomes will continue to be collected in a prospectively maintained database.

As the innovation spreads to other institutions, surgeons and, if possible, their teams should strongly consider visiting surgeons and institutions with experience in the procedure to observe and discuss their experience focusing on feasibility and patient safety aspects. Surgeons attempting to adopt the procedure at their institution should proceed through similar preparatory steps as discussed in the pre-clinical stage, and stages 1 and 2a such as simulation, animal, and/or cadaveric practice, preceptoring, and data and technical review. Required preparation will vary based on surgeon robotic experience. Surgeons should continue to track patient safety data, including adverse events, prospectively. After a significant number of patients have undergone the procedure, investigators can develop patient-centered outcomes to assess benefit from a patient perspective. Furthermore, additional study endpoints should be identified that reflect the values of surgeons and patients including clinical outcomes that demonstrate procedure efficacy and safety measures.

Stage 3: Assessment—pivotal study

In this stage, a pivotal comparison occurs between the innovative procedure and, if possible, the current gold standard approach. When feasible, a multicenter, multi-surgeon randomized trial should be performed with an accompanying cost-effectiveness analysis to quantify the value of the innovation as costs may vary between institutions. Cluster-randomized or expertise-based randomized controlled trials are possible trial variations. In situations where randomization is not possible, controlled interrupted time series or observational designs using appropriate statistical methodologies (e.g., propensity scoring or multivariable adjustment) to minimize bias should be considered. A full safety analysis should be performed documenting all adverse events and their severity. The primary safety and efficacy endpoints identified in stage 2b should be evaluated in this expanded trial. Trials should be registered with the appropriate governing body and the relevant guidelines for reporting study results should be followed (e.g., CONSORT).

Surgeon skill accounts for a significant proportion of the variation in outcomes and adequately accounting for this potentially confounding factor remains a significant challenge in surgical trials (16). However, advances in medical imaging analytics (e.g., radiomics), artificial intelligence (e.g., machine learning), and kinematic evaluation have improved the accuracy and objective performance assessment of surgeon proficiency (17). Machine learning-based assessment tools determine surgeon performance with accuracy rates over 80% (17). While technological advancement will continue to improve our ability to discern and objectively quantify surgeon proficiency, available evidence supports the inclusion of surgeon skill assessment in surgical trials (18).

Stage 4: Long-term study—identify rare and later outcomes

The main purpose of the final stage is to establish safety and efficacy in a larger, more heterogenous population as the procedure becomes the new best practice as determined by the definitive comparison study performed in stage 3. Registries should be established to collect data long-term so investigators can recognize late or uncommon safety outcomes and evaluate trends in outcomes to identify variations in quality. For example, in cases where robotic distal pancreatectomy was performed for a malignant condition, relevant long-term outcomes include local recurrence, disease-free survival, and overall survival. Furthermore, registries provide the opportunity for ongoing feedback to surgeons and industry. Finally, registries should be monitored to identify changes in procedure utilization or patient inclusion.

Conclusions

Surgical innovation drives healthcare quality. Without a formal evaluation framework, a trial-and-error approach may result in significant patient harm. A prudent and structured evaluation framework focused on patient safety and efficacy, such as that found in the IDEAL framework, provides robust assessment of innovative technologies and techniques. Reporting of safety measures and clinical outcomes should be standardized to allow for valid comparison between studies. The large-scale use of prospective databases and registries improves outcomes and safety monitoring in both the short- and long-term. While randomized controlled trials remain the gold standard for efficacy and safety assessment, alternative study designs such as interrupted time series or observational cohort studies provide efficient ways to evaluate innovative surgical technologies or techniques. Furthermore, machine learning-based assessment tools provide accurate and objective surgeon performance assessment and can be incorporated into surgical trials to account for variation in surgeon skill. Finally, transparency throughout the innovative process should be coupled with full and accurate reporting of outcomes. A pragmatic regulatory framework can provide evidence-based evaluation while facilitating surgical innovation and minimizing patient risk.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the editorial office, Laparoscopic Surgery for the series “Implementation of New Technology/Innovation in Surgery”. The article has undergone external peer review.

Peer Review File: Available at https://ls.amegroups.com/article/view/10.21037/ls-22-36/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://ls.amegroups.com/article/view/10.21037/ls-22-36/coif). The series “Implementation of New Technology/Innovation in Surgery” was commissioned by the editorial office without any funding or sponsorship. AE served as the unpaid Guest Editor of the series and serves as an unpaid editorial board member of Laparoscopic Surgery from February 2022 to January 2024. AT served as the unpaid Guest Editor of the series and serves as an unpaid editorial board member of Laparoscopic Surgery from December 2020 to November 2022. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Barkun JS, Aronson JK, Feldman LS, et al. Evaluation and stages of surgical innovations. Lancet 2009;374:1089-96. [Crossref] [PubMed]
2. Birkmeyer JD, Stukel TA, Siewers AE, et al. Surgeon volume and operative mortality in the United States. N Engl J Med 2003;349:2117-27. [Crossref] [PubMed]
3. Hopper AN, Jamison MH, Lewis WG. Learning curves in surgical practice. Postgrad Med J 2007;83:777-9. [Crossref] [PubMed]
4. Wind JJ, Anderson DE. From prefrontal leukotomy to deep brain stimulation: the historical transformation of psychosurgery and the emergence of neuroethics. Neurosurg Focus 2008;25:E10. [Crossref] [PubMed]
5. Reitsma AM, Moreno JD. Ethics of innovative surgery: US surgeons’ definitions, knowledge, and attitudes. J Am Coll Surg 2005;200:103-10. [Crossref] [PubMed]
6. McLeod RS, Wright JG, Solomon MJ, et al. Randomized controlled trials in surgery: Issues and problems. Surgery 1996;119:483-6. [Crossref] [PubMed]
7. Solomon MJ, McLeod RS. Surgery and the randomised controlled trial: past, present and future. Med J Aust 1998;169:380-3. [Crossref] [PubMed]
8. McCulloch P, Altman DG, Campbell WB, et al. No surgical innovation without evaluation: the IDEAL recommendations. Lancet 2009;374:1105-12. [Crossref] [PubMed]
9. Hirst A, Philippou Y, Blazeby J, et al. No Surgical Innovation Without Evaluation: Evolution and Further Development of the IDEAL Framework and Recommendations. Ann Surg 2019;269:211-20. [Crossref] [PubMed]
10. Birchley G, Ives J, Huxtable R, et al. Conceptualising Surgical Innovation: An Eliminativist Proposal. Health Care Anal 2020;28:73-97. [Crossref] [PubMed]
11. Blakely B, Selwood A, Rogers WA, et al. Macquarie Surgical Innovation Identification Tool (MSIIT): a study protocol for a usability and pilot test. BMJ Open 2016;6:e013704. [Crossref] [PubMed]
12. George EI, Brand TC, LaPorta A, et al. Origins of Robotic Surgery: From Skepticism to Standard of Care. JSLS 2018;22:e2018. [Crossref] [PubMed]
13. UPDATE: Caution with Robotically-Assisted Surgical Devices in Mastectomy: FDA Safety Communication. 8/20/2021. Available online: https://www.fda.gov/medical-devices/safety-communications/update-caution-robotically-assisted-surgical-devices-mastectomy-fda-safety-communication
14. Marcus RK, Lillemoe HA, Caudle AS, et al. Facilitation of Surgical Innovation: Is It Possible to Speed the Introduction of New Technology While Simultaneously Improving Patient Safety? Ann Surg 2019;270:937-41. [Crossref] [PubMed]
15. Cole A, O’Neill P, Sampson C, et al. Barriers to Uptake of Minimal Access Surgery in the United Kingdom. OHE Consulting Report, London: Office of Health Economics; 2018.
16. Curtis NJ, Foster JD, Miskovic D, et al. Association of Surgical Skill Assessment With Clinical Outcomes in Cancer Surgery. JAMA Surg 2020;155:590-8. [Crossref] [PubMed]
17. Lam K, Chen J, Wang Z, et al. Machine learning for technical skill assessment in surgery: a systematic review. NPJ Digit Med 2022;5:24. [Crossref] [PubMed]
18. Ghaferi AA, Dimick JB. Considering Surgeon Skill in Randomized Clinical Trials. JAMA Surg 2020;155:599. [Crossref] [PubMed]