A review of innovations in surgical education: melding metacognition and technology

Introduction

One of the fundamental questions in surgical education is how to optimize training on a personal level to accelerate an individual’s growth into expertise. The classic S-shaped learning curve illustrates the process by which an individual gains proficiency through experience (Figure 1). The initial phase, represented by the steep part of the curve, consists of building foundational knowledge that allows the individual to rapidly mature and gain skills. This period of explosive growth leads to a plateau, at which point additional experience leads to only incremental improvements. Traditionally, the key determinate to achieving expertise is the amount of time on a task the individual has, either the number of hours or iterations that are performed. In most cases, the individual needs to perform a given task hundreds or thousands of times to achieve fluency. This concept served as the foundation of surgical training over the last century, referred to as the Halstedian training model. In 1897, William S Halsted, the chairman of the Department of Surgery at Johns Hopkins University, first introduced the educational system in which surgical trainees learned to properly manage a disease process or perform an operation through repetition, trial-and-error, and sheer volume. In this on-the-job training model, surgical residents essentially resided in the hospital in order to become proficient and complete their training (1). However, in the 1980s, the tragic death of Libby Zion led to a plummeting tolerance for medical error and galvanized resident duty hour reform efforts. While laudable goals, the unintended consequence is that surgical trainees now graduate with far fewer opportunities to gain critical experience as previous generations, limiting their ability to become experts through volume alone (1). For a model based on volume, duty hour restrictions are a fundamental problem that require a re-envisioning of the system that compensates for this gap in experience. In other words, surgical education requires a new model that shifts the learning curve to the left and steepens it to enable trainees to achieve fluency with fewer iterations in real world scenarios. Fortunately, developments in adult learning theory and technology hold promise for creating such a system.

Figure 1 The learning curve. Adapted by authors James Lee and Catherine McManus from Hermann Ebbinghaus’ 1885 depiction of the learning curve.

A successful training paradigm should allow the trainee to learn foundational material in advance so that when faced with a real-world situation, the individual does not need to focus on rudimentary details. Lessons from the field of metacognition, or “how we think about thinking”, have the potential to steepen the learning curve by leveraging the cognitive aspects of achieving expertise and coupling them with advances in technology to practice new skills in a low-stakes environment. More specifically, the metacognitive principles of cognitive pre-training, deliberate practice, and building better mental models are particularly useful in optimizing surgical training. The concept of applying metacognition and technology to the advancement of surgical education was recently presented as the Kui and Wai Fong Lectureship at the 6^th Annual Meeting of the Society of Asian Academic Surgeons and will be published in the Journal of Surgical Research as a memorialization of the meeting (2). The purpose of this review is to expand upon the topics that were discussed and provide a broad summary of the impact of technology on surgical education.

Cognitive pre-training

Cognitive pre-training is a method in which the learner is given access to all of the relevant information needed to learn the foundations of their job prior to participating in real-world situations. The learner absorbs the relevant material (videos, training manuals, simulations, etc.) in a low stakes environment so that when confronted with a real-world situation, the learner has already become skilled at the fundamentals. An analogy from the non-medical world is that National Aeronautics and Space Administration (NASA) astronauts spend countless hours in class and simulators before they undertake a mission. Experts ranging from all walks of life have adopted this method of mental practice because it is highly effective. For example, Driskell and colleagues conducted a meta-analysis looking at various skills from violin playing to dart-throwing and concluded that thinking about performing a task achieves almost 2/3 the benefit of actually performing the task (3). In other words, mental rehearsal is almost as beneficial as physical practice. However, the effectiveness of pre-training depends on the quality of the educational material. If programs offer high-quality, comprehensive pre-training material, they stand an excellent chance of training excellent surgeons. However, most educational platforms focus on generalizable facts, such as indications for an operation and basic anatomy, and not on the critical nuances of practice such as the dozens of potential pitfalls and pearls that may arise during surgery or the precise technique of individual surgeons. Therefore, the resources that most trainees typically have access to do not provide the level of detail they need to accelerate their learning curve.

For many, the first exposure to the field of surgery happens in medical school. After years of traditional classroom-based learning, medical students venture into the clinical setting and the focus of teaching shifts from a didactic foundation to a practical one. This is especially evident on the surgery clerkship rotation when students are first exposed to the operating room (OR), which can be an unfamiliar and often intimidating environment (4). To prepare students for this new environment, they have access to a written guide accompanied by an orientation and a tour of the OR. While knowing physically where the OR is and what it looks like is logistically important, being in the OR with a patient during a surgery is an entirely different experience. In a meta-analysis published in 2019, Hexter and colleagues found that emotional, socio-environmental and organizational factors were key elements that influenced medical student learning in the OR (5). Students reported that barriers to learning in the OR included not knowing what to expect, unfamiliarity with the different staff roles (including their own), and not understanding basic OR etiquette. This perceived lack of preparation led to reports of feeling nervous in 96% of students and having a fear of incompetence in 89% of students (5,6). In order to improve the initial experience in the OR, which can influence the student’s perception of the field of surgery and their ability to learn on the surgery rotation, students must become as familiar as possible with the environment ahead of time. Stuparich and colleagues found that creating an OR orientation video that demonstrates everything from how to introduce yourself to the patient and the team members in the OR, to how to maintain sterile technique, to where to stand during the surgery improved the student’s subjective experience and their objective performance based on feedback from OR staff (7). Furthermore, Patel and colleagues demonstrated that having students participate in a virtual OR session or orientation in a simulated OR suite led to an improvement in a student’s skills, knowledge and attitudes toward the OR (8). We have created a video-based curriculum that prepares medical students for their experience in the OR and are currently studying its effectiveness compared to traditional means of preparation (written materials, in person tour, etc.). By utilizing technology such as a video or virtual reality (VR) platform, students will already have mastered the basics of the OR environment and only then are they able to appreciate and take advantage of the unique hands-on learning opportunities on their surgery rotation.

Technology provides a means of building and disseminating more robust educational content. For example, a wiki is a type of webpage that allows a community of users to contribute content and maintain a robust knowledgebase. The wiki serves as a dynamic resource that gives learners immediate access to the most relevant information. Expanding on this concept, we built a Wikipedia for medicine called COACH, a multimedia platform with expert review. COACH utilizes technology to collect and disseminate specific, granular details about a disease or an operation that can be updated by any user with expert oversight. Consequently, trainees have constant access to the most up-to-date surgical techniques and surgeon preferences and can maximize cognitive pre-training efforts to prepare for the clinical setting. In 2016, Fingeret et al. conducted a retrospective review of medical students to determine the impact of COACH on medical student performance during the surgery clerkship. On multivariable analysis, the authors found that performance on the National Board of Medical Examiners (NBME) Surgery Shelf Exam, evaluations, and the use of the COACH platform predicted improved performance on the surgery clerkship. Tellingly, higher rates of utilizing COACH led to better performance. In contrast, the number of cases the student scrubbed into did not improve performance. In other words, the results suggest that utilizing a virtual learning platform like COACH had more of an impact on student performance compared to being in the OR (9).

COACH aims to build local knowledgebases in all areas of medicine across all institutions and unite them into a universal knowledgebase controlled by a powerful search engine. Innovative platforms like COACH can serve as the foundation for cognitive pre-training and provide trainees with the educational material that can advance them two-thirds the way to expertise. Then comes practice.

Deliberate practice

Experts in the field of deliberate practice argue that a specific type of practice is critical to becoming an expert. For example, Mihaly Csikszentmihalyi describes the state of flow, which is characterized by being completely immersed in a task so that one’s thoughts and actions are in perfect harmony and autotelism (the activity is its own reward). In order for an activity to reach a state of flow it must have a clear goal, be slightly more challenging than their skill level, provide constant feedback to the learner on their progress, and keep them fully engaged in the task (10). For example, in rock-climbing an individual has a clear goal (to get to the top) by a route they feel is challenging enough but not too challenging and they get constant performance feedback (they fall or continue to climb).

Another form of ideal practice is known as Deliberate Practice, defined by Anders Ericsson as repetitive practice that incorporates feedback and correction. For example, when studying violinists at a leading conservatory in Germany, Ericsson and colleagues found that it was not only the number of hours of practice that distinguished an expert from the rest, but also the form that practice took (11). It was not enough to practice just by doing repetitions, in fact, rote practice often ingrained bad habits. Instead, one must harness the wisdom of a more skilled expert or coach to course correct in the moment. Like Csikszentmihalyi, Ericsson found that effective practice requires an individual to push beyond their comfort zone and use expert feedback to identify weaknesses and adjust. In surgery, there are abundant measurable outcomes to reflect success (bleeding, complications, etc.) but having access to an expert to help make adjustments is limited. However, technology can facilitate access to the systems that experts use to become experts, which can sometimes be better than having an in-person coach. For example, advances in robotic surgery have led to the development of machine learning algorithms that have the potential to provide objective feedback to surgical trainees. In 2018, Fard et al. used machine learning algorithms to analyze movement trajectory data of the robotic arms and build a framework that could distinguish novice versus expert surgeons (12). Additionally, Winkler-Schwartz and colleagues demonstrated the ability of machine learning to determine the level of expertise among neurosurgeons, fellows, residents, and medical students when performing a neurological tumor resection with 90% accuracy (13). Thus, surgical trainees in the future may not only be able to learn from real-world expert clinical instruction but also from objective feedback provided by high quality, robust machine learning algorithms (14).

In the world of surgery, building high quality algorithms must be done in a structured, organized fashion in order to translate the “art of surgery” into objective measures. One way to accomplish this is by using qualitative research tools such as the Delphi method.

The Delphi method is technique that brings structure and order to expert decision making and can ultimately result in group consensus on a given topic or lead to the development of a systematic approach to a given task (15). The first round of the Delphi analysis is a qualitative collection of data in which a diverse number of experts are asked open-ended questions to collect key insights. That data is then organized into common themes and collated into a survey tool. Using validated instruments, the survey undergoes multiple rounds of feedback from experts until a specified stop criterion is reached, for example reaching group consensus or completing a set number of rounds (15).

One potential application of the Delphi method is to develop formal expectations of medical students in the OR. One educational strategy to improve learning in the OR is to not only have a clear introduction and orientation as previously mentioned, but also to create clear learning objectives for students in the OR (5). The teaching priority in the OR is often largely focused on the surgical resident or fellow developing and refining their surgical technique (16). Residents and faculty may rely on a “hidden curriculum” or a set of unwritten rules for how medical students are expected to behave and succeed in the OR. This leaves the student with the daunting task of fulfilling an undefined role and may lead to barriers in learning and an inability for the student to demonstrate their knowledge (17). Consequently, the distinct role of the medical student in the OR should be explicitly defined. Yule and colleagues used a modified Delphi process to establish consensus on essential non-technical skills for surgical trainees. Given that the development of technical skills is not the priority for medical students in the OR, a similar methodology could be adapted to create learning objectives specific for the student that focus on a demonstration of knowledge, communication, situational awareness, teamwork and professionalism (18). Development of these objectives would provide clarity to the students on what and how to prepare and has the potential to improve the learning environment. Furthermore, these learning objectives could be incorporated into a checklist and implemented via a tool such as a smartphone application, for faculty and residents to provide feedback and assess a student’s performance.

Another way to use the Delphi method is to build an expert system for an operation. For example, Madani et al. used the Delphi method to construct an expert system for thyroidectomy (19). While a group of surgeons are unlikely to agree on a single best approach to a given operation, this methodology highlights the principles that are most important and minimizes the less important items such as personal preference for instruments. Experts then rank the importance of specific items and based on the overall group input, items that are critical to achieving expertise can be identified. Along with Dr. Madani, we utilized the Delphi method to create a conceptual framework for laparoscopic adrenalectomy based on interviews and the literature (20). Experts from the American Association of Endocrine Surgeons (AAES) identified 60 tasks, 55 cognitive behaviors, and 84 potential errors that were funneled into 8 procedural steps and 6 fundamental principles. The steps of the operation were codified into a flowchart that demonstrated not only the basic information but also emphasized critical principles and expert advice. Based on how the experts rated a given principle, learners can appreciate its level of importance. As more experts contribute, the tool becomes more sophisticated and robust. Consequently, any trainee seeking to learn a laparoscopic adrenalectomy will be able to benefit from advice from a large community of experts. This framework lays the foundation for creating a video library of directed clips that illustrate a given step of the operation. Ideally this type of library would be created by specialty societies, such as the AAES, rather than individual institutions in order to capture opinions from a diverse group. Frameworks such as these may allow more effective dissemination of expert knowledge, which in turn will maximize the ability of trainees to perform deliberate practice. One of the critical end results of lots of deliberate practice is building more robust and numerous mental models.

Building better mental models

A mental model is defined as visualizing a whole entity as more than just the sum of individual parts. For example, when identifying the recurrent laryngeal nerve in thyroid surgery, a novice typically runs through each of the criteria used to find the nerve: it travels perpendicular to the inferior thyroid artery in the tracheaoesophageal groove and lies anterior to the upper parathyroid and posterior to the lower parathyroid. However, as the novice gains more experience, they eventually stop actively going through the specific criteria to locate it and can rely on their innate sense of where it is. An expert endocrine surgeon has had so much deliberate practice and has seen so many iterations and variations of the nerve that they are able to look at the field and immediately tell you where the nerve is located. Deliberate practice leads to improved mental models, which leads to more effective practice, which then further refines mental models.

Another key element in achieving expertise is the quality and quantity of the mental models that one has access to. In the Halstedian training model, the most effective way to create robust surgical mental models was to observe and perform an operation thousands of times. However, as we shift away from this training paradigm, there is a need to create high quality, accessible mental models. One way that technology achieves this goal is in the development of more sophisticated anatomic imaging. In endocrine surgery for example, 4D CT scans can assist in identifying parathyroid glands by demonstrating the washout characteristics of contrast over time. For individuals who have a difficult time translating a 2D CT image into a 3D picture, radiologists are able to build a 3D reconstruction of the scan, allowing the novice to better visualize the anatomy and plan the operation. In more complex cases, such as a pelvic paraganglioma involving the iliac vessels, a 3D anatomic reconstruction that allows the surgeon to visualize the anatomy from all dimensions can help immensely in planning the operative approach. Furthermore, printing 3D anatomic models brings mental imagery into a physical reality, taking things one step closer to the real world. However, such resources may not be easily accessible or readily available to all, which can limit their impact and effectiveness. Instead, individuals could ideally take advantage of advances in technology to train their brains to build better mental models.

In order to improve mental imagery in training, investigators have tested the role of virtual environments, such as systematic video game training, in enhancing surgical skills. For example, Schlickum and colleagues found that groups randomized to playing 30–60 minutes of videos games per day had better performance in endoscopic simulators (21). Similar findings supported this concept that cognitive training with virtual environments has the potential to improve surgical skills by providing an immersive, hands-on operative experience. For example, in 2020 Blumstein et al. conducted a randomized trial comparing VR training to standard guide (SG) training for intramedullary nailing (IMN) of a tibia among first- and second-year medical students. The authors found that the VR group had higher post training assessment scores, a higher percentage of steps completed correctly, and higher average improvement compared to the SG group (22). Additionally, VR trainers provide the flexibility for trainees to instantly and independently access high-quality skills training at their convenience. Furthermore, machine learning algorithms may be able to specifically tailor the VR experience to the individual trainee’s deficits and provide personalized, objective feedback (14).

In addition, lessons in creating mental models from the field of medical illustration may allow individuals to become better at making mental models. One of the core skills of an artist is to translate mental representations into 3D space. In order to practice this task, individuals can perform specific exercises such as taking an image of a flattened-out-cube with different images on each surface, mentally reconstructing the cube, and correctly matching it with the corresponding 3D image. In other words, the individual has to mentally manipulate the 2D image and visualize it as a 3D object. Surgeons must also become experts at translating mental imagery into reality. We are currently developing a set of exercises designed to improve mental modeling for surgery-specific skills, such as how to effectively retract tissue. Eventually we hope to disseminate these exercises in an app-based platform and then test whether trainees’ performance on the Fundamentals of Laparoscopic Surgery and Fundamentals of Endoscopic Surgery tests improve pre and post intervention.

Summary

In summary, utilizing advances in technology to explore the key principles of metacognition can lead to a new training paradigm that allows individuals to reach expertise more effectively. While the new paradigm still must offer real world experience, extensive pre-training and practice will allow learners to become experts of the foundational skills and basic knowledge ahead of time. That way, when the learner is in the real situation, they can focus on the lessons that can only be learned on the wards or in the OR with patients.

Acknowledgments

Funding: None.

Footnote

Provenance and Peer Review: This article was commissioned by the Guest Editors (Allan Tsung and Aslam Ejaz) for the series “Implementation of New Technology/Innovation in Surgery” published in Laparoscopic Surgery. The article has undergone external peer review.

Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at https://ls.amegroups.com/article/view/10.21037/ls-22-11/coif). The series “Implementation of New Technology/Innovation in Surgery” was commissioned by the editorial office without any funding or sponsorship. JAL is the Founder of Collectedmed, of which COACH is a subsidiary and he currently has no financial ties to the company. JAL is secretary of American Association of Endocrine Surgeons The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Wallack MK, Chao L. Resident work hours: the evolution of a revolution. Arch Surg 2001;136:1426-31; discussion 1432. [Crossref] [PubMed]
2. Lee JA. Kui and Wai Fong Lectureship: Optimizing Surgical Education Through Metacognition and Technology. J Surg Res 2022;277:A12-A17. [Crossref] [PubMed]
3. Driskell JE, Copper C, Moran A. Does Mental Practice Enhance Performance? J Appl Psychol 1994;79:481-91. [Crossref]
4. Croghan SM, Phillips C, Howson W. The operating theatre as a classroom: a literature review of medical student learning in the theatre environment. Int J Med Educ 2019;10:75-87. [Crossref] [PubMed]
5. Hexter AT, O'Dowd-Booth C, Hunter A. Factors that influence medical student learning in the operating room. Med Teach 2019;41:555-60. [Crossref] [PubMed]
6. Fernando N, McAdam T, Cleland J, et al. How can we prepare medical students for theatre-based learning? Med Educ 2007;41:968-74. [Crossref] [PubMed]
7. Stuparich M, Melnyk A, Hohmann H, et al. The effectiveness of an operating room etiquette video on medical student comfort in the gynecologic operating room. Education in the Health Professions 2020;3:110. [Crossref]
8. Patel V, Aggarwal R, Osinibi E, et al. Operating room introduction for the novice. Am J Surg 2012;203:266-75. [Crossref] [PubMed]
9. Fingeret AL, Martinez RH, Hsieh C, et al. Watch what happens: using a web-based multimedia platform to enhance intraoperative learning and development of clinical reasoning. Am J Surg 2016;211:384-9. [Crossref] [PubMed]
10. Csíkszentmihályi M. Flow and the psychology of discovery and invention. New York: Harper Collins; 1996.
11. Ericsson K, Krampe R, Tesch-Römer C. The Role of Deliberate Practice in the Acquisition of Expert Performance. Psychol Rev 1993;100:363-406. [Crossref]
12. Fard MJ, Ameri S, Darin Ellis R, et al. Automated robot-assisted surgical skill evaluation: Predictive analytics approach. Int J Med Robot 2018; [Crossref] [PubMed]
13. Winkler-Schwartz A, Yilmaz R, Mirchi N, et al. Machine Learning Identification of Surgical and Operative Factors Associated With Surgical Expertise in Virtual Reality Simulation. JAMA Netw Open 2019;2:e198363. [Crossref] [PubMed]
14. Rogers MP, DeSantis AJ, Janjua H, et al. The future surgical training paradigm: Virtual reality and machine learning in surgical education. Surgery 2021;169:1250-2. [Crossref] [PubMed]
15. McMillan SS, King M, Tully MP. How to use the nominal group and Delphi techniques. Int J Clin Pharm 2016;38:655-62. [Crossref] [PubMed]
16. Irani JL, Greenberg JA, Blanco MA, et al. Educational value of the operating room experience during a core surgical clerkship. Am J Surg 2010;200:167-72. [Crossref] [PubMed]
17. Rogers DA, Boehler ML, Roberts NK, et al. Using the hidden curriculum to teach professionalism during the surgery clerkship. J Surg Educ 2012;69:423-7. [Crossref] [PubMed]
18. Yule S, Gupta A, Blair PG, et al. Gathering Validity Evidence to Adapt the Non-technical Skills for Surgeons (NOTSS) Assessment Tool to the United States Context. J Surg Educ 2021;78:955-66. [Crossref] [PubMed]
19. Madani A, Watanabe Y, Vassiliou M, et al. Defining competencies for safe thyroidectomy: An international Delphi consensus. Surgery 2016;159:86-94, 96-101. [Crossref] [PubMed]
20. Madani A, Grover K, Kuo JH, et al. Defining the competencies for laparoscopic transabdominal adrenalectomy: An investigation of intraoperative behaviors and decisions of experts. Surgery 2020;167:241-9. [Crossref] [PubMed]
21. Schlickum MK, Hedman L, Enochsson L, et al. Systematic video game training in surgical novices improves performance in virtual reality endoscopic surgical simulators: a prospective randomized study. World J Surg 2009;33:2360-7. [Crossref] [PubMed]
22. Blumstein G, Zukotynski B, Cevallos N, et al. Randomized Trial of a Virtual Reality Tool to Teach Surgical Technique for Tibial Shaft Fracture Intramedullary Nailing. J Surg Educ 2020;77:969-77. [Crossref] [PubMed]