Evaluation of visual ergonomics in microsurgery: a real-time video processing solution
Gaukhar Mukash, Paavo Vartianen, Mastaneh Torkamani-Azar, Zeynel Karadis, Mehdi Faraz, Roman Bednarik, Pasi A. Karjalainen, Matti Iso-Mustajärvi, Ahmed Hussein
TL;DR
This study evaluates visual challenges in microsurgery and tests new technologies to improve surgeons' ergonomics and efficiency.
Contribution
The study introduces and evaluates two novel assistive technologies: context-preserving magnification and instrument transparency.
Findings
69% of surgeons expressed interest in testing prototypes of the proposed solutions in real surgery.
The context-preserving magnification solution reduced task completion time by 40% for 60% of participants.
Instrument obstruction and loss of context remain significant challenges in high magnification microsurgery.
Abstract
Surgeons’ visual and ergonomical challenges are long-standing concerns since the use of microscopes in surgical procedures. Although devices have been improved in the last few decades, the problem of narrow visual fields in high magnification surgeries persists. This study aims to identify the visual ergonomics challenges among microsurgeons and assess the need for novel assistive surgical solutions. The study consisted of two parts: a survey and a hands-on experiment. Sixteen surgeons from neurosurgery and otolaryngology (ENT) specialties were invited to semi-structured interviews on challenges encountered when using surgical microscopes and evaluation of the two proposed technological solutions: CPM (context-preserving magnification) and IT (instrument transparency). Following a demonstration of these software solutions, a survey utilizing a Likert scale was administered. The second…
Genes, proteins, chemicals, diseases, species, mutations and cell lines named across the full text — each resolved to its canonical identifier and authoritative record.
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8- —Innovation Funding Agency Business Finland
- —https://doi.org/10.13039/501100002341Research Council of Finland
- —University of Eastern Finland (including Kuopio University Hospital)
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSurgical Simulation and Training · Digital Imaging in Medicine · Facial Trauma and Fracture Management
Introduction
Visualization is one half of the sensory ergonomics along with tactile perception in microsurgery [28]. Visual ergonomics in microsurgery is affected by several factors such as the type of device used and procedure [2]. MIS Microsurgery is often conducted using surgical microscopes, endoscopes, and exoscopes and recently with surgical robots. The use of surgical microscopes, the most common image guidance device in neurosurgery, imposes a range of visual challenges that can affect surgical performance. The most common ones are narrow fields of view, reduction of depth perception, instrument obstruction, and limited visual clues. Alongside ergonomic issues from handling the microscope, limited contact with tissue environment and sensitivity of structures make interruptions in workflow more harmful [14]. Surgeons engaged in microsurgery must therefore exhibit exceptional eye-hand coordination, heightened dexterity, and increased cognitive workload to manage these challenges effectively [2, 25]. Some of these requirements can be approached by training fine motor skills and by built-in features such as zooming and focusing [5, 8].
Optical zooming in the field of view reduces contextual awareness which makes it difficult for surgeons to identify nearby structures. The presence of multiple instruments within the operating field further obstructs the view from the already limited visual field [15]. Current surgical microscopes require manual adjustments for focus, magnification, and positioning, which can interrupt the surgical flow and consume time during the surgery [12]. Moreover, there are other visual challenges during microsurgery that cannot be fully addressed by a microscope or by other devices alone. For instance, high magnification can affect depth perception (stereopsis) so that the distance between different tissues becomes distorted. Secondly, frequent switch between different zoom levels requires additional cognitive work from the surgeons to adjust to the new surgical view potentially leading to increased error rates [4, 6, 11]. These challenges emphasize the need for advanced visualization solutions that can enhance surgical precision and efficiency.
The current study aims to develop a visual solution that address challenges of visual ergonomics in microsurgeries. First, we identify the key visual ergonomic issues encountered by neurosurgeons using structured questionnaire and interview. Next, based on these insights we develop and evaluate the visual solutions that are aimed at addressing these challenges. The first feature of our solution combines images from two cameras to produce a single expanded view based on the projective transformation techniques. The second feature involves instrument transparency/subtraction from the operating field. Our solution is applicable for surgical microscopes and relies on the multi-camera setup that provides complementary views (Fig. 1). Our hypotheses are (1) visual ergonomics issues are perceived as relevant and frequently encountered during microsurgery and (2) proposed solutions particularly context preservation expansion can improve task performance and reduce workflow interruptions compared to standard microscope settings.Fig. 1. Conceptual comparison of the conventional visualization and the proposed solution in neurosurgery. a A neurosurgeon uses a microscopic-assisted endoscopic setup requiring frequent alternation between the microscopic eyepiece and the endoscopic display. b A neurosurgeon utilizing image processing-based visualization solution, where image from both microscope and endoscope is processed in real time into a single, non-overlapping, view on one display
Materials and methods
This study was conducted in two sequential parts: study part 1 and study part 2. The study part 1 was conducted first, in winter-spring 2024 and earlier in the development process with most participants being from Finland. This study focused on the assessment of visual ergonomics challenges and users’ interests and involved a survey with a video demonstration aimed at assessing the commonly encountered visual ergonomics challenges. In addition, we showed the in-development videos of our proposed features to neurosurgeons to receive their professional evaluation of the feasibility and usability of these features in improving the visual ergonomic challenges in real-world settings. In the study part 2, we focused on user feedback and performance and conducted a hands-on evaluation of proposed solutions using simulated surgical tasks in comparison’ with conventional microscope setup. This approach was used to capture neurosurgeons’ experience and direct the development of features. Study part 2 was carried out later in fall 2024 when the working prototype was available.
Study part 1: survey with video demonstration
The cohort of interest includes practicing neurosurgeons, neurosurgery residents, and medical students who utilize surgical microscopes for training and surgical practice.
Participant selection. The study subjects were recruited at the Microsurgery Center of Eastern Finland, Kuopio University Hospital’s site and online by Webropol, the University of Eastern Finland (UEF) online survey tool. Participants were selected based on their surgical specialty, experience working with surgical microscopes, and performing microsurgeries. Surgical residents of all years of study were also invited if they had some experience working with surgical microscopes. Surgeons of other specialties that do not involve microscopic surgery and medical students were excluded. Participants were recruited from January 2024 to May 2024.
Ethics statement**.** This study was conducted after approval by the UEF Committee on Research Ethics in accordance with the Finnish Medical Research Act (1999/488). All participants were instructed on personal data use and provided informed consent.
Survey design**.** The survey consisted of two parts: (a) A semi-structured interview and (b) a written questionnaire. Interview questions were designed in three sections: demographics, current challenges, and alternatives (proposed solutions) (Table 1). The demographic data included hospital units, position, and professional experience in years. The questions on current challenges aimed at identifying visual ergonomics difficulties; examples or open-ended explanations (guidance Table 1) were provided by the interviewer whenever possible. In the last section titled as alternatives, the use of other minimally invasive surgery (MIS) devices such as endoscopes was also assessed. Table 1. Semi-structured interview questions and guidance contentSectionsQuestionsDemographic dataHospital unit, position, and experience in yearsChallengesWhat kind of surgeries do you perform on a regular basis using surgical microscopes?^a^What are the most common challenges that you experience when using surgical microscopes?How often do you use built-in features of surgical microscopes like zooming in/out, focus adjustments, lightning, etc.?Do you think adjusting these features consumes time?Do you experience visual fatigue often?^b^Have you ever had trouble with depth perception during surgery? If so, can you describe what happened and how it affected your performance?AlternativesHave you ever needed to use endoscopic-assisted microscopy? How much time is needed to set it up?^c^Can you name any benefits for setups used in endoscopic-assisted microscopy?^a^ Type of surgery (vascular, skull base) and % of surgeries that require microscope use^b^ Related to device use^c^ If exoscope was used before, the participants were asked about its benefits
Study participants were interviewed face-to-face or online in the presence of one or two team members on their experience with surgical microscopes (measured in years), common type of surgeries they perform and challenges they encounter. As the interview was semi-structured, participants were free to share their opinions during the conversation's natural flow.
The second part of the survey, a questionnaire, consisted of open-ended and multiple-choice questions that helped to quantify the visual ergonomic difficulties followed by Likert scale questions on the proposed solutions that were played back for participants using two demonstration videos (Fig. 2, Table 2). The 5-point Likert scale questions were structured to evaluate participants’ perceived usability and possibility to use proposed solutions numerically. They also allow us to assess the neurosurgeons’ perceived usability and potential advantages of the two solutions in real surgeries. The questionnaire was divided into four parts based on the content, namely, Challenges, context-preserving magnification (CPM), instrument transparency (IT), and application.Fig. 2. Screenshots from Solution 1 (context-preserving magnification or CPM) and Solution 2 (instrument transparency or IT) videos. a Solution 1 encompasses two fields of vision on a single microscopic view. The central area shows a magnified area of interest, which represents the surgeons’ performing area while the surrounding area appears in the field of view at a lower magnification level. Which provides an additional visual information area. b IT video shows the side-to-side comparison of standard appearance of surgical instrument and semi-transparent version of itTable 2Questionnaire content. CPM: Context-preserving magnification. IT: Instrument transparencySectionsQuestions ChallengesQ1. What kind of challenges have you experienced during microscopic surgeries?Q2. Does changing the zoom level interrupt the workflow?Q3. How is the zoom control in the microscope you often use?Q4. How much time in seconds does it take to change the zoom level?Solution 1 (Context-preserving magnification, CPM) & Solution 2 (Instrument transparency, IT)Q1How much would you like to try the prototype of Solution 1/Solution2?Q2Did you find Solution 1 to be helpful for identifying more anatomical landmarks?How do you expect the effectiveness of Solution 2 on your visual ergonomics of the surgical scene?Q3Did Solution 1 cause any disorientations?Do you find Solution 2 presented any artifacts or visual noise?Q4Would you be comfortable using Solution 1 in a real surgical setting?Q5Do you think Solution 1 could help in saving surgery time in avoiding the frequent exchange between different zoom levels?Do you think that using solution 2 would improve your performance as a surgeon?Q6Do you think that using Solution 1 would improve your performance as a surgeon?ApplicationHow do you think these solutions are helpful in demonstrating the surgical field on the operating room (OR) screens for other OR personnel?How do you think these solutions are helpful in demonstrating the surgical field in the video management systems and surgical video recordings for educational purposes?In what ways do you think the proposed surgical software solutions could improve your surgical outcomes?Would you recommend the proposed surgical software solutions to other surgeons?
Demonstration videos**.** In the CPM and IT sections of the questionnaire, participants were shown short video demonstrations of the solutions (Fig. 2a and b). Solution 1 demonstration included a 28-s-long video clip composed of two geometrically aligned videos of a brain tissue (standard microscopic view and CPM applied to microscopic view). The aim of CPM was to combine the video outputs from a microscope and a secondary camera into one output. The area of interest was magnified to a greater extent compared to the surrounding area, thus allowing the operating surgeons to avoid changing zoom levels to explore surrounding areas (Fig. 2a). Instrument transparency demonstration included a 50-s-long video of two standard microscopic views of brain tissue in which a pair of forceps were placed over the tooth tissue (Fig. 2b). The tissue forceps were moved over the brain tissue twice and locked twice. The left microscopic view shows an application of IT which makes the body of the instrument, but not its tips and outline edges, transparent. This is in contrast with the standard microscope view of the same scene on the right.
Study part 2: the prototype and hands-on experiment
Prototype setup**.** The hardware system consists of two USB 3.0 cameras (Ximea, Germany) of resolution 3.1 Mpix, the first camera equipped with a zoom lens (focal length range:18—108 mm) acts as a microscope. The second camera equipped with a fixed focal length lens (12.5 mm) is used by a proprietary algorithm to expand the field of view (Fig. 3). The cameras are connected to USB 3 ports of a laptop PC. The proprietary algorithm runs on the Microsoft Visual studio IDE, utilizing CUDA computing in Windows 11. Essential hardware specifications of the laptop are; CPU: Intel Core (TM) i7-11850H, RAM: 32 GB, and GPU: Nvidia RTX 3070 mobile. The algorithm utilizes both CPU and GPU processing of the laptop. Details of the algorithm and its implementation are proprietary to the project team. In This study focuses on the usability and feasibility testing from the perspective of neurosurgeons as the primary users. Especially, to enable visualization of the expanded view on a screen, while maintaining size of the microscope view large enough. A mathematical algorithm to deform the image of the secondary camera was developed. The frame rate and time delay of the video processing were tested by the team members, and by surgeons, who have experience in using digital microscopes and endoscopes in surgeries, to get subjective feedback.Fig. 3. Screenshots from the prototype test task. a Single camera output video where a zoomed section of the temporal cortex is visible. A metal pin is placed in the center of the field and a ring is placed on the outer corner of field. Zooming out and focusing are needed to identify the ring location at the beginning of the task. b Two camera output videos where a whole temporal cortes of brain is visible
Hands-on prototype testing**.** The test was conducted in a training environment at the World Federation of Neurosurgical Societies (WFNS) educational day at the 14th Asian Congress of Neurological Surgeons, Cairo, between 18 to 20th of December 2024. Twelve neurosurgeons were asked to perform a task on a brain silicone model (TemporalBox, UpSurgeOn Srl, Italy). The task consisted of placing a ring on top of a pin using toothed forceps and participants were free to use either one or both hands, according to their preference (Fig. 3). First, participants were asked to perform the task looking at a display that demonstrated output from a single camera, mimicking conventional microscopic view (Fig. 3a). Instructions on how to adjust camera setups including the zoom and focus controls were explained beforehand. Next, participants were asked to perform the same task using our CPM (Fig. 3b). Time taken to perform the task, workflow interruptions, and task outcome which is placing a ring on top of a pin were measured in both settings. Workflow interruptions were defined as conditions when a participant removes a hand from the surgical field to adjust camera setup. If hands were removed for other reasons than microscope related such as handling the instrument it was not considered as workflow interruption. It is important to reiterate that participants were different between study part 1 (questionnaires and demos) and study part 2 (task conductance).
Data analysis**.** The primary research methods of this study were interviews and questionnaires. The demographic data was demonstrated as percentages and means. Qualitative data was grouped based on the frequency of similar responses. Multiple-choice questions and Likert scale questions were analyzed using SPSS. Time taken to complete the tasks was compared between two settings using the paired t-test and workflow interruptions (ordinal data) was compared using Wilcoxon matched-pairs test due to small sample size. The Cronbach’s alpha method was used to assess multiple choice questions (MCQ) reliability for sections of Solution 1 (context-preserving magnification) and Solution 2 (instrument transparency). All figures were drawn on GraphPad Prism 10 software.
Results
Demographics**.** Sixteen surgeons from four countries (Finland, Scotland, Germany, and UK) were recruited between the period of 29th of January to 17th of May 2024. Most surgeons (94%) were practicing neurosurgeons or completing residence in neurosurgery working at Neurosurgery unit and only one ENT (ear-nose-throat) surgeon was recruited (Table 3). The average length of experience in microsurgery operations was 15.6 years. The variety of surgeries ranged from spinal surgeries to skull base surgeries. The ratio for use of microscope in operated surgeries varied from 60 to 100%, although the estimation was subjective and best on the participants’ deduction. Table 3. Participant demographics (N = 16)VariableNumber%Hospital unitNeurosurgery1594ENT surgery16PositionHead of unit212.5Chief surgeon212.5Assistant850Resident425Experience in neurosurgery < 5 years4255–10 years531.2510–20 years318.7520 + years524
Visual ergonomics challenges**.** In response to the interview questions, the most common type of visual challenge in microsurgery was the visual obstruction by surgical instruments followed by blurring of some structures due to limited depth of field (Fig. 4a). Loss of context with high magnification constituted the least common challenge as reported by 7/16 surgeons. Other visual challenges reported in a free-form comment were loss of orientation in the operating field, insufficient lighting, narrow corridor of view, lack of 3D vision, depth identification and loss of instrument shaft from the view.Fig. 4. Responses to Q1 and Q2 of the section on challenges from the questionnaire of the semi-structured interviews (N = 16). a Q1 answer options a. Loss of context with high magnification, b. Visual obstruction by surgical instruments, c. Blurring of some structures due to limited depth of field, d. Blurring of surgical instruments and light reflection, e. Loss of certain areas from visual field (surgical blind spots) (b) Workflow interruption during microscope use
All participants stated that they use the zoom, focus, and light adjustment features of surgical microscopes all the time. 25% of participants reported that adjusting these features consumes a considerable amount of time from surgery (Fig. 4b). Another 25% agreed that to some extent it takes time from surgery, while half of the participants answered negatively.
As each microscope has different control settings, participants were asked on the method of zoom control and how long it takes to adjust it in their practice (Fig. 5). Hand control was the most common (75%) setup for adjusting the zoom level followed by mouthpiece (50%), foot control (20%) and voice control (6.25%) (Fig. 5A). It takes about 3–5 s to adjust zoom level for the majority of surgeons while 25% of respondents spend 5–15 s for zoom adjustment (Fig. 5B). Interestingly, none of the participants takes less than 3 s or more than 15 s for zoom accommodation.Fig. 5. Microscope control method by surgeons (a) and time taken to adjust zoom level in seconds (b) from semi-structured interviews (N = 16)
Visual fatigue was common for half of the participants, but only five of them mentioned its association with ergonomic factors in their practice while the rest experienced it due to pre-existing eye diseases such as myopia and hyperopia (Fig. 6a). Depth perception in high magnification surgeries was commonly reported by nine surgeons while the rest of respondents answered that they do not encounter such problem (Fig. 6b). Among free-form answers, it was noted that some of the surgeons adapt to uncomfortable body positions for better visualization. One of the respondents admitted that depth perception problem persists for 2–3 min after changing positions from the main view to the assistant view.Fig. 6. Visual fatigue (a) and depth perception (stereopsis) difficulty (b) during high magnification microsurgery from semi-structured interviews (N = 16)
Alternatives. In the next set of questions, we assessed participants’ views on the use of alternative devices such as the endoscope. For that reason, participants were asked about their experience of using endoscopes and benefits of endoscope-assisted microscopic neurosurgery in a free form. Only four participants had ever used endoscope-assisted microscopic setup, and their experience was limited to one-two procedures per year. Two surgeons had received training on surgical models with endoscope but had not used it during real operations. The main advantage of endoscope assisted microsurgery was the better visualization of narrow and deep fields of view compared to microscope used alone; however, it was also noted that the endoscope size may not be suitable for skull base surgeries. In addition, four surgeons had used exoscope and two of them stated the advantages of exoscope over microscope in terms of manoeuvring flexibility and biopsy sampling. Two other surgeons did not consider the exoscope superior and instead noted that its use requires adaptation.
Proposed Solutions**.** 69% of respondents agreed that they would like to try the prototypes of both solutions (Fig. 7). More than half of the respondents stated that they would like to try Solution 1 (context preserving magnification) in a real surgical setting while the percentage was 81% for Solution 2 (instrument transparency). 81% of surgeons positively evaluated assistance of deep learning-based context-preserving magnification to be helpful for identifying more anatomical landmarks (Fig. 7a, Q2). The potential of Solution 1 (context preservation) to decrease the operating time and improve surgeon’s performance was supported by over 35% of participants. However, some surgeons (44%) stated that seeing two different levels of zoom can cause disorientation and can take time to adapt. Over 60% of respondents agreed that instrument transparency in Solution 2 can be effective in improving visual ergonomics during high magnification surgeries (Fig. 7b). Although one fifth of surgeons stated that Solution 2 can cause visual artifacts, more than half agreed that it can improve their surgical performance. The internal consistency of the questionnaire was high for CPM (α = 0.92) and lower for IT (α = 0.48).Fig. 7. Results of Likert-scale evaluations for Solution 1 (a, context-preserving magnification or CPM) and Solution 2 (b, instrument transparency or IT), N = 16. Questions are provided on Table 2. Scores from 1 to 5 given based on the level of agreement: strongly agree – score 5, strongly disagree – 1. Scoring is reversed for Q3 (‘Did Solution 1 cause any disorientations?’ and ‘Do you find Solution 2 presented any artifacts or visual noise?’) for both solutions as the question addresses disadvantages, e.g. strongly agree – 1, strongly disagree – 5. Each score’s proportion is given in different shades of grey. The lighter the shade the higher the agreement level. Some of the respondents did not answer to MCQ of Solution 2, therefore proportion for Q2-Q5 in part b is below 1.0
Application potentials**.** In the last section of the questionnaire presented in Table 2, we assessed the potential application of the proposed solutions in real surgeries in the OR and educational settings. Surgeons (n = 5) noted that the context preserving magnification solution with two fields of view can be helpful in improving the comprehension of anatomy and use of microsurgery instruments. The instrument transparency feature was evaluated as a practical solution to overcome the constant blockage caused by instruments in use within the surgical field. Participants indicated the instrument transparency solution would provide a better teaching experience for other personnel involved in the operation and even medical students or residents shadowing or observing the surgery. Surgeons also proposed that IT could be help prevent or reduce or the risk of injuries to vital physiological structures caused by compression. Regarding IT, bleeding control was seen as main usage. Although, some of the surgeons agreed that proposed solutions are advantageous only to the surgeons, 11 participants stated that they will recommend it to other surgeons.
Prototype testing**.** 58% of participants spent 40% less time completing the task using Solution 1 (context preserving magnification) compared to standard setting (Fig. 8A). The standard setting implies the same setup but only one video output from a single camera. However, the overall results indicate that there is no significant difference in the time taken to complete the task between Solution 1 and standard setting use (p > 0.05). On the other hand, there was significant reduction in workflow interruptions (p = 0.02). Notably, CPM had an advantage of eliminating microscope adjustments for this task which prevented interruption of workflow. Additionally, for one participant, there was no difference in task completion time, and four participants spent more time longer to complete the task using the CPM. Participants who found CPM beneficial in terms of saving time also noted that CPM is useful when a larger object is out of view and would require significant zoom out even though the view can be defocused. All participants were able to accomplish the task goal of placing the ring.Fig. 8. Time (a) and workflow interruption measurements (b) during prototype testing, N = 12
Discussion
The study aimed to identify visual ergonomics challenges during microsurgery and assess the need and advantage of novel software solution compared to current microscopic features. Interview results showed that the most common type of visual challenge during high magnification surgery is obstruction by surgical instruments (94%) followed by blurring of some structures due to limited depth of field (81%). Other issues encountered include disparity in depth perception and narrow field of view. Most common microscope related reported issues were workflow interruption by adjusting zoom and focus (50%) and depth misperception (56%). Other difficulties reported in the free form were of ergonomical origin such as uncomfortable body position and muscular pain. While novel solutions tested by neurosurgeons have shown to decrease the task completion rate compared to the conventional microscope, they did not result in a significant reduction in workflow interruptions.
Visual ergonomics is a fundamental concern across various fields of surgery including ophthalmology, otolaryngology and reconstructive surgery [26]. In neurosurgery, surgeons often face prolonged positions commonly during high precision anastomosis [30]. These ergonomic challenges are closely related to visual challenges of working with microscopes and are often compromised to having a better view of surgical area [14]. Addressing visual ergonomics can significantly mitigate musculoskeletal disorders associated with neurosurgical practice improving surgeons’s performance and patient output [16, 27]. Our study reinforces these findings and quantifies the prevalence of visual ergonomics challenges in neurosurgery.
Recently adjustments to microscopes have been improved by technical advancements, such as mouthpieces and voice control. However, these innovations are not widely used due to usability issues [12]. Depth perception (stereopsis) is an important skill in microsurgery and essential for training in image guided surgery. Impaired depth perception is associated with decrease in surgical performance and learning curve [7]. Although stereopsis can differ significantly between individuals, microsurgery skills can be enhanced by technology assisted learning [10]. Exoscopes and endoscopes were developed to address 3D environment visualization and specialized hand–eye coordination with both having own advantages and disadvantages. Despite recent advancement endoscopic modalities still suffer from lower imaging resolution while exoscopes still remain inferior in depth perception to standard microscopes [3, 22]. Here we showed that depth discrimination is still a common problem in microscopic surgery which is not addressed widely.
Both exoscope and endoscope in a combination with microscope act as secondary cameras to provide view from different perspective. Endoscope assisted microsurgery (EAMS) is a promising method in certain pediatric procedures, but the issue of operating field obstruction by instruments and visual workload increases [9, 20]. In addition, visual output of these instruments is displayed on a separate screen which requires surgeons to alternate between microscope’s ocular and screen. Here, we developed an algorithm that combines visual outputs from microscope and a secondary camera into one. The idea of context-preserving magnification and instrument transparency have been largely supported by surgeons (Fig. 7). Our solution addresses the issues of instrument obstruction, loss of context and depth perception to some extent. Furthermore, the need of introducing a new display is eliminated and output is observed in real-time.
Current developments aimed at addressing visual ergonomics challenges across various microsurgical specialties are presented in Table 4. Many of these technologies are in the early stage of development and highlight the relevance of proposed solutions with different approaches. Below we describe key features of current developments are compare them with our own solutions. Table 4. Comparison of current developments on visual obstruction during microsurgeryCPM & ITCARET by Okubo et al. [21, 24]NOTES by Lerotic et al. [17]Visual compensation system by Koreeda et al. [13]InstrumentsMicroscope and 2nd cameraLaparoscopeEndoscopeEndoscopeNumber of cameras22–31–2?2Real-time processingYesYesNoYesProcessing time20 ms111 msNA62.4 msInstrument transparencyYesNoNoYesContext-preserving magnificationYesNoYesNo
Miranda-Luna et al. proposed construction of the panoramic view of bladder neoplasm endoscopic images, but it is applicable only for static images. It requires image analysis after the procedure and cannot affect the decision-making during the intervention [19]. Lerotic et al. presented a method that utilizes parallax correction scheme that extends the visual field during Natural Orifice Transluminal Endoscopic Surgery (NOTES). It gives advantage in viewing peripheral tissue without disturbing the focal view, but it has the same limitation as the previous study, only static images can be obtained [17]. The real-time field expansion is challenging in microsurgeries as usually only one camera is used e.g. endoscope or microscope. The introduction of the second camera allows to obtain video from another angle. This method was utilized by Okubo et al. [21, 24] who developed camera-retracted trocar (CARET). CARET is a conventional trocar with a built-in tiny camera at the tip that can be retracted during insertion and expanded inside the abdominal cavity. Simultaneous usage of two CARETs during laparoscopic surgery produced panoramic image by mosaicking and tracking methods. Benefits of this method are that it can produce images even in the case of 0% overlap and processing speed is 9fps faster than the conventional mosaicking method. However, tracking accuracy is still low. In 2021, the method was further improved using Oriented Robust Binary Simultaneous Localization and Mapping (ORB-SLAM). The new approach has higher accuracy and speed (21 frames per second (fps)) as it does not depend on image overlap [1].
Instrument transparency, which is one of our solution features, has been developed using the endoscope by Koreeda et al. They utilized image transformation and segmentation to combine video outputs from two endoscopes into one [13]. Like our solution, the tip and outline of the instrument remain visible while the shaft is semitransparent. However, the experimental setup consisted of fixed endoscopes whereas in real surgical setting this is seldom the case.
Overall, these emerging solutions indicate a need for addressing visual ergonomics in microsurgery and the early approach methods. Our work aims to contribute to the issue in the neurosurgery field and overcome limitations encountered by other developments focusing on feedback from neurosurgeons.
The current limitation of our solution remains the placement of the second camera adjacent to the microscope. Although the camera can be in a fixed position relative to microscope, the narrow corridor of the operating field can block the view of the secondary camera. This limitation is under evaluation by employing EAMS setup where the endoscope acts as a secondary camera. Secondly, the current prototype setup has limited processing time which is around 20 fps whereas the conventional surgical microscope video rate is 60 fps [29]. Low video processing time affected the performance of participants by interfering with hand and eye coordination during the task. As noted from the participant feedback, speed lag required time for performers to get acquainted with lower fps compared to their conventional setting. New tests with more powerful hardware, particularly workstation PC can eliminate frame rate decrease and consequently speed lag. Finally, we had a limited number of participants for both interviews (N = 16) and prototype testing (N = 12) which limits the statistical power of correlations. A larger study sample and prior training are needed to evaluate the impact of a solution.
With increased attention to visual ergonomics in surgery the importance of assessment tools is in demand. Compared to postural ergonomics, which has been extensively studied and incorporated into training, visual ergonomics assessment remains limited. In this study we used a questionnaire to explore the neurosurgeons’ perceptions, but future research should include quantitative and objective evaluation tools such as eye tracking, gaze analysis and head and neck movements[16, 18, 23]. In addition, assessment of new developments should include some evaluation tools and simulation of real surgical scenes e.g. narrow passages.
Conclusion
We quantified and surveyed types of visual challenges among neurosurgeons and proposed software-based solutions. The solutions, namely context-preserving magnification (CPM) and instrument transparency (IT) showed potential of mitigating these two visual ergonomics challenges.
The reference list from the paper itself. Each links out to its DOI / PubMed record.
- 1Koskinen J (2022) Computational Evaluation of Microsurgical Skills. University of Eastern Finland, Joensuu. http://urn.fi/URN:ISBN:978-952-61-4743-7
- 2Lerotic M, Chung AJ, Clark J, Valibeik S, Yang GZ (2008) Dynamic view expansion for enhanced navigation in natural orifice transluminal endoscopic surgery. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5242 LNCS, no. PART 2, pp. 467–475, 10.1007/978-3-540-85990-1_5610.1007/978-3-540-85990-1_5618982638 · doi ↗ · pubmed ↗
- 3Takada C, Suzuki T, Afifi A, Nakaguchi T (2017) Hybrid tracking and matching algorithm for mosaicking multiple surgical views. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10170 LNCS, pp. 24–35, 10.1007/978-3-319-54057-3_3