Problem being addressed
Automatic surgical gesture recognition is fundamental for improving intelligence in robot-assisted surgery, such as conducting complicated tasks of surgery surveillance and skill evaluation; however, current methods treat each frame individually and produce the outcomes without effective consideration on future information.
A framework based on reinforcement learning and tree search for joint surgical gesture segmentation and classification. An agent is trained to segment and classify the surgical video in a human-like manner whose direct decisions are re-considered by tree search appropriately.
Advantages of this solution
With the integration of complementary information from distinct models, the suggested framework is able to achieve better performance than baseline methods using either of the neural networks. For an overall evaluation, the developed approach consistently outperforms the existing methods on the suturing task in terms of accuracy.
Solution originally applied in these industries
Possible New Application of the Work
Automatically recognizing the robotic gestures in surgical process plays an important role for surgery surveillance, automatic skill assessment, and surgery training. Identifying which action is being operated is crucial for the medical students to better understand the surgical procedure ad learn the sequence of actions.
Gesture recognition is widely used in gaming industry. In vision-based interfaces for video games, gestures are used as commands for the games instead of pressing buttons on a keyboard or moving a mouse. In these interfaces, unintentional movements and continuous gestures must be supported to provide the user with a more natural interface.
Source URL: #############