Smooth operator

Smooth operator

Problem being addressed

Automatic surgical gesture recognition is fundamental for improving intelligence in robot-assisted surgery​,​ such as conducting complicated tasks of surgery surveillance and skill evaluation; however​,​ current methods treat each frame individually and produce the outcomes without effective consideration on future information.


A framework based on reinforcement learning and tree search for joint surgical gesture segmentation and classification. An agent is trained to segment and classify the surgical video in a human-like manner whose direct decisions are re-considered by tree search appropriately.

Advantages of this solution

With the integration of complementary information from distinct models, the suggested framework is able to achieve better performance than baseline methods using either of the neural networks. For an overall evaluation, the developed approach consistently outperforms the existing methods on the suturing task in terms of accuracy.

Solution originally applied in these industries


Healthcare Sector

Possible New Application of the Work


Education Sector

Automatically recognizing the robotic gestures in surgical process plays an important role for surgery surveillance, automatic skill assessment, and surgery training. Identifying which action is being operated is crucial for the medical students to better understand the surgical procedure ad learn the sequence of actions.


Entertainment Industry

Gesture recognition is widely used in gaming industry. In vision-based interfaces for video games, gestures are used as commands for the games instead of pressing buttons on a keyboard or moving a mouse. In these interfaces, unintentional movements and continuous gestures must be supported to provide the user with a more natural interface.

Author of original research described in this blitzcard: Xiaojie Gao, Yueming Jin, Qi Dou, Pheng-Ann Heng


Name of the author who conducted the original research that this blitzcard is based on.

Source URL: #############check-icon

search-iconBrowse all blitzcards