MultiBypass Surgical Action Triplet Challenge 2026

Surgical workflow analysis has evolved in the types of intraoperative signals analyzed, with the goal of providing context-aware decision support to improve patient outcomes, identify surgical events, and support preoperative planning and postoperative analysis. Recent work has expanded analysis from coarse-grained levels (phases, steps) to fine-grained levels (action triplets). Action triplets describe a surgical scene using a triplet of <instrument, verb, target>.

However, with the advent of foundation models and the increasing need for model generalization across multiple centers, current surgical action triplet datasets face limitations in generalization assessment. CholecT50, while widely used, is monocentric with lower interaction density compared to multi-bypass procedures, which involve more instruments and actions, contributing to greater density and variability in instrument-anatomy interactions.

To address these gaps, we introduce MultiBypass-4C-T40, a large multi-centric dataset for complex Roux-en-Y gastric bypass surgery labeled with surgical action triplets. The dataset comprises 40 videos across 4 centers, with two centers providing training, validation, and future public test data, and two additional centers exclusively for testing. Our Future Public Test Set is approximately 5x larger than existing datasets, with a substantial Hidden Test Set distributed across three centers for robust generalization evaluation.

The challenge Task

Develop deep learning methods for online surgical action triplet recognition (instrument, verb, target) from Roux-en-Y gastric bypass surgery videos. Models must process frames causally — no access to future frames. This challenge establishes a new research direction focused on generalization across multiple centers, organized in conjunction with EndoVis.

Task Description

The task is online surgical action triplet recognition, where the model provides triplet predictions for a current surgical frame and optionally uses context strictly from previous frames. No access to future frames is allowed for evaluation. Participants will develop algorithms to recognize action triplets directly from the provided surgical videos.

• Instrument Recognition: Identifying surgical instruments present in the scene
• Verb Recognition: Classifying the action being performed
• Target Recognition: Identifying the anatomical structure being acted upon
• Triplet Association: Correctly associating these components into valid triplets (85 triplet classes)

Learn more on the Challenge website

---

This event is organized by University of Strasbourg, CNRS, INSERM, ICube & IHU Strasbourg. It will take place within the framework of the MICCAI 2026 conference.