Alexander Jaus

Alexander Jaus

Postdoctoral Researcher in Computer Vision

SMC 2026

IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning

Q. Yin, D. Wen, K. Peng, D. Schneider, Z. Zhong, A. Jaus, Z. Marinov, J. Wei, R. Liu, J. Zheng, Y. Chen, C. Zhang, L. Qi, R. Stiefelhagen

IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2026

Paper
Action Understanding Interactive Segmentation

Key contributions

  • A correction-driven interactive framework for dense temporal action segmentation that reuses every human edit to improve future collaboration.
  • Uncertainty-aware boundary scribbles combined with local proposal modeling, cost-aware query planning, and structured propagation.
  • A human study showing improved labeling quality per unit of effort and better boundary accuracy over time.
IMPACT-Scribe's closed loop: uncertainty-aware boundary scribbles, cost-aware query planning, and correction-driven adaptation.
IMPACT-Scribe's closed loop: uncertainty-aware boundary scribbles, cost-aware query planning, and correction-driven adaptation.

How it works

Dense temporal annotation of procedural activity videos is vital for action understanding and embodied intelligence but remains labor-intensive due to reactive tools. Each correction is treated as an isolated edit, limiting reuse of information on annotator uncertainty and model reliability. We introduce IMPACT-Scribe, a correction-driven framework for dense labeling that uses each correction to improve future human–machine collaboration.

IMPACT-Scribe combines uncertainty-aware boundary scribble supervision, local proposal modeling, cost-aware query planning, structured propagation, and correction-driven adaptation. Experiments and a human study show that this closed-loop design improves labeling quality per effort, enhances boundary accuracy, and fosters better human-machine interaction over time.

Citation

Yin, Q., Wen, D., Peng, K., Schneider, D., Zhong, Z., Jaus, A., Marinov, Z., Wei, J., Liu, R., Zheng, J., Chen, Y., Zhang, C., Qi, L., & Stiefelhagen, R. (2026). IMPACT-Scribe: Interactive Temporal Action Segmentation with Boundary Scribbles and Query Planning. IEEE International Conference on Systems, Man, and Cybernetics (SMC).