Alexander Jaus

Alexander Jaus

Postdoctoral Researcher in Computer Vision

SMC 2026

IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction

H. Zhang, D. Wen, K. Peng, D. Schneider, Z. Zhong, A. Jaus, Z. Marinov, J. Wei, R. Liu, J. Zheng, Y. Chen, Y. Zhang, Y. Luo, L. Qi, R. Stiefelhagen

IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2026

Paper
Action Understanding

Key contributions

  • A mixed-initiative framework that constructs onset-anchored HOI event graphs for egocentric procedural video.
  • A trust-calibrated controller choosing among direct queries, confirmed suggestions, and conservative completions, with risk-bounded atomic rollback protecting confirmed decisions.
  • A user study (9 participants): 13.5% fewer manual annotation actions and zero confirmed-field violations.
IMPACT-HOI incrementally builds onset-anchored HOI event graphs under a trust-calibrated, risk-bounded controller.
IMPACT-HOI incrementally builds onset-anchored HOI event graphs under a trust-calibrated, risk-bounded controller.

How it works

We present IMPACT-HOI, a mixed-initiative framework for annotating egocentric procedural video by constructing structured event graphs for Human-Object Interactions (HOI), motivated by the need for high-quality structured supervision for learning robot manipulation from human demonstration. IMPACT-HOI frames this task as the incremental resolution of a partially specified, onset-anchored event state.

A trust-calibrated controller selects among direct queries, human-confirmed suggestions, and conservative completions based on empirical annotator behavior and evidence quality. A risk-bounded execution protocol, utilizing atomic rollback, ensures that human-confirmed decisions are preserved against conflicting automated updates. A user study with 9 participants shows a 13.5% reduction in manual annotation actions, a 46.67% event match rate, and zero confirmed-field violations under the studied protocol.

Citation

Zhang, H., Wen, D., Peng, K., Schneider, D., Zhong, Z., Jaus, A., Marinov, Z., Wei, J., Liu, R., Zheng, J., Chen, Y., Zhang, Y., Luo, Y., Qi, L., & Stiefelhagen, R. (2026). IMPACT-HOI: Supervisory Control for Onset-Anchored Partial HOI Event Construction. IEEE International Conference on Systems, Man, and Cybernetics (SMC).