Alexander Jaus

Alexander Jaus

Postdoctoral Researcher in Computer Vision

ISBI 2025

LIMIS: Towards Language-based Interactive Medical Image Segmentation (Oral)

L. Heinemann*, A. Jaus*, Z. Marinov, M. Kim, M. F. Spadea, J. Kleesiek, R. Stiefelhagen

denotes shared first authorship

International Symposium on Biomedical Imaging (ISBI), 2025

Paper
Interactive Segmentation Foundation Models Medical Imaging

Key contributions

  • The first purely language-based interactive medical image segmentation model, adapting Grounded SAM to the medical domain.
  • A language-driven interaction strategy that lets clinicians refine segmentation masks using only natural-language commands.
  • Expert-validated quality and usability across three public medical datasets, confirming hands-free interactive segmentation.
LIMIS produces an initial mask from medical foundation models, then lets radiologists refine it through language alone.
LIMIS produces an initial mask from medical foundation models, then lets radiologists refine it through language alone.

How it works

Within this work, we introduce LIMIS: The first purely language-based interactive medical image segmentation model. We achieve this by adapting Grounded SAM to the medical domain and designing a language-based model interaction strategy that allows radiologists to incorporate their knowledge into the segmentation process. LIMIS produces high-quality initial segmentation masks by leveraging medical foundation models and allows users to adapt segmentation masks using only language, opening up interactive segmentation to scenarios where physicians require using their hands for other tasks. We evaluate LIMIS on three publicly available medical datasets in terms of performance and usability with experts from the medical domain confirming its high-quality segmentation masks and its interactive usability

Citation

Heinemann, L., Jaus, A., Marinov, Z., Kim, M., Spadea, M. F., Kleesiek, J., & Stiefelhagen, R. (2025, April). LIMIS: Towards Language-Based Interactive Medical Image Segmentation. In 2025 IEEE 22nd International Symposium on Biomedical Imaging (ISBI) (pp. 1-5). IEEE.