Alexander Jaus

Alexander Jaus

Postdoctoral Researcher in Computer Vision

arXiv 2023

Towards unifying anatomy segmentation: automated generation of a full-body ct dataset via knowledge aggregation and anatomical guidelines

A. Jaus, C. Seibold, K. Hermann, A. Walter, K. Giske, J. Haubold, J. Kleesiek, R. Stiefelhagen

arXiv, 2023

Explore Paper
Medical Imaging Segmentation Datasets

Key contributions

  • An expert-free pseudo-label aggregation and refinement pipeline combining multiple public anatomy datasets via nnU-Net ensembles.
  • The Dense Anatomical Prediction (DAP) Atlas: 533 whole-body CTs with 142 voxel-level anatomical labels — the first dense full-body CT annotation release.
  • Anatomical plausibility post-processing using textbook anatomy rules, achieving 85% Dice on BTCV without fine-tuning.
  • A trained unified anatomy segmentation model predicting all 142 structures in a single pass, enabling body composition, surgery planning, and treatment monitoring applications.
A sample of the DAP Atlas: a coronal labeled slice next to two 3D renderings — without and with surrounding tissue.
A sample of the DAP Atlas: a coronal labeled slice next to two 3D renderings — without and with surrounding tissue.

How it works

We address the limitation of current medical image segmentation models that focus only on isolated body regions by developing a holistic full-body anatomical dataset. Recognizing the impracticality of large-scale expert annotations for hundreds of structures, we employ a fully automated pseudo-label aggregation and refinement strategy to generate high-quality labels without direct expert involvement.

Inspired by approaches from natural image segmentation, we combine scattered anatomical knowledge from multiple sources and ensure anatomical plausibility through post-processing techniques guided by textbook anatomy. We aggregate fragmented knowledge from public datasets and refine labels using anatomical rules, which allows us to automate dataset creation and develop a unified segmentation model.

Our resulting dataset comprises 533 whole-body CT scans annotated with 142 anatomical structures, marking the first release of dense, voxel-wise full-body CT annotations. This dataset enables training of models for diverse clinical applications, such as body composition analysis, surgery planning, and cancer treatment monitoring, without the bottleneck of manual labeling.

The approach is validated through expert reviews, anatomical plausibility checks, and out-of-distribution performance (85% Dice on BTCV without fine-tuning). The dataset and models are publicly available to foster research in holistic medical image segmentation.

Explore it yourself

DAP-Atlas sample: a labeled coronal CT slice next to two 3D renderings — bones and organs, and the full body
The DAP-Atlas: a labeled coronal slice alongside two 3D renderings — bones & organs, and the full body. Scroll through the axial CT below and hover to identify each structure.

Scroll here to load the interactive viewer…

– / – Hover the image

Citation

Jaus, Alexander; Seibold, Constantin; Hermann, Kelsey; Walter, Alexandra; Giske, Kristina; Haubold, Johannes; Kleesiek, Jens; Stiefelhagen, Rainer; . (2023). "Towards unifying anatomy segmentation: automated generation of a full-body ct dataset via knowledge aggregation and anatomical guidelines." arXiv preprint arXiv:2307.13375..