Towards unifying anatomy segmentation: automated generation of a full-body ct dataset via knowledge aggregation and anatomical guidelines

We address the limitation of current medical image segmentation models that focus only on isolated body regions by developing a holistic full-body anatomical dataset. Recognizing the impracticality of large-scale expert annotations for hundreds of structures, we employ a fully automated pseudo-label aggregation and refinement strategy to generate high-quality labels without direct expert involvement.

Inspired by approaches from natural image segmentation, we combine scattered anatomical knowledge from multiple sources and ensure anatomical plausibility through post-processing techniques guided by textbook anatomy. We aggregate fragmented knowledge from public datasets and refine labels using anatomical rules, which allows us to automate dataset creation and develop a unified segmentation model.

Our resulting dataset comprises 533 whole-body CT scans annotated with 142 anatomical structures, marking the first release of dense, voxel-wise full-body CT annotations. This dataset enables training of models for diverse clinical applications, such as body composition analysis, surgery planning, and cancer treatment monitoring, without the bottleneck of manual labeling.

The approach is validated through expert reviews, anatomical plausibility checks, and out-of-distribution performance (85% Dice on BTCV without fine-tuning). The dataset and models are publicly available to foster research in holistic medical image segmentation.