Publications | Benjamin Billot

2025

Journal
A probabilistic histological atlas of the human brain for MRI segmentation

Adrià Casamitjana, Matteo Mancini, Eleanor Robinson, and 21 more authors

Nature, 2025

Abs Bib HTML

In human neuroimaging, brain atlases are essential for segmenting regions of interest (ROIs) and comparing subjects in a common coordinate frame. State-of-the-art atlases derived from histology provide exquisite three-dimensional cytoarchitectural maps but lack probabilistic labels throughout the whole brain: that is, the likelihood of each location belonging to a given ROI. Here we present NextBrain, a probabilistic histological atlas of the whole human brain. We developed artificial intelligence-enabled methods to align roughly 10,000 histological sections from five whole brain hemispheres into three-dimensional volumes and to produce delineations for 333 ROIs on these sections. We also created a companion Bayesian tool for automatic segmentation of these ROIs in magnetic resonance imaging (MRI) scans. We showcase two applications of the atlas: segmentation of ultra-high-resolution ex vivo MRI and volumetric analysis of Alzheimer’s disease using in vivo MRI. We publicly release raw and aligned data, an online visualization tool, the atlas, the segmentation tool, and ground truth delineations for a high-resolution ex vivo hemisphere used in validation. By enabling researchers worldwide to automatically analyse brain MRIs at a higher level of granularity, NextBrain holds promise to increase the specificity of findings and accelerate our quest to understand the human brain in health and disease.
@article{casamitjana_probabilistic_2025, title = {A probabilistic histological atlas of the human brain for {MRI} segmentation}, author = {Casamitjana, Adrià and Mancini, Matteo and Robinson, Eleanor and Peter, Loïc and Annunziata, Roberto and Althonayan, Juri and Crampsie, Shauna and Blackburn, Emily and Billot, Benjamin and Atzeni, Alessia and Puonti, Oula and Balbastre, Yaël and Schmidt, Peter and Hughes, James and Augustinack, Jean and Edlow, Brian and Zöllei, Lilla and Thomas, David and Kliemann, Dorit and Bocchetta, Martina and Strand, Catherine and Holton, Janice and Jaunmuktane, Zane and Iglesias, Juan Eugenio}, journal = {Nature}, year = {2025}, }
Journal
Benchmarking of deep learning methods for generic MRI multi-organ abdominal segmentation

Deepa Krishnaswamy, Cosmin Ciausu, Steve Pieper, and 3 more authors

Journal of Medical Imaging, 2025

Abs Bib PDF

Recent advances in deep learning have led to robust automated tools for segmentation of abdominal computed tomography (CT). Meanwhile, segmentation of magnetic resonance imaging (MRI) is substantially more challenging due to the inherent signal variability and the increased effort required for annotating training datasets. Hence, existing approaches are trained on limited sets of MRI sequences, which might limit their generalizability. To characterize the landscape of MRI abdominal segmentation tools, we present here a comprehensive benchmarking of the three state-of-the-art and open-source models: MRSegmentator, MRISegmentator-Abdomen, and TotalSegmentator MRI. Since these models are trained using labor-intensive manual annotation cycles, we also introduce and evaluate ABDSynth, a SynthSeg-based model purely trained on widely available CT segmentations (no real images). More generally, we assess accuracy and generalizability by leveraging three public datasets (not seen by any of the evaluated methods during their training), which span all major manufacturers, five MRI sequences, as well as a variety of subject conditions, voxel resolutions, and fields-of-view. Our results reveal that MRSegmentator achieves the best performance and is most generalizable. In contrast, ABDSynth yields slightly less accurate results, but its relaxed requirements in training data make it an alternative when the annotation budget is limited. The evaluation code and datasets are given for future benchmarking at this https URL, along with inference code and weights for ABDSynth.
@article{krishnaswamy_benchmarking_2025, title = {Benchmarking of deep learning methods for generic {MRI} multi-organ abdominal segmentation}, author = {Krishnaswamy, Deepa and Ciausu, Cosmin and Pieper, Steve and Kikinis, Ron and Billot, Benjamin and Fedorov, Andrey}, journal = {Journal of Medical Imaging}, year = {2025}, }
Journal
A contrast-agnostic method for ultra-high resolution claustrum segmentation

Chiara Mauri, Ryan Fritz, Jocelyn Mora, and 5 more authors

Human Brain Mapping, 2025

Abs Bib HTML PDF

The claustrum is a band-like gray matter structure located between putamen and insula whose exact functions are still actively researched. Its sheet-like structure makes it barely visible in in vivo magnetic resonance imaging (MRI) scans at typical resolutions, and neuroimaging tools for its study, including methods for automatic segmentation, are currently very limited. In this paper, we propose a contrast- and resolution-agnostic method for claustrum segmentation at ultra-high resolution (0.35 mm isotropic); the method is based on the SynthSeg segmentation framework, which leverages the use of synthetic training intensity images to achieve excellent generalization. In particular, SynthSeg requires only label maps to be trained, since corresponding intensity images are synthesized on the fly with random contrast and resolution. We trained a deep learning network for automatic claustrum segmentation, using claustrum manual labels obtained from 18 ultra-high resolution MRI scans (mostly ex vivo). We demonstrated the method to work on these 18 high resolution cases (Dice score = 0.632, mean surface distance = 0.458 mm, and volumetric similarity = 0.867 using 6-fold cross validation (CV)), and also on in vivo T1-weighted MRI scans at typical resolutions (≈1 mm isotropic). We also demonstrated that the method is robust in a test–retest setting and when applied to multimodal imaging (T2-weighted, proton density, and quantitative T1 scans). To the best of our knowledge this is the first accurate method for automatic ultra-high resolution claustrum segmentation, which is robust against changes in contrast and resolution. The method is released at https://github.com/chiara-mauri/claustrum_segmentation and as part of the neuroimaging package FreeSurfer.
@article{mauri_contrast_2025, title = {A contrast-agnostic method for ultra-high resolution claustrum segmentation}, author = {Mauri, Chiara and Fritz, Ryan and Mora, Jocelyn and Billot, Benjamin and Iglesias, Juan Eugenio and Van Leemput, Koen and Augustinack, Jean and Greve, Douglas}, journal = {Human Brain Mapping}, year = {2025}, volume = {46}, number = {12}, }
Journal
Brain MRI signatures across sex and CSF Alzheimer’s disease biomarkers

You Cheng, Yingnan He, Karthik Gopinath, and 10 more authors

Brain Communications, 2025

Abs Bib HTML PDF

The relationship between cerebrospinal fluid (CSF) biomarkers of Alzheimer’s disease and neurodegenerative effects is not fully understood. This study investigates neurodegeneration patterns across CSF Alzheimer’s disease biomarker groups, the association of brain volumes with CSF amyloid and tau status and sex differences in these relationships in a clinical neurology sample. MRI and CSF Alzheimer’s disease biomarkers data were analysed in 306 patients of the Mass General Brigham healthcare system aged 50+ (mean age = 68.4 ± 8.8 years; 43.1% female), who had lumbar punctures within 1 year of clinical MRI scans. We first analysed neurodegeneration patterns across four biomarker groups: 60 controls (A−T−&CU; amyloid negative, tau negative, cognitively unimpaired), 25 A+T− (amyloid positive, tau negative), 121 A+T+ (amyloid positive, tau positive) and 100 other dementia (A−T−&CI; amyloid negative, tau negative, cognitively impaired). Second, we examined volumetric associations with amyloid (amyloid positive, tau negative versus control) and tau in the presence of amyloid (amyloid positive, tau positive versus amyloid positive, tau negative) across 52 brain areas. Third, we examined sex differences in these relationships. Finally, we validated core analyses across three independent datasets—NACC (National Alzheimer’s Coordinating Center), ADNI (Alzheimer’s Disease Neuroimaging Initiative) and EPAD (European Prevention of Alzheimer’s Dementia)—totalling 3137 participants, and performed meta-analyses to obtain more robust estimates. We observed distinct neurodegeneration patterns across biomarker groups, with disrupted connectivity (brain volume covariance networks) in amyloid positive and other dementia groups, while amyloid and tau negative, cognitively unimpaired controls exhibited the most connected network. Amyloid was associated with subcortical, cerebellar and brainstem atrophy, with consistent association observations in the thalamus and amygdala across all four datasets. Tau in the presence of amyloid demonstrated general brain shrinkage through enlargement of extracerebral CSF, alongside unexpected ventricle shrinkages. Sex-based analyses revealed that A+T+ (amyloid positive, tau positive) had lower sex differences in connectivity patterns compared with other groups. Sex differences were also noted in amyloid-related ventricular volume changes. This study reveals how amyloid and tau affect brain connectivity and volume across sex and CSF biomarker groups, emphasizing global brain changes and sex differences. By leveraging automated pipelines and advanced MRI and biomarker analyses, we extracted meaningful and replicable findings from heterogeneous clinical samples from real-world data. The meta-analyses across four datasets enhance the generalizability of our results.
@article{cheng_brain_2025, title = {Brain {MRI} signatures across sex and {CSF} {Alzheimer}’s disease biomarkers}, author = {Cheng, You and He, Yingnan and Gopinath, Karthik and Billot, Benjamin and Iglesias, Juan Eugenio and Wu, Chao-Yi and Dodge, Hiroko and Wills, Anne-Marie and Carlyle, Becky and Kivisäkk, Pia and Hyman, Bradley and Arnold, Steven and Das, Sudeshna}, journal = {Brain Communications}, year = {2025}, volume = {7}, number = {3}, }
Conf.
Spatial regularisation for improved accuracy and interpretability in keypoint-based registration

Benjamin Billot, Ramya Muthukrishnan, Esra Abaci Turk, and 4 more authors

In MICCAI, 2025

Abs Bib HTML PDF

Unsupervised registration strategies bypass requirements in ground truth transforms or segmentations by optimising similarity metrics between fixed and moved volumes. Among these methods, a recent subclass of approaches based on unsupervised keypoint detection stand out as very promising for interpretability. Specifically, these methods train a network to predict feature maps for fixed and moving images, from which explainable centres of mass are computed to obtain point clouds, that are then aligned in closed-form. However, the features returned by the network often yield spatially diffuse patterns that are hard to interpret, thus undermining the purpose of keypoint-based registration. Here, we propose a three-fold loss to regularise the spatial distribution of the features. First, we use the KL divergence to model features as point spread functions that we interpret as probabilistic keypoints. Then, we sharpen the spatial distributions of these features to increase the precision of the detected landmarks. Finally, we introduce a new repulsive loss across keypoints to encourage spatial diversity. Overall, our loss considerably improves the interpretability of the features, which now correspond to precise and anatomically meaningful landmarks. We demonstrate our three-fold loss in foetal rigid motion tracking and brain MRI affine registration tasks, where it not only outperforms state-of-the-art unsupervised strategies, but also bridges the gap with state-of-the-art supervised methods. Our code is available at https://github.com/BenBillot/spatial_regularisation
@inproceedings{billot_spatial_2025, title = {Spatial regularisation for improved accuracy and interpretability in keypoint-based registration}, author = {Billot, Benjamin and Muthukrishnan, Ramya and Abaci Turk, Esra and Grant, Ellen and Ayache, Nicholas and Delingette, Herv{\'e} and Golland, Polina}, booktitle = {MICCAI}, year = {2025}, pages = {583--593}, }
Conf.
Robust fetal pose estimation across gestational ages via cross-population augmentation

Sebastian iaz, Benjamin Billot, Neel Dey, and 5 more authors

In MICCAI, 2025

Abs Bib HTML PDF

Fetal motion is a critical indicator of neurological development and intrauterine health, yet its quantification remains challenging, particularly at earlier gestational ages (GA). Current methods track fetal motion by predicting the location of annotated landmarks on 3D echo planar imaging (EPI) time-series, primarily in third-trimester fetuses. The predicted landmarks enable simplification of the fetal body for downstream analysis. While these methods perform well within their training age distribution, they consistently fail to generalize to early GAs due to significant anatomical changes in both mother and fetus across gestation, as well as the difficulty of obtaining annotated early GA EPI data. In this work, we develop a cross-population data augmentation framework that enables pose estimation models to robustly generalize to younger GA clinical cohorts using only annotated images from older GA cohorts. Specifically, we introduce a fetal-specific augmentation strategy that simulates the distinct intrauterine environment and fetal positioning of early GAs. Our experiments find that cross-population augmentation yields reduced variability and significant improvements across both older GA and challenging early GA cases. By enabling more reliable pose estimation across gestation, our work potentially facilitates early clinical detection and intervention in challenging 4D fetal imaging settings. Code is available at https://github.com/sebodiaz/cross-population-pose.
@inproceedings{diaz_robust_2025, title = {Robust fetal pose estimation across gestational ages via cross-population augmentation}, author = {iaz, Sebastian and Billot, Benjamin and Dey, Neel and Zhang, Molin and Abaci Turk, Esra and Grant, Ellen and Golland, Polina and Adalsteinsson, Elfar}, booktitle = {MICCAI}, year = {2025}, pages = {549--559}, }
Conf.
Fetuses made simple: modeling and tracking of fetal shape and pose

Yingcheng Liu, Peiqi Wang, Sebastian Diaz, and 4 more authors

In MICCAI, 2025

Abs Bib HTML PDF

Transrectal ultrasound (TRUS) image-guidance is crucial in monitoring prostate cancer by enabling precise lesion targeting for biopsy and radiotherapy. Image-guided navigation can be further improved by manually overlaying a 3D preoperative MRI scan to benefit from its superior contrast and resolution. Here we propose to automate this procedure, which we formulate as MRI-TRUS 3D rigid registration. However, this task faces three challenges: the large domain gap between MRI and TRUS; the relative spherical symmetry of the prostate that complicates rotation estimation; the potential large misalignments due to the wide range of ultrasound probe orientations relative to the MR coordinate frame. Here we present MUReg, a new pipeline for MRI-TRUS rigid registration. First, we initialize translations by aligning the prostate centers of mass derived from segmentations. Then, we regress rotations with an attention-based CNN that takes segmentations as inputs (to eliminate the domain gap) and that is trained with a novel loss penalizing displacement errors to alleviate the symmetry issue. Finally, we enhance the robustness of these initial predictions by using an iterative closest point algorithm. Overall, MUReg significantly outperforms state-of-the-art optimization and deep learning methods on four different metrics.
@inproceedings{liu_fetuses_2025, title = {Fetuses made simple: modeling and tracking of fetal shape and pose}, author = {Liu, Yingcheng and Wang, Peiqi and Diaz, Sebastian and Abaci Turk, Esra and Billot, Benjamin and Grant, Ellen and Golland, Polina}, booktitle = {MICCAI}, year = {2025}, pages = {189--198}, }
Conf.
Robust rigid MRI-TRUS registration in prostate cancer using attention-CNN and ICP

Manasi Kattel, Benjamin Billot, Federica Facente, and 2 more authors

In MICCAI – ASMUS workshop, 2025

Abs Bib HTML PDF

Transrectal ultrasound (TRUS) image-guidance is crucial in monitoring prostate cancer by enabling precise lesion targeting for biopsy and radiotherapy. Image-guided navigation can be further improved by manually overlaying a 3D preoperative MRI scan to benefit from its superior contrast and resolution. Here we propose to automate this procedure, which we formulate as MRI-TRUS 3D rigid registration. However, this task faces three challenges: the large domain gap between MRI and TRUS; the relative spherical symmetry of the prostate that complicates rotation estimation; the potential large misalignments due to the wide range of ultrasound probe orientations relative to the MR coordinate frame. Here we present MUReg, a new pipeline for MRI-TRUS rigid registration. First, we initialize translations by aligning the prostate centers of mass derived from segmentations. Then, we regress rotations with an attention-based CNN that takes segmentations as inputs (to eliminate the domain gap) and that is trained with a novel loss penalizing displacement errors to alleviate the symmetry issue. Finally, we enhance the robustness of these initial predictions by using an iterative closest point algorithm. Overall, MUReg significantly outperforms state-of-the-art optimization and deep learning methods on four different metrics.
@inproceedings{kattel_robust_2025, title = {Robust rigid {MRI}-{TRUS} registration in prostate cancer using attention-{CNN} and {ICP}}, author = {Kattel, Manasi and Billot, Benjamin and Facente, Federica and Delingette, Herv{\'e} and Ayache, Nicholas}, booktitle = {MICCAI -- ASMUS workshop}, year = {2025}, pages = {65--75}, }
Conf.
Multi-stage CNN for fast registration of 3D preoperative CTs to 2D intraoperative X-rays

Federica Facente, Benjamin Billot, Vivek Gopalakrishnan, and 6 more authors

In MICCAI – COLAS workshop, 2025

Abs Bib HTML PDF

Minimally invasive interventions often rely on live 2D X-rays for image guidance. Yet, anatomical localization and procedural accuracy can be enhanced by spatial alignment of these intraoperative X-rays with 3D preoperative computed tomographies (CTs). This 3D/2D registration problem is typically formulated as pose estimation of the X-ray source relatively to the CT, which is done by simulating synthetic X-rays from 2D projections of CT volumes. However, the optimization-based refinement used by the state-of-the-art deep learning approach takes several seconds, thus exceeding the allowed time budget in live image guidance. In this paper, we propose LXPose (Live X-ray Pose estimation), a selfsupervised multi-stage 3D/2D registration framework for real-time image guidance. LXpose removes the dependency on optimization and leverages a two-stage CNN trained with a projection loss to ensure high accuracy and computational efficiency. Moreover, we apply extensive data augmentation to mitigate the domain gap between simulated and real X-rays. Overall, LXPose yields comparable 2D registration error to the state-of-the-art method, while reducing inference time to 20ms, which demonstrates the potential of LXPose for real-time clinical deployment. Our code is available at https://github.com/fedefacente/LXPose.git.
@inproceedings{facente_multi_2025, title = {Multi-stage CNN for fast registration of 3D preoperative CTs to 2D intraoperative X-rays}, author = {Facente, Federica and Billot, Benjamin and Gopalakrishnan, Vivek and Kattel, Manasi and Wei, Wen and Golland, Polina and Delingette, Herv{\'e} and Ayache, Nicholas and Berthet-Rayne, Pierre}, booktitle = {MICCAI -- COLAS workshop}, year = {2025}, }
Conf.
Learning general-purpose biomedical volume representations using randomized synthesis

Neel Dey, Benjamin Billot, Hallee Wong, and 5 more authors

In ICLR, 2025

Abs Bib HTML PDF

Current volumetric biomedical foundation models struggle to generalize as public 3D datasets are small and do not cover the broad diversity of medical procedures, conditions, anatomical regions, and imaging protocols. We address this by creating a representation learning method that instead anticipates strong domain shifts at training time itself. We first propose a data engine that synthesizes highly variable training samples that would enable generalization to new biomedical contexts. To then train a single 3D network for any voxel-level task, we develop a contrastive learning method that pretrains the network to be stable against nuisance imaging variation simulated by the data engine, a key inductive bias for generalization. This network’s features can be used as robust representations of input images for downstream tasks and its weights provide a strong, dataset-agnostic initialization for finetuning on new datasets. As a result, we set new standards across both multimodality registration and few-shot segmentation, a first for any 3D biomedical vision model, all without (pre-)training on any existing dataset of real images.
@inproceedings{dey_learning_2025, title = {Learning general-purpose biomedical volume representations using randomized synthesis}, author = {Dey, Neel and Billot, Benjamin and Wong, Hallee and Wang, Clinton and Ren, Mengwei and Grant, Ellen and Dalca, Adrian and Golland, Polina}, booktitle = {ICLR}, year = {2025}, }

2024

Journal
Synthetic data in generalizable, learning-based neuroimaging

Karthik Gopinath, Andrew Hoopes, Daniel C. Alexander, and 24 more authors

Imaging Neuroscience, 2024

Abs Bib HTML PDF

Synthetic data have emerged as an attractive option for developing machine-learning methods in human neuroimaging, particularly in magnetic resonance imaging (MRI)—a modality where image contrast depends enormously on acquisition hardware and parameters. This retrospective paper reviews a family of recently proposed methods, based on synthetic data, for generalizable machine learning in brain MRI analysis. Central to this framework is the concept of domain randomization, which involves training neural networks on a vastly diverse array of synthetically generated images with random contrast properties. This technique has enabled robust, adaptable models that are capable of handling diverse MRI contrasts, resolutions, and pathologies, while working out-of-the-box, without retraining. We have successfully applied this method to tasks such as whole-brain segmentation (SynthSeg), skull-stripping (SynthStrip), registration (SynthMorph, EasyReg), super-resolution, and MR contrast transfer (SynthSR). Beyond these applications, the paper discusses other possible use cases and future work in our methodology. Neural networks trained with synthetic data enable the analysis of clinical MRI, including large retrospective datasets, while greatly alleviating (and sometimes eliminating) the need for substantial labeled datasets, and offer enormous potential as robust tools to address various research goals.
@article{gopinath_synthetic_2025, title = {Synthetic data in generalizable, learning-based neuroimaging}, author = {Gopinath, Karthik and Hoopes, Andrew and Alexander, Daniel C. and Arnold, Steven E. and Balbastre, Yael and Billot, Benjamin and Casamitjana, Adrià and Cheng, You and Chua, Russ Yue Zhi and Edlow, Brian L. and Fischl, Bruce and Gazula, Harshvardhan and Hoffmann, Malte and Keene, C. Dirk and Kim, Seunghoi and Kimberly, W. Taylor and Laguna, Sonia and Larson, Kathleen E. and Van Leemput, Koen and Puonti, Oula and Rodrigues, Livia M. and Rosen, Matthew S. and Tregidgo, Henry F. J. and Varadarajan, Divya and Young, Sean I. and Dalca, Adrian V. and Iglesias, Juan Eugenio}, journal = {Imaging Neuroscience}, year = {2024}, volume = {2}, }
Journal
SE(3)-equivariant and noise-invariant 3D rigid motion tracking in brain MRI

Benjamin Billot, Neel Dey, Daniel Moyer, and 5 more authors

IEEE Transactions on Medical Imaging, 2024

Abs Bib HTML PDF

Rigid motion tracking is paramount in many medical imaging applications where movements need to be detected, corrected, or accounted for. Modern strategies rely on convolutional neural networks (CNN) and pose this problem as rigid registration. Yet, CNNs do not exploit natural symmetries in this task, as they are equivariant to translations (their outputs shift with their inputs) but not to rotations. Here we propose EquiTrack, the first method that uses recent steerable SE(3)-equivariant CNNs (E-CNN) for motion tracking. While steerable E-CNNs can extract corresponding features across different poses, testing them on noisy medical images reveals that they do not have enough learning capacity to learn noise invariance. Thus, we introduce a hybrid architecture that pairs a denoiser with an E-CNN to decouple the processing of anatomically irrelevant intensity features from the extraction of equivariant spatial features. Rigid transforms are then estimated in closed-form. EquiTrack outperforms state-of-the-art learning and optimisation methods for motion tracking in adult brain MRI and fetal MRI time series. Our code is available at https://github.com/BBillot/EquiTrack.
@article{billot_se3_2024, title = {{SE(3)}-equivariant and noise-invariant {3D} rigid motion tracking in brain {MRI}}, author = {Billot, Benjamin and Dey, Neel and Moyer, Daniel and Hoffmann, Malte and Abaci Turk, Esra and Gagoski, Borjan and Grant, Ellen and Golland, Polina}, journal = {IEEE Transactions on Medical Imaging}, year = {2024}, volume = {43}, number = {11}, pages = {4029--4040}, }
Conf.
Network conditioning for synergistic learning on partial annotations

Benjamin Billot, Daniel Moyer, Neerav Karani, and 4 more authors

In MIDL, 2024

Abs Bib HTML PDF

The robustness and accuracy of multi-organ segmentation networks is limited by the scarcity of labels. A common strategy to alleviate the annotation burden is to use partially labelled datasets, where each image can be annotated for a subset of all organs of interest. Unfortunately, this approach causes inconsistencies in the background class since it can now include target organs. Moreover, we consider the even more relaxed setting of region-based segmentation, where voxels can be labelled for super-regions, thus causing further inconsistencies across annotations. Here we propose CoNeMOS (Conditional Network for Multi-Organ Segmentation), a framework that leverages a label-conditioned network for synergistic learning on partially labelled region-based segmentations. Conditioning is achieved by combining convolutions with expressive Feature-wise Linear Modulation (FiLM) layers, whose parameters are controlled by an auxiliary network. In contrast to other conditioning methods, FiLM layers are stable to train and add negligible computation overhead, which enables us to condition the entire network. As a result, the network can learn where it needs to extract shared or label-specific features, instead of imposing it with the architecture (e.g., with different segmentation heads). By encouraging flexible synergies across labels, our method obtains state-of-the-art results for the segmentation of challenging low-resolution fetal MRI data. Our code is available at https://github.com/BBillot/CoNeMOS.
@inproceedings{billot_network_2024, title = {Network conditioning for synergistic learning on partial annotations}, author = {Billot, Benjamin and Moyer, Daniel and Karani, Neerav and Hoffmann, Malte and Abaci Turk, Esra and Grant, Ellen and Golland, Polina}, booktitle = {MIDL}, year = {2024}, }
Conf.
AnyStar: domain randomized universal star-convex 3D instance segmentation

Neel Dey, Mazdak Abulnaga, Benjamin Billot, and 4 more authors

In Winter Conference on Applications of Computer Vision, 2024

Abs Bib HTML PDF

Star-convex shapes arise across bio-microscopy and radiology in the form of nuclei, nodules, metastases, and other units. Existing instance segmentation networks for such structures train on densely labeled instances for each dataset, which requires substantial and often impractical manual annotation effort. Further, significant reengineering or fine tuning is needed when presented with new datasets and imaging modalities due to changes in contrast, shape, orientation, resolution, and density. We present AnyStar, a domain-randomized generative model that simulates synthetic training data of blob-like objects with randomized appearance, environments, and imaging physics to train general-purpose star-convex instance segmentation networks. As a result, networks trained using our generative model do not require annotated images from unseen datasets. A single network trained on our synthesized data accurately 3D segments C.elegans and P.dumerilii nuclei in fluorescence microscopy, mouse cortical nuclei in µCT, zebra fish brainn uclei in EM, and placental cotyledons in human fetal MRI, all without any retraining, fine tuning, transfer learning, or domain adaptation. Code is available at https://github.com/neel-dey/AnyStar.
@inproceedings{dey_anystar_2024, title = {{AnyStar}: domain randomized universal star-convex {3D} instance segmentation}, author = {Dey, Neel and Abulnaga, Mazdak and Billot, Benjamin and Abaci Turk, Esra and Grant, Ellen and Dalca, Adrian and Golland, Polina}, booktitle = {Winter Conference on Applications of Computer Vision}, year = {2024}, pages = {7578--7588}, }
Conf.
Association of brain MRI signatures with Alzheimer disease CSF biomarkers

You Cheng, Karthik Gopinath, Benjamin Billot, and 8 more authors

In Alzheimer’s Association International Conference, 2024

Abs Bib HTML PDF

The relationship between CSF measures of Alzheimer disease (AD) pathologies and their neurodegenerative signatures is not fully understood. This study seeks to employ machine learning approaches on clinical MRI data to identify patterns associated with amyloid and tau, aiming to guide diagnosis and therapeutic interventions. Method We selected brain volumes that differed significantly between AD pathology and control groups. Then we utilized logistic LASSO regression to identify and compare neurodegenerative signatures associated with amyloid and tau burdens using selected brain volumes. Our methods were applied to a dataset that included clinical MRI scans from patients who also had cerebrospinal fluid (CSF) biomarkers assessed within a year of the MRI scan (Figure 1). Model performance was measured using the cross-validation area under the receiver operating characteristic curve (AUROC), as well as the accuracy, sensitivity, and specificity in a 25% held-out test set. Result For amyloid status, among 30 amyloid-positive, tau-negative (A+T-) patients and 24 cognitively unimpaired A-T- controls, we identified 25 significant features associated with amyloid status, including subcortical areas (e.g., amygdala, thalamus, hippocampus), cortical areas (e.g., inferior temporal cortex, inferior parietal cortex, precuneus), ventricles (lateral ventricle, inferior lateral ventricle, and the 3rd ventricle), cerebellum (cerebellum white matter and cerebellum cortex), and brain stem (Figure 2A). Our model with these features yielded an AUROC of 0.792 (95% CI: [0.783, 0.801]), and a sensitivity of 0.857 at specificity of 1 (Figure 2B). In the assessment of tau status, where the amyloid-negative, tau-positive group (A-T+) included 14 patients and the control group comprised 24. The volumes showed significant difference associated with tau status, including subcortical areas (amygdala, ventral diencephalon, hippocampus), the 4th ventricle, and cerebellum white matter (Figure 3A). The model achieved an AUROC of 0.858 (95% C.I. [0.85, 0.866]), with a sensitivity of 1 at specificity of 1 (Figure 3B and C). Conclusion The study underscores the potential of using widely available clinical MRI images to measure AD and associated pathologies in the brain, which hold significant promise for guiding treatment interventions.
@inproceedings{cheng_association_2024, title = {Association of brain {MRI} signatures with {Alzheimer} disease {CSF} biomarkers}, author = {Cheng, You and Gopinath, Karthik and Billot, Benjamin and Iglesias, Juan Eugenio and Touroutoglou, Alexandra and Kivisäkk, Pia and Wu, Chao-Yi and Dodge, Hiroko and Hyman, Bradley and Arnold, Steven and Das, Sudeshna}, booktitle = {Alzheimer's Association International Conference}, year = {2024}, }
Conf.
Quantifying white matter hyperintensity and brain volumes in heterogeneous clinical and low-field portable MRI

Pablo Laso, Stefano Cerri, Annabel Sorby-Adams, and 21 more authors

In ISBI, 2024

Abs Bib HTML PDF

Brain atrophy and white matter hyperintensity (WMH) are critical neuroimaging features for ascertaining brain injury in cerebrovascular disease and multiple sclerosis. Automated segmentation and quantification is desirable but existing methods require high-resolution MRI with good signal-to-noise ratio (SNR). This precludes application to clinical and low-field portable MRI (pMRI) scans, thus hampering large-scale tracking of atrophy and WMH progression, especially in underserved areas where pMRI has huge potential. Here we present a method that segments white matter hyperintensity and 36 brain regions from scans of any resolution and contrast (including pMRI) without retraining. We show results on eight public datasets and on a private dataset with paired high- and low-field scans (3T and 64mT), where we attain strong correlation between the WMH (ρ=.85) and hippocampal volumes (ρ=.89) estimated at both fields. Our method is publicly available as part of FreeSurfer, at: http://surfer.nmr.mgh.harvard.edu/fswiki/WMH-SynthSeg.
@inproceedings{laso_quantifying_2024, title = {Quantifying white matter hyperintensity and brain volumes in heterogeneous clinical and low-field portable {MRI}}, author = {Laso, Pablo and Cerri, Stefano and Sorby-Adams, Annabel and Guo, Jennifer and Mateen, Farrah and Goebl, Philipp and Wu, Jiaming and Liu, Peirong and Li, Hongwei and Young, Sean and Billot, Benjamin and Puonti, Oula and Sze, Gordon and Payabavash, Sam and DeHavenon, Adam and Sheth, Kevin and Rosen, Matthew and Kirsch, John and Strisciuglio, Nicola and Wolterink, Jelmer and Eshaghi, Arman and Barkhof, Frederik and Kimberly, Taylor and Iglesias, Juan Eugenio}, booktitle = {ISBI}, year = {2024}, }

2023

Journal
Robust machine learning segmentation for large-scale analysis of heterogeneous clinical brain MRI datasets

Benjamin Billot, Magdamo Colin, You Cheng, and 2 more authors

Proceedings of the National Academy of Sciences (PNAS), 2023

Abs Bib HTML PDF

Every year, millions of brain MRI scans are acquired in hospitals, which is a figure considerably larger than the size of any research dataset. Therefore, the ability to analyze such scans could transform neuroimaging research. Yet, their potential remains untapped since no automated algorithm is robust enough to cope with the high variability in clinical acquisitions (MR contrasts, resolutions, orientations, artifacts, and subject populations). Here, we present SynthSeg+, an AI segmentation suite that enables robust analysis of heterogeneous clinical datasets. In addition to whole-brain segmentation, SynthSeg+ also performs cortical parcellation, intracranial volume estimation, and automated detection of faulty segmentations (mainly caused by scans of very low quality). We demonstrate SynthSeg+ in seven experiments, including an aging study on 14,000 scans, where it accurately replicates atrophy patterns observed on data of much higher quality. SynthSeg+ is publicly released as a ready-to-use tool to unlock the potential of quantitative morphometry.
@article{billot_robust_2023, title = {{Robust} machine learning segmentation for large-scale analysis of heterogeneous clinical brain {MRI} datasets}, author = {Billot, Benjamin and Colin, Magdamo and Cheng, You and Das, Sudeshna and Iglesias, Juan Eugenio}, journal = {{Proceedings} of the {National} {Academy} of {Sciences} ({PNAS})}, year = {2023}, volume = {120}, number = {9}, }
Journal
Linking brain structure, cognition, and sleep: insights from clinical data

Ruoqi Wei, Wolfgang Ganglberger, Haoqi Sun, and 11 more authors

Sleep, 2023

Abs Bib HTML PDF

To use relatively noisy routinely collected clinical data (brain magnetic resonance imaging (MRI) data, clinical polysomnography (PSG) recordings, and neuropsychological testing), to investigate hypothesis-driven and data-driven relationships between brain physiology, structure, and cognition.We analyzed data from patients with clinical PSG, brain MRI, and neuropsychological evaluations. SynthSeg, a neural network-based tool, provided high-quality segmentations despite noise. A priori hypotheses explored associations between brain function (measured by PSG) and brain structure (measured by MRI). Associations with cognitive scores and dementia status were studied. An exploratory data-driven approach investigated age-structure-physiology-cognition links.Six hundred and twenty-three patients with sleep PSG and brain MRI data were included in this study; 160 with cognitive evaluations. Three hundred and forty-two participants (55%) were female, and age interquartile range was 52 to 69 years. Thirty-six individuals were diagnosed with dementia, 71 with mild cognitive impairment, and 326 with major depression. One hundred and fifteen individuals were evaluated for insomnia and 138 participants had an apnea–hypopnea index equal to or greater than 15. Total PSG delta power correlated positively with frontal lobe/thalamic volumes, and sleep spindle density with thalamic volume. rapid eye movement (REM) duration and amygdala volume were positively associated with cognition. Patients with dementia showed significant differences in five brain structure volumes. REM duration, spindle, and slow-oscillation features had strong associations with cognition and brain structure volumes. PSG and MRI features in combination predicted chronological age (R2 = 0.67) and cognition (R2 = 0.40).Routine clinical data holds extended value in understanding and even clinically using brain-sleep-cognition relationships.
@article{wei_linking_2023, title = {Linking brain structure, cognition, and sleep: insights from clinical data}, author = {Wei, Ruoqi and Ganglberger, Wolfgang and Sun, Haoqi and Hadar, Peter and Gollub, Randy and Pieper, Steve and Billot, Benjamin and Au, Rhoda and Iglesias, Juan Eugenio and Cash, Sydney and Kim, Soriul and Shin, Chol and Westover, Brandon and Thomas, Robert}, journal = {Sleep}, year = {2023}, volume = {47}, number = {2}, }
Journal
Brain growth charts for quantitative analysis of pediatric clinical brain MRI scans with limited imaging pathology

Jenna Schabdach, Eric Schmitt, Susan Sotardi, and 23 more authors

Radiology, 2023

Abs Bib HTML PDF

Clinically acquired brain MRI scans represent a valuable but underused resource for investigating neurodevelopment dueto their technical heterogeneity and lack of appropriate controls. These barriers have curtailed retrospective studies of clinical brainMRI scans compared with more costly prospectively acquired research-quality brain MRI scans.Purpose: To provide a benchmark for neuroanatomic variability in clinically acquired brain MRI scans with limited imaging pathology(SLIPs) and to evaluate if growth charts from curated clinical MRI scans differed from research-quality MRI scans or were influencedby clinical indication for the scan.Materials and Methods: In this secondary analysis of preexisting data, clinical brain MRI SLIPs from an urban pediatric health care system(individuals aged ≤22 years) were scanned across nine 3.0-T MRI scanners. The curation process included manual review of signed ra-diology reports and automated and manual quality review of images without gross pathology. Global and regional volumetric imagingphenotypes were measured using two image segmentation pipelines, and clinical brain growth charts were quantitatively compared withcharts derived from a large set of research controls in the same age range by means of Pearson correlation and age at peak volume.Results: The curated clinical data set included 532 patients (277 male; median age, 10 years [IQR, 5–14 years]; age range, 28 days af-ter birth to 22 years) scanned between 2005 and 2020. Clinical brain growth charts were highly correlated with growth charts derivedfrom research data sets (22 studies, 8346 individuals [4947 male]; age range, 152 days after birth to 22 years) in terms of normativedevelopmental trajectories predicted by the models (median r = 0.979).Conclusion: The clinical indication of the scans did not significantly bias the output of clinical brain charts. Brain growth charts derivedfrom clinical controls with limited imaging pathology were highly correlated with brain charts from research controls, suggesting thepotential of curated clinical MRI scans to supplement research data sets.
@article{schabdach_brain_2023, title = {Brain growth charts for quantitative analysis of pediatric clinical brain {MRI} scans with limited imaging pathology}, author = {Schabdach, Jenna and Schmitt, Eric and Sotardi, Susan and Vossough, Arastoo and Andronikou, Savvas and Roberts, Timothy and Huang, Hao and Padmanabhan, Viveknarayanan and Ortiz-Rosa, Alfredo and Gardner, Margaret and Covitz, Sydney and Bedford, Saashi and Mandal, Ayan and Chaiyachati, Barbara and White, Simon and Bullmore, Edward and Bethlehem, Richard and Shinohara, Russell and Billot, Benjamin and Iglesias, Juan Eugenio and Ghosh, Satrajit and Gur, Raquel and Satterthwaite, Theodore and Roalf, David and Seidlitz, Jakob and Alexander-Bloch, Aaron}, journal = {Radiology}, year = {2023}, volume = {309}, number = {1}, }
Journal
Machine learning of dissection photographs and surface scanning for quantitative 3D neuropathology

Harsha Gazula, Henry Tregidgo, Benjamin Billot, and 23 more authors

eLife, 2023

Abs Bib HTML

We present open-source tools for 3D analysis of photographs of dissected slices of human brains, which are routinely acquired in brain banks but seldom used for quantitative analysis. Our tools can: (i) 3D reconstruct a volume from the photographs and, optionally, a surface scan; and (ii) produce a high-resolution 3D segmentation into 11 brain regions, independently of the slice thickness. Our tools can be used as a substitute for ex vivo magnetic resonance imaging (MRI), which requires access to an MRI scanner, ex vivo scanning expertise, and considerable financial resources. We tested our tools on synthetic and real data from two NIH Alzheimer’s Disease Research Centers. The results show that our methodology yields accurate 3D reconstructions, segmentations, and volumetric measurements that are highly correlated to those from MRI. Our method also detects expected differences between post mortem confirmed Alzheimer’s disease cases and controls. The tools are available in our widespread neuroimaging suite “FreeSurfer” (https://surfer.nmr.mgh.harvard.edu/fswiki/PhotoTools).
@article{gazula_machine_2023, title = {{Machine} learning of dissection photographs and surface scanning for quantitative {3D} neuropathology}, author = {Gazula, Harsha and Tregidgo, Henry and Billot, Benjamin and Balbastre, Yaël and William-Ramirez, Jonathan and Herisse, Rogeny and Deden-Binder, Lucas and Casamitjana, Adria and Melief, E. and Latimer, C. and Kilgore, M. and Montine, M. and Robinson, Eleanor and Blackburn, Emily and Marshall, M. and Connors, T. and Oakley, D. and Frosch, M. and Young, Sean and Van Leemput, Koen and Dalca, Adrian and Fischl, Bruce and Mac Donald, C. and Keene, C. and Hyman, B. and Iglesias, Juan Eugenio}, journal = {eLife}, year = {2023}, volume = {12}, }
Journal
SynthSR: A public AI tool to turn heterogeneous clinical brain scans into high-resolution T1-weighted images for 3D morphometry

Juan Eugenio Iglesias, Benjamin Billot, Yaël Balbastre, and 7 more authors

Science Advances, 2023

Abs Bib HTML PDF

Every year, millions of brain magnetic resonance imaging (MRI) scans are acquired in hospitals across the world. These have the potential to revolutionize our understanding of many neurological diseases, but their morphometric analysis has not yet been possible due to their anisotropic resolution. We present an artificial intelligence technique, “SynthSR,” that takes clinical brain MRI scans with any MR contrast (T1, T2, etc.), orientation (axial/coronal/sagittal), and resolution and turns them into high-resolution T1 scans that are usable by virtually all existing human neuroimaging tools. We present results on segmentation, registration, and atlasing of >10,000 scans of controls and patients with brain tumors, strokes, and Alzheimer’s disease. SynthSR yields morphometric results that are very highly correlated with what one would have obtained with high-resolution T1 scans. SynthSR allows sample sizes that have the potential to overcome the power limitations of prospective research studies and shed new light on the healthy and diseased human brain. A public AI tool turns clinical brain MRI of any resolution/contrast into 1mm T1 scans compatible with neuroimaging pipelines.
@article{iglesias_synthsr_2023, title = {SynthSR: A public AI tool to turn heterogeneous clinical brain scans into high-resolution T1-weighted images for 3D morphometry}, author = {Iglesias, Juan Eugenio and Billot, Benjamin and Balbastre, Yaël and Magdamo, Colin and Arnold, Steven and Das, Sudeshna and Edlow, Brian and Alexander, Daniel and Golland, Polina and Fischl, Bruce}, journal = {Science Advances}, year = {2023}, volume = {9}, number = {5}, }
Journal
SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining

Benjamin Billot, Douglas Greve, Oula Puonti, and 5 more authors

Medical Image Analysis, 2023

Abs Bib HTML PDF

Despite advances in data augmentation and transfer learning, convolutional neural networks (CNNs) difficultly generalise to unseen domains. When segmenting brain scans, CNNs are highly sensitive to changes in resolution and contrast: even within the same MRI modality, performance can decrease across datasets. Here we introduce SynthSeg, the first segmentation CNN robust against changes in contrast and resolution. SynthSeg is trained with synthetic data sampled from a generative model conditioned on segmentations. Crucially, we adopt a domain randomisation strategy where we fully randomise the contrast and resolution of the synthetic training data. Consequently, SynthSeg can segment real scans from a wide range of target domains without retraining or fine-tuning, which enables straightforward analysis of huge amounts of heterogeneous clinical data. Because SynthSeg only requires segmentations to be trained (no images), it can learn from labels obtained by automated methods on diverse populations (e.g., ageing and diseased), thus achieving robustness to a wide range of morphological variability. We demonstrate SynthSeg on 5,000 scans of six modalities (including CT) and ten resolutions, where it exhibits unparallelled generalisation compared with supervised CNNs, state-of-the-art domain adaptation, and Bayesian segmentation. Finally, we demonstrate the generalisability of SynthSeg by applying it to cardiac MRI and CT scans.
@article{billot_synthseg_2023, title = {SynthSeg: {Segmentation} of brain {MRI} scans of any contrast and resolution without retraining}, author = {Billot, Benjamin and Greve, Douglas and Puonti, Oula and Thielscher, Axel and Van Leemput, Koen and Fischl, Bruce and Dalca, Adrian and Iglesias, Juan Eugenio}, journal = {{Medical} {Image} {Analysis}}, year = {2023}, volume = {86}, }
Journal
Quantitative Brain Morphometry of Portable Low-Field-Strength MRI Using Super-Resolution Machine Learning

Juan Eugenio Iglesias, Riana Schleicher, Sonia Laguna, and 7 more authors

Radiology, 2023

Abs Bib HTML PDF

Background Portable, low-field-strength (0.064-T) MRI has the potential to transform neuroimaging but is limited by low spatial resolution and low signal-to-noise ratio. Purpose To implement a machine learning super-resolution algorithm that synthesizes higher spatial resolution images (1-mm isotropic) from lower resolution T1-weighted and T2-weighted portable brain MRI scans, making them amenable to automated quantitative morphometry. Materials and Methods An external high-field-strength MRI data set (1-mm isotropic scans from the Open Access Series of Imaging Studies data set) and segmentations for 39 regions of interest (ROIs) in the brain were used to train a super-resolution convolutional neural network (CNN). Secondary analysis of an internal test set of 24 paired low- and high-field-strength clinical MRI scans in participants with neurologic symptoms was performed. These were part of a prospective observational study (August 2020 to December 2021) at Massachusetts General Hospital (exclusion criteria: inability to lay flat, body habitus preventing low-field-strength MRI, presence of MRI contraindications). Three well-established automated segmentation tools were applied to three sets of scans: high-field-strength (1.5–3 T, reference standard), low-field-strength (0.064 T), and synthetic high-field-strength images generated from the low-field-strength data with the CNN. Statistical significance of correlations was assessed with Student t tests. Correlation coefficients were compared with Steiger Z tests. Results Eleven participants (mean age, 50 years ± 14; seven men) had full cerebrum coverage in the images without motion artifacts or large stroke lesion with distortion from mass effect. Direct segmentation of low-field-strength MRI yielded nonsignificant correlations with volumetric measurements from high field strength for most ROIs (P > .05). Correlations largely improved when segmenting the synthetic images: P values were less than .05 for all ROIs (eg, for the hippocampus [r = 0.85; P < .001], thalamus [r = 0.84; P = .001], and whole cerebrum [r = 0.92; P < .001]). Deviations from the model (z score maps) visually correlated with pathologic abnormalities. Conclusion This work demonstrated proof-of-principle augmentation of portable MRI with a machine learning super-resolution algorithm, which yielded highly correlated brain morphometric measurements to real higher resolution images. © RSNA, 2022 Online supplemental material is available for this article. See also the editorial by Ertl-Wagner amd Wagner in this issue. An earlier incorrect version appeared online.
@article{iglesias_quantitative_2023, title = {Quantitative Brain Morphometry of Portable Low-Field-Strength MRI Using Super-Resolution Machine Learning}, author = {Iglesias, Juan Eugenio and Schleicher, Riana and Laguna, Sonia and Billot, Benjamin and Schaefer, Pamela and McKaig, Brenna and Goldstein, Joshua and Sheth, Kevin and Rosen, Matthew and Kimberly, Taylor}, journal = {Radiology}, year = {2023}, volume = {306}, number = {3}, }
Conf.
Domain-agnostic segmentation of thalamic nuclei from joint structural and diffusion MRI

Henry Tregidgo, Sonja Soskic, Mark Olchanyi, and 9 more authors

In MICCAI, 2023

Abs Bib HTML PDF

The human thalamus is a subcortical brain structure that comprises dozens of nuclei with different function and connectivity, which are affected differently by disease. For this reason, there is growing interest in studying the thalamic nuclei in vivo with MRI. Tools are available to segment the thalamus from 1 mm T1 scans, but the image contrast is too faint to produce reliable segmentations. Some tools have attempted to refine these boundaries using diffusion MRI information, but do not generalise well across diffusion MRI acquisitions. Here we present the first CNN that can segment thalamic nuclei from T1 and diffusion data of any resolution without retraining or fine tuning. Our method builds on our histological atlas of the thalamic nuclei and silver standard segmentations on high-quality diffusion data obtained with our recent Bayesian adaptive segmentation tool. We combine these with an approximate degradation model for fast domain randomisation during training. Our CNN produces a segmentation at 0.7 mm isotropic resolution, irrespective of the resolution of the input. Moreover, it uses a parsimonious model of the diffusion signal (fractional anisotropy and principal eigenvector) that is compatible with virtually any set of directions and b-values, including huge amounts of legacy data. We show results of our proposed method on three heterogeneous datasets acquired on dozens of different scanners. The method is publicly available at freesurfer.net/fswiki/ThalamicNucleiDTI.
@inproceedings{tregidgo_domain_2022, title = {Domain-agnostic segmentation of thalamic nuclei from joint structural and diffusion {MRI}}, author = {Tregidgo, Henry and Soskic, Sonja and Olchanyi, Mark and Althonayan, Juri and Billot, Benjamin and Maffei, Chiara and Golland, Polina and Yendiki, Anastasia and Alexander, Daniel and Bocchetta, Martina and Rohrer, Jonathan and Iglesias, Juan Eugenio}, booktitle = {MICCAI}, year = {2023}, pages = {247--257}, }
Conf.
Equivariant and denoising CNNs to decouple intensity and spatial features for motion tracking in fetal brain MRI

Benjamin Billot, Daniel Moyer, Neerav Karani, and 4 more authors

In MIDL, 2023

Abs Bib HTML PDF

Equivariance in convolutional neural networks (CNN) has been a long-sought property, as it would ensure robustness to expected effects in the data. Convolutional filters are by nature translation-equivariant, and rotation-equivariant kernels were proposed recently. While these filters can be paired with learnable weights to form equivariant networks (E-CNN), we show here that such E-CNNs have a limited learning capacity, which makes them fragile against even slight changes in intensity distribution. This sensitivity to intensity changes presents a major challenge in medical imaging where many noise sources can randomly corrupt the data, even for consecutive scans of the same subject. Here, we propose a hybrid architecture that successively decouples intensity and spatial features: we first remove irrelevant noise in the data with a denoising CNN, and then use an E-CNN to extract robust spatial features. We demonstrate our method for motion tracking in fetal brain MRI, where it considerably outperforms standard CNNs and E-CNNs.
@inproceedings{billot_equivariant_2023, title = {Equivariant and denoising {CNNs} to decouple intensity and spatial features for motion tracking in fetal brain {MRI}}, author = {Billot, Benjamin and Moyer, Daniel and Karani, Neerav and Hoffmann, Malte and Abaci Turk, Esra and Grant, Ellen and Golland, Polina}, booktitle = {MIDL}, year = {2023}, }

2022

Journal
In vivo hypothalamic regional volumetry across the frontotemporal dementia spectrum

Noah Shapiro, Emily Todd, Benjamin Billot, and 5 more authors

NeuroImage, 2022

Abs Bib HTML PDF

Frontotemporal dementia (FTD) is a spectrum of diseases characterised by language, behavioural and motor symptoms. Among the different subcortical regions implicated in the FTD symptomatology, the hypothalamus regulates various bodily functions, including eating behaviours which are commonly present across the FTD spectrum. The pattern of specific hypothalamic involvement across the clinical, pathological, and genetic forms of FTD has yet to be fully investigated, and its possible associations with abnormal eating behaviours have yet to be fully explored.
@article{shapiro_in_vivo_2022, title = {In vivo hypothalamic regional volumetry across the frontotemporal dementia spectrum}, author = {Shapiro, Noah and Todd, Emily and Billot, Benjamin and Cash, David and Iglesias, Juan Eugenio and Jason, Warren and Rohrer, Jonathan and Bocchetta, Martina}, journal = {NeuroImage}, year = {2022}, volume = {35}, }
Journal
SynthMorph: Learning Contrast-Invariant Registration Without Acquired Images

Malte Hoffmann, Benjamin Billot, Douglas Greve, and 3 more authors

IEEE Transactions on Medical Imaging, 2022

Abs Bib HTML PDF

We introduce a strategy for learning image registration without acquired imaging data, producing powerful networks agnostic to contrast introduced by magnetic resonance imaging (MRI). While classical registration methods accurately estimate the spatial correspondence between images, they solve an optimization problem for every new image pair. Learning-based techniques are fast at test time but limited to registering images with contrasts and geometric content similar to those seen during training. We propose to remove this dependency on training data by leveraging a generative strategy for diverse synthetic label maps and images that exposes networks to a wide range of variability, forcing them to learn more invariant features. This approach results in powerful networks that accurately generalize to a broad array of MRI contrasts. We present extensive experiments with a focus on 3D neuroimaging, showing that this strategy enables robust and accurate registration of arbitrary MRI contrasts even if the target contrast is not seen by the networks during training. We demonstrate registration accuracy surpassing the state of the art both within and across contrasts, using a single model. Critically, training on arbitrary shapes synthesized from noise distributions results in competitive performance, removing the dependency on acquired data of any kind. Additionally, since anatomical label maps are often available for the anatomy of interest, we show that synthesizing images from these dramatically boosts performance, while still avoiding the need for real intensity images. Our code is available at doic https://w3id.org/synthmorph.
@article{hoffmann_synthmorph_2022, title = {SynthMorph: Learning Contrast-Invariant Registration Without Acquired Images}, author = {Hoffmann, Malte and Billot, Benjamin and Greve, Douglas and Iglesias, Juan Eugenio and Fischl, Bruce and Dalca, Adrian}, journal = {IEEE Transactions on Medical Imaging}, year = {2022}, volume = {41}, number = {3}, pages = {543--558}, }
Conf.
Robust segmentation of brain MRI in the wild with hierarchical CNNs and no retraining

Benjamin Billot, Magdamo Colin, Steven Arnold, and 2 more authors

In MICCAI, 2022

Abs Bib HTML PDF

Retrospective analysis of brain MRI scans acquired in the clinic has the potential to enable neuroimaging studies with sample sizes much larger than those found in research datasets. However, analysing such clinical images “in the wild” is challenging, since subjects are scanned with highly variable protocols (MR contrast, resolution, orientation, etc.). Nevertheless, recent advances in convolutional neural networks (CNNs) and domain randomisation for image segmentation, best represented by the publicly available method SynthSeg, may enable morphometry of clinical MRI at scale. In this work, we first evaluate SynthSeg on an uncurated, heterogeneous dataset of more than 10,000 scans acquired at Massachusetts General Hospital. We show that SynthSeg is generally robust, but frequently falters on scans with low signal-to-noise ratio or poor tissue contrast. Next, we propose SynthSeg, a novel method that greatly mitigates these problems using a hierarchy of conditional segmentation and denoising CNNs. We show that this method is considerably more robust than SynthSeg, while also outperforming cascaded networks and state-of-the-art segmentation denoising methods. Finally, we apply our approach to a proof-of-concept volumetric study of ageing, where it closely replicates atrophy patterns observed in research studies conducted on high-quality, 1 mm, T1-weighted scans. The code and trained model are publicly available at https://github.com/BBillot/SynthSeg.
@inproceedings{billot_robust_2022, title = {Robust segmentation of brain {MRI} in the wild with hierarchical {CNNs} and no retraining}, author = {Billot, Benjamin and Colin, Magdamo and Arnold, Steven and Das, Sudeshna and Iglesias, Juan Eugenio}, booktitle = {MICCAI}, year = {2022}, pages = {538--548}, }
Conf.
Super-resolution of portable low-field MRI in real scenarios: integration with denoising and domain adaptation

Sonia Laguna, Riana Schleicher, Benjamin Billot, and 7 more authors

In MIDL, 2022

Abs Bib HTML PDF

Portable low-field MRI has the potential to revolutionize neuroimaging, by enabling point-of-care imaging and affordable scanning in underserved areas. The lower resolution and signal-to-noise ratio of these scans preclude image analysis with existing tools. Super-resolution (SR) methods can overcome this limitation, but: (i) training with downsampled high-field scans fails to generalize; and (ii) training with paired low/high-field data is hard due to the lack of perfectly aligned images. Here, we present an architecture that combines denoising, SR and domain adaptation modules to tackle this problem. The denoising and SR components are pretrained in a supervised fashion with large amounts of existing high-resolution data, whereas unsupervised learning is used for domain adaptation and end-to-end finetuning. We present preliminary results on a dataset of 11 low-field scans. The results show that our method enables segmentation with existing tools, which yield ROI volumes that correlate strongly with those derived from high-field scans (ρ > 0.8).
@inproceedings{laguna_super_2022, title = {Super-resolution of portable low-field {MRI} in real scenarios: integration with denoising and domain adaptation}, author = {Laguna, Sonia and Schleicher, Riana and Billot, Benjamin and Schaefer, Pamela and McKaig, Brenna and Goldstein, Joshua and Sheth, Kevin and Rosen, Matthew and Kimberly, Taylor and Iglesias, Juan Eugenio}, booktitle = {MIDL}, year = {2022}, }

2021

Journal
A deep learning toolbox for automatic segmentation of subcortical limbic structures from MRI images

Douglas Greve, Benjamin Billot, Devani Cordero, and 6 more authors

NeuroImage, 2021

Abs Bib HTML PDF

A tool was developed to automatically segment several subcortical limbic structures (nucleus accumbens, basal forebrain, septal nuclei, hypothalamus without mammillary bodies, the mammillary bodies, and fornix) using only a T1-weighted MRI as input. This tool fills an unmet need as there are few, if any, publicly available tools to segment these clinically relevant structures. A U-Net with spatial, intensity, contrast, and noise augmentation was trained using 39 manually labeled MRI data sets. In general, the Dice scores, true positive rates, false discovery rates, and manual-automatic volume correlation were very good relative to comparable tools for other structures. A diverse data set of 698 subjects were segmented using the tool; evaluation of the resulting labelings showed that the tool failed in less than 1% of cases. Test-retest reliability of the tool was excellent. The automatically segmented volume of all structures except mammillary bodies showed effectiveness at detecting either clinical AD effects, age effects, or both. This tool will be publicly released with FreeSurfer (surfer.nmr.mgh.harvard.edu/fswiki/ScLimbic). Together with the other cortical and subcortical limbic segmentations, this tool will allow FreeSurfer to provide a comprehensive view of the limbic system in an automated way.
@article{greve_deep_2021, title = {A deep learning toolbox for automatic segmentation of subcortical limbic structures from MRI images}, author = {Greve, Douglas and Billot, Benjamin and Cordero, Devani and Hoopes, Andrew and Hoffmann, Malte and Dalca, Adrian and Fischl, Bruce and Iglesias, Juan Eugenio and Augustinack, Jean}, journal = {NeuroImage}, year = {2021}, volume = {244}, }
Journal
Joint super-resolution and synthesis of 1mm isotropic MP-RAGE volumes from clinical MRI exams with scans of different orientation, resolution and contrast

Juan Eugenio Iglesias, Benjamin Billot, Yaël Balbastre, and 7 more authors

NeuroImage, 2021

Abs Bib HTML PDF

Most existing algorithms for automatic 3D morphometry of human brain MRI scans are designed for data with near-isotropic voxels at approximately 1 mm resolution, and frequently have contrast constraints as well-typically requiring T1-weighted images (e.g., MP-RAGE scans). This limitation prevents the analysis of millions of MRI scans acquired with large inter-slice spacing in clinical settings every year. In turn, the inability to quantitatively analyze these scans hinders the adoption of quantitative neuro imaging in healthcare, and also precludes research studies that could attain huge sample sizes and hence greatly improve our understanding of the human brain. Recent advances in convolutional neural networks (CNNs) are producing outstanding results in super-resolution and contrast synthesis of MRI. However, these approaches are very sensitive to the specific combination of contrast, resolution and orientation of the input images, and thus do not generalize to diverse clinical acquisition protocols – even within sites. In this article, we present SynthSR, a method to train a CNN that receives one or more scans with spaced slices, acquired with different contrast, resolution and orientation, and produces an isotropic scan of canonical contrast (typically a 1 mm MP-RAGE). The presented method does not require any preprocessing, beyond rigid coregistration of the input scans. Crucially, SynthSR trains on synthetic input images generated from 3D segmentations, and can thus be used to train CNNs for any combination of contrasts, resolutions and orientations without high-resolution real images of the input contrasts. We test the images generated with SynthSR in an array of common downstream analyses, and show that they can be reliably used for subcortical segmentation and volumetry, image registration (e.g., for tensor-based morphometry), and, if some image quality requirements are met, even cortical thickness morphometry. The source code is publicly available at https://github.com/BBillot/SynthSR.
@article{iglesias_joint_2021, title = {Joint super-resolution and synthesis of 1mm isotropic MP-RAGE volumes from clinical MRI exams with scans of different orientation, resolution and contrast}, author = {Iglesias, Juan Eugenio and Billot, Benjamin and Balbastre, Yaël and Tabari, Azadeh and Conklin, John and {Gilberto González}, R. and Alexander, Daniel C. and Golland, Polina and Edlow, Brian L. and Fischl, Bruce}, journal = {NeuroImage}, year = {2021}, volume = {237}, }
Conf.
Joint segmentation of multiple sclerosis lesions and brain anatomy in MRI scans of any contrast and resolution with CNNs

Benjamin Billot, Stefano Cerri, Koen Van Leemput, and 2 more authors

In ISBI, 2021

Abs Bib HTML PDF

We present the first deep learning method to segment Multiple Sclerosis lesions and brain structures from MRI scans of any (possibly multimodal) contrast and resolution. Our method only requires segmentations to be trained (no images), as it leverages the generative model of Bayesian segmentation to generate synthetic scans with simulated lesions, which are then used to train a CNN. Our method can be retrained to segment at any resolution by adjusting the amount of synthesised partial volume. By construction, the synthetic scans are perfectly aligned with their labels, which enables training with noisy labels obtained with automatic methods. The training data are generated on the fly, and aggressive augmentation (including artefacts) is applied for improved generalisation. We demonstrate our method on two public datasets, comparing it with a state-of-the-art Bayesian approach implemented in FreeSurfer, and dataset specific CNNs trained on real data. The code is available at https://github.com/BBillot/SynthSeg.
@inproceedings{billot_joint_2021, title = {Joint segmentation of multiple sclerosis lesions and brain anatomy in {MRI} scans of any contrast and resolution with {CNNs}}, author = {Billot, Benjamin and Cerri, Stefano and Van Leemput, Koen and Dalca, Adrian and Iglesias, Juan Eugenio}, booktitle = {ISBI}, year = {2021}, pages = {1971--1974}, }
Conf.
Learning MRI contrast-agnostic registration

Malte Hoffmann, Benjamin Billot, Juan Eugenio Iglesias, and 2 more authors

In ISBI, 2021

Abs Bib HTML PDF

We introduce a strategy for learning image registration without acquired imaging data, producing powerful networks agnostic to magnetic resonance imaging (MRI) contrast. While classical methods accurately estimate the spatial correspondence between images, they solve an optimization problem for every new image pair. Learning methods are fast at test time but limited to images with contrasts and geometric content similar to those seen during training. We propose to remove this dependency using a generative strategy that exposes networks to a wide range of images synthesized from segmentations during training, forcing them to generalize across contrasts. We show that networks trained within this framework generalize to a broad array of unseen MRI contrasts and surpass classical state-of-the-art brain registration accuracy by up to 12.4 Dice points for a variety of tested contrast combinations. Critically, training on arbitrary shapes synthesized from noise distributions results in competitive performance, removing the dependency on acquired data of any kind. Additionally, since anatomical label maps are often available for the anatomy of interest, we show that synthesizing images from these dramatically boosts performance, while still avoiding the need for real intensity images during training.
@inproceedings{hoffmann_learning_2021, title = {Learning {MRI} contrast-agnostic registration}, author = {Hoffmann, Malte and Billot, Benjamin and Iglesias, Juan Eugenio and Fischl, Bruce and Dalca, Adrian}, booktitle = {ISBI}, year = {2021}, pages = {899-903}, }

2020

Journal
Automated segmentation of the hypothalamus and associated subunits in brain MRI

Benjamin Billot, Martina Bocchetta, Emily Todd, and 3 more authors

NeuroImage, 2020

Abs Bib HTML PDF

Despite the crucial role of the hypothalamus in the regulation of the human body, neuroimaging studies of this structure and its nuclei are scarce. Such scarcity partially stems from the lack of automated segmentation tools, since manual delineation suffers from scalability and reproducibility issues. Due to the small size of the hypothalamus and the lack of image contrast in its vicinity, automated segmentation is difficult and has been long neglected by widespread neuroimaging packages like FreeSurfer or FSL. Nonetheless, recent advances in deep machine learning are enabling us to tackle difficult segmentation problems with high accuracy. In this paper we present a fully automated tool based on a deep convolutional neural network, for the segmentation of the whole hypothalamus and its subregions from T1-weighted MRI scans. We use aggressive data augmentation in order to make the model robust to T1-weighted MR scans from a wide array of different sources, without any need for preprocessing. We rigorously assess the performance of the presented tool through extensive analyses, including: inter- and intra-rater variability experiments between human observers; comparison of our tool with manual segmentation; comparison with an automated method based on multi-atlas segmentation; assessment of robustness by quality control analysis of a larger, heterogeneous dataset (ADNI); and indirect evaluation with a volumetric study performed on ADNI. The presented model outperforms multi-atlas segmentation scores as well as inter-rater accuracy level, and approaches intra-rater precision. Our method does not require any preprocessing and runs in less than a second on a GPU, and approximately 10 seconds on a CPU. The source code as well as the trained model are publicly available at https://github.com/BBillot/hypothalamus_seg, and will also be distributed with FreeSurfer.
@article{billot_automated_2020, title = {Automated segmentation of the hypothalamus and associated subunits in brain {MRI}}, author = {Billot, Benjamin and Bocchetta, Martina and Todd, Emily and Dalca, Adrian and Rohrer, Jonathan and Iglesias, Juan Eugenio}, journal = {NeuroImage}, year = {2020}, volume = {223}, }
Conf.
Partial volume segmentation of brain MRI scans of any resolution and contrast

Benjamin Billot, Eleanor Robinson, Adrian Dalca, and 1 more author

In MICCAI, 2020

Abs Bib HTML PDF

Partial voluming (PV) is arguably the last crucial unsolved problem in Bayesian segmentation of brain MRI with probabilistic atlases. PV occurs when voxels contain multiple tissue classes, giving rise to image intensities that may not be representative of any one of the underlying classes. PV is particularly problematic for segmentation when there is a large resolution gap between the atlas and the test scan, e.g., when segmenting clinical scans with thick slices, or when using a high-resolution atlas. Forward models of PV are realistic and simple, as they amount to blurring and subsampling a high resolution (HR) volume into a lower resolution (LR) scan. Unfortunately, segmentation as Bayesian inference quickly becomes intractable when “inverting” this forward PV model, as it requires marginalizing over all possible anatomical configurations of the HR volume. In this work, we present PV-SynthSeg, a convolutional neural network (CNN) that tackles this problem by directly learning a mapping between (possibly multi-modal) LR scans and underlying HR segmentations. PV-SynthSeg simulates LR images from HR label maps with a generative model of PV, and can be trained to segment scans of any desired target contrast and resolution, even for previously unseen modalities where neither images nor segmentations are available at training. PV-SynthSeg does not require any preprocessing, and runs in seconds. We demonstrate the accuracy and flexibility of our method with extensive experiments on three datasets and 2,680 scans. The code is available at https://github.com/BBillot/SynthSeg.
@inproceedings{billot_partial_2020, title = {Partial volume segmentation of brain {MRI} scans of any resolution and contrast}, author = {Billot, Benjamin and Robinson, Eleanor and Dalca, Adrian and Iglesias, Juan Eugenio}, booktitle = {MICCAI}, year = {2020}, pages = {177--187}, }
Conf.
A learning strategy for contrast-agnostic MRI segmentation

Benjamin Billot, Douglas Greve, Koen Van Leemput, and 3 more authors

In MIDL, 2020

Abs Bib HTML PDF

We present a deep learning strategy for contrast-agnostic semantic segmentation of unpreprocessed brain MRI scans, without requiring additional training or fine-tuning for new modalities. Classical Bayesian methods address this segmentation problem with unsupervised intensity models, but require significant computational resources. In contrast, learning-based methods can be fast at test time, but are sensitive to the data available at training. Our proposed learning method, SynthSeg, leverages a set of training segmentations (no intensity images required) to generate synthetic scans of widely varying contrasts on the fly during training. These scans are produced using the generative model of the classical Bayesian segmentation framework, with randomly sampled parameters for appearance, deformation, noise, and bias field. Because each mini-batch has a different synthetic contrast, the final network is not biased towards any specific MRI contrast. We comprehensively evaluate our approach on four datasets comprising over 1,000 subjects and four MR contrasts. The results show that our approach successfully segments every contrast in the data, performing slightly better than classical Bayesian segmentation, and three orders of magnitude faster. Moreover, even within the same type of MRI contrast, our strategy generalizes significantly better across datasets, compared to training using real images. Finally, we find that synthesizing a broad range of contrasts, even if unrealistic, increases the generalization of the neural network. Our code and model are open source at: https://github.com/BBillot/SynthSeg.
@inproceedings{billot_learning_2020, title = {A learning strategy for contrast-agnostic {MRI} segmentation}, author = {Billot, Benjamin and Greve, Douglas and Leemput, Koen Van and Fischl, Bruce and Iglesias, Juan Eugenio and Dalca, Adrian}, booktitle = {MIDL}, year = {2020}, pages = {75--93}, }

2019

Conf.
Image Synthesis with a Convolutional Capsule Generative Adversarial Network

Cher Bass, Tianhong Dai, Benjamin Billot, and 5 more authors

In MIDL, 2019

Abs Bib HTML PDF

Machine learning for biomedical imaging often suffers from a lack of labelled training data. One solution is to use generative models to synthesise more data. To this end, we introduce CapsPix2Pix, which combines convolutional capsules with the \textttpix2pix framework, to synthesise images conditioned on class segmentation labels. We apply our approach to a new biomedical dataset of cortical axons imaged by two-photon microscopy, as a method of data augmentation for small datasets. We evaluate performance both qualitatively and quantitatively. Quantitative evaluation is performed by using image data generated by either CapsPix2Pix or \textttpix2pix to train a U-net on a segmentation task, then testing on real microscopy data. Our method quantitatively performs as well as \textttpix2pix, with an order of magnitude fewer parameters. Additionally, CapsPix2Pix is far more capable at synthesising images of different appearance, but the same underlying geometry. Finally, qualitative analysis of the features learned by CapsPix2Pix suggests that individual capsules capture diverse and often semantically meaningful groups of features, covering structures such as synapses, axons and noise.
@inproceedings{cher_image_2019, title = {Image Synthesis with a Convolutional Capsule Generative Adversarial Network}, author = {Bass, Cher and Dai, Tianhong and Billot, Benjamin and Arulkumaran, Kai and Creswell, Antonia and Clopath, Claudia and {De Paola}, Vincenzo and Bharath, Anil}, booktitle = {MIDL}, year = {2019}, pages = {39--62}, }
Conf.
Deep Reinforcement Learning for Subpixel Neural Tracking

Tianhong Dai, Magda Dubois, Kai Arulkumaran, and 7 more authors

In MIDL, 2019

Abs Bib HTML PDF

Automatically tracing elongated structures, such as axons and blood vessels, is a challenging problem in the field of biomedical imaging, but one with many downstream applications. Real, labelled data is sparse, and existing algorithms either lack robustness to different datasets, or otherwise require significant manual tuning. Here, we instead learn a tracking algorithm in a synthetic environment, and apply it to tracing axons. To do so, we formulate tracking as a reinforcement learning problem, and apply deep reinforcement learning techniques with a continuous action space to learn how to track at the subpixel level. We train our model on simple synthetic data and test it on mouse cortical two-photon microscopy images. Despite the domain gap, our model approaches the performance of a heavily engineered tracker from a standard analysis suite for neuronal microscopy. We show that fine-tuning on real data improves performance, allowing better transfer when real labelled data is available. Finally, we demonstrate that our model’s uncertainty measure—a feature lacking in hand-engineered trackers—corresponds with how well it tracks the structure.
@inproceedings{dai_deep_2019, title = {Deep Reinforcement Learning for Subpixel Neural Tracking}, author = {Dai, Tianhong and Dubois, Magda and Arulkumaran, Kai and Campbell, Jonathan and Bass, Cher and Billot, Benjamin and Uslu, Fatmatulzehra and de Paola, Vincenzo and Clopath, Claudia and Bharath, Anil}, booktitle = {MIDL}, year = {2019}, pages = {130--150}, }