Current Environment:

Research | Overview

Research overview

Algorithm development: We develop and release medical image analysis and machine/deep learning algorithms and software tools, for image segmentation, registration, classification, prediction, visualization, integration with non-image data, and information comprehension.

Computational neuroscience: We compile lifespan, big-data brain MRI and non-MRI elements to quantitatively characterize normal brain development in structure and function across the lifespan. We emphasize on spatiotemporal granularity, sex differences and hemispheric differences, all toward more precision neuroscience and precision medicine.

Translational/clinical research: We study brain abnormalities, for abnormality detection, patient phenotyping, outcome prediction, treatment evaluation, as well as imaging and non-imaging diagnostic and prognostic biomarkers. The abnormalities we study span from subtle (e.g., malnutrition), mild/moderate (e.g., hypoxic ischemia, atrophy, aging), to severe (e.g., brain tumor) conditions, and include relatively common (psychiatry) as well as rare diseases (e.g., Sturge-Weber).

Research topics

Abnormality detection

Background: Deep learning has been widely used to detect lesions in brain MRIs. However, a major challenge is the systematic low accuracy of deep learning in detecting small diffuse lesions, where the Dice overlap between expert- and computer-detected lesion regions is typically 0.3-0.7, compared to the high accuracy of deep learning in detecting big focal lesions, where the Dice overlap is usually 0.8-0.95 [Zhang2021a].

Brain age biomarker

Background: Machine learning can predict an individual’s “brain age” from the brain MRI. Differences of ML-predicted and actual chronological ages show the accelerated or delayed aging, and can be associated with diseases, lifestyle, socioeconomics, genetic, and other environment influences.

Algorithm: We recently developed deep learning algorithms to predict children’s brain ages [He2020], and lifespan ages in 0-100 years [He2021]. The lifespan age predictor is based on (a) explicitly splitting a T1-weighted MRI into contrast and morphometry channels [Ou2011; Ou2014]; (b) feature-level multi-channel fusion, with (c) a proposed multi-channel fusion-with-attention convolutional neural network (FiA-Net); and (d) powered by >16,000 brain MRIs acquired during 0-100 years [Pereira2021].

Accuracy: The figure below shows that each component in our algorithm played their expected role and reduced the prediction errors.

Overall, our FiA-Net achieved promising accuracy compared to the state-of-the-art multi-channel fusion convolutional neural networks -- lower mean absolute error (MAE) and higher correlation between predicted and actual chronological ages.

The above compared different algorithms on the same dataset. Below is the comparison on different recent studies each reporting their best performance in the data they chose. Our work is unique in that we covered 0-100 years of age (long purple bar), and the error is promising compared to other studies with similar age ranges (red dots).



Applications: We are applying this algorithm and software to study how different diseases and environmental factors deviate human brain from normal aging.


Brain atlases

Background: A brain MRI atlases is a virtual brain image that is computed/constructed to represent the average anatomy and the average image intensity of all individuals in a cohort. Atlases are fundamental tools in computational neuroscience — they are targets for image registration, guidelines for image segmentation, references for abnormality detection, and sources to enable temporal brain development studies.

Algorithms: We developed unbiased DRAMMS group-wise registration algorithms for atlas construction [Ou2011, Ou2014, Ou2017]. It generates the average brain anatomy, average voxel-wise intensity, and the standard deviation voxel-wise intensity from a cohort of MRIs. The atlas is unbiased toward any individual’s anatomy.

0-6 years ADC Atlases: We computed normal atlases for apparent diffusion coefficient (ADC) maps in 10 age groups densely sampling 0-6 years of age: 0-2 weeks, 2 weeks-3 months, 3-6 months, 6-9 months, 9-12 months, 1-2 years, 2-3 years, 3-4 years, 4-5 years, and 5-6 years [Ou2017]. It is the first set of normal brain ADC atlases for early childhood at this spatiotemporal resolution.

Atlas-Quantified 0-6-Year Brain Development: Age-specific atlases allow the study of how an average brain changes over time while repeatedly scanning actual volunteers is difficult or even impossible (frequently in infant stages, or over the lifespan). The figure below show the logic — even each individual contributes only a cross-sectional (single-visit) brain MRI, the atlases allow us to study brain development across time.

By this logic, we quantified the spatiotemporal brain ADC development patterns in four aspects: (1) trajectory of ADC change; (2) rate of change; (3) maturation age of ADC values; and (4) hemispheric asymmetry of mature ages. The figure below shows the maturation age (in years) of ADC values, which are related to myelination, at the voxel level. Visually, a central-to-perepheral, posterior-to-anterior growth pattern can be appreciated. Quantitatively, we identified hemispheric differences in ADC development and maturation in infant brain [Sotardi2021]. The work received Editorial Highlight in the Radiology journal in 2021.

0-100 years Lifespan Atlases: Our going work is constructing atlases in 10 coarse and 36 fine-scale age groups for 0-100 years of age.


Image registration

Background: Image registration is to put two or more images into the same coordinate system with anatomic correspondences. It is the basis for studying across-subject commonality or differences, for studying longitudinal image changes (e.g., disease progression or normal development), for aligning multi-modal images and fusing their complementary information, and for abnormality detection (e.g., comparing abnormal with normal).

Algorithms: We have developed the Deformable Registration via Attribute Matching and Mutual-Saliency Weighting (DRAMMS) deformable (non-rigid) medical image analysis. Its core concepts are two-fold: (a) in places where we can find correspondences, we characterize each voxel with rich texture information (multi-scale and multi-orientation texture features), so we can find correspondences more accurately; (b) in places where correspondences are missing (e.g., when registering histology cut to MRI, or when registering a normal atlas to a tumor-bearing image), the algorithm should automatically sense the missing correspondences, and automatically assign different confidence to different image regions, and let the registration be driven by regions where reliable correspondences can be established.

Validations: The publicly available DRAMMS algorithm and software has been extensively validated in different organs (brain, cardiac, prostate, breast images), in normal and disease, in different registration settings (longitudinal, across-subject, multi-modality; 2D and 3D), and in different ages (fetal, neonatal, and up to 100 years). Results were quantitatively compared to 14 other state-of-the-art image registration algorithms, for accuracy, generality, and robustness.

Lifespan big-data MRI

Background: Big data is reshaping many aspects of healthcare. Brain MRI studies with over 1,000 participants are often considered a relatively big sample size. Recent Nature, Science, PNAS, and other studies used 10,000 or more publicly-shared brain MRIs, or 20,000-50,000 brain MRIs if combining public and private data. Two trends are clear: (a) while many studies used private data, public data is ideal for transparency and replication; (b) lifespan coverage is important, especially to include participants in both extremities of ages in the 0-100 years range.

Our ongoing work: We compiled >95,000 brain MRIs from >71,000 healthy participants from public domains, covering 0-100 years of age [Pereira2021]. Using the first patch, we have derived early-childhood brain atlases [Ou2017, Sotardi2021], and built brain age predictors [He2020, He2021]. The age distribution is unbalanced but covers 0-100 years (figure below). Comprehensive non-MRI and MRI information exists for subsets of the data:

  1. MRI and other neuroimaging (PET, EEG, etc.)
  2. neurocognitive tests
  3. psychiatric/behavior data
  4. demographic and clinical tests/reports
  5. substance exposure
  6. genetics
  7. biospecimen
  8. environment and socioeconomic status (SES)
  9. targeted disease populations
  10. laboratory data

Neuroscientific/Clinical Opportunities: Big-data, publicly-available, typically-developing, across-the-lifespan brain MRI and non-MRI data opens up tremendous opportunities. Examples include to study lifespan brain development, to early screen abnormalities as deviation from normal, to build age predictors, to quantify genotype-phenotype associations, to understand environment, lifestyle, socioeconomic factors for brain health, to create and maintain an up-to-date knowledgebase, and more. Many opportunities were previously underpowered or even less practical. Public data also increases transparency and replication.

Technical Opportunities: Tremendous opportunities call for technical advancement. Open-portal cloud computing is a key direction.