To embrace big-data neuroimaging, harmonizing the site effect in resting-state functional magnetic resonance imaging (R-fMRI) data fusion is a fundamental challenge. A comprehensive evaluation of potentially effective harmonization strategies, particularly with specifically collected data, has been scarce, especially for R-fMRI metrics. Here, we comprehensively assess harmonization strategies from multiple perspectives, including tests on residual site effect, individual identification, test-retest reliability, and replicability of group-level statistical results, on widely used R-fMRI metrics across various datasets, including data obtained from participants with repetitive measures at different scanners. For individual identifiability (i.e., whether the same subject could be identified across R-fMRI data scanned across different sites), we found that, while most methods decreased site effects, the Subsampling Maximum-mean-distance based distribution shift correction Algorithm (SMA) and parametric unadjusted CovBat outperformed linear regression models, linear mixed models, ComBat series and invariant conditional variational auto-encoder in clustering accuracy. Test-retest reliability was better for SMA and parametric adjusted CovBat than unadjusted ComBat series and parametric unadjusted CovBat in the number of overlapped voxels. At the same time, SMA was superior to the latter in replicability in terms of the Dice coefficient and the scale of brain areas showing sex differences reproducibly observed across datasets. Furthermore, SMA better detected reproducible sex differences of ALFF under the site-sex confounded situation. Moreover, we designed experiments to identify the best target site features to optimize SMA identifiability, test-retest reliability, and stability. We noted both sample size and distribution of the target site matter and introduced a heuristic formula for selecting the target site. In addition to providing practical guidelines, this work can inform continuing improvements and innovations in harmonizing methodologies for big R-fMRI data.

SCI