Cookies on this website

We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings.

BACKGROUND: In recent years, analyses of event related potentials/fields have moved from the selection of a few components and peaks to a mass-univariate approach in which the whole data space is analyzed. Such extensive testing increases the number of false positives and correction for multiple comparisons is needed. METHOD: Here we review all cluster-based correction for multiple comparison methods (cluster-height, cluster-size, cluster-mass, and threshold free cluster enhancement - TFCE), in conjunction with two computational approaches (permutation and bootstrap). RESULTS: Data driven Monte-Carlo simulations comparing two conditions within subjects (two sample Student's t-test) showed that, on average, all cluster-based methods using permutation or bootstrap alike control well the family-wise error rate (FWER), with a few caveats. CONCLUSIONS: (i) A minimum of 800 iterations are necessary to obtain stable results; (ii) below 50 trials, bootstrap methods are too conservative; (iii) for low critical family-wise error rates (e.g. p=1%), permutations can be too liberal; (iv) TFCE controls best the type 1 error rate with an attenuated extent parameter (i.e. power<1).

Original publication




Journal article


J Neurosci Methods

Publication Date





85 - 93


Cluster-based statistics, ERP, Family-wise error rate, Monte-Carlo simulations, Multiple comparison correction, Threshold free cluster enhancement, Brain, Cluster Analysis, Computer Simulation, Datasets as Topic, Electroencephalography, Evoked Potentials, Humans, Monte Carlo Method, Signal Processing, Computer-Assisted, Software