Noise, Model Order and Reliability噪声、模型阶数与可靠性

11.1 Beyond the noiseless square model超越无噪声的方阵模型

The theory of the previous chapters assumes that $\bx = \bA\bs$ holds exactly and that the number of sources is known. Real recordings violate this in three ways that matter. Sensors add noise, so the data follow $\bx = \bA\bs + \bn$. The number of sources is unknown and usually much smaller than the number of sensors: a 64-channel EEG recording or an fMRI data set with hundreds of time points is reduced to a few dozen components, and someone has to decide how many. Finally, ICA algorithms are iterative and randomly initialized and their contrasts have several local optima, so two runs on the same data can return different components.

前面各章的理论假设 $\bx = \bA\bs$ 严格成立，而且源的个数已知。真实数据在三个要紧的方面违背这些假设。传感器会引入噪声，所以数据服从 $\bx = \bA\bs + \bn$。源的个数未知，而且通常远小于传感器个数：一份 64 导联的 EEG 记录，或一份有几百个时间点的 fMRI 数据，往往被降到几十个成分，而降到多少需要有人决定。最后，ICA 算法是迭代的、随机初始化的，其对比函数有多个局部最优，所以在同一份数据上运行两次，可能得到不同的成分。

This chapter takes up the three questions in turn: what noise does to ICA (Section 11.2第 11.2 节), how to choose the number of components and what happens when it is wrong (Section 11.3第 11.3 节 and Section 11.4第 11.4 节), and how to judge whether an estimated component can be trusted (Section 11.5第 11.5 节 and Section 11.6第 11.6 节).

本章依次讨论这三个问题：噪声对 ICA 有什么影响（Section 11.2第 11.2 节）；如何选择成分个数，以及选错了会怎样（Section 11.3第 11.3 节与Section 11.4第 11.4 节）；如何判断一个估计出的成分是否可信（Section 11.5第 11.5 节与Section 11.6第 11.6 节）。

11.2 Noisy ICA含噪 ICA

The noisy model is

含噪模型为

with $\bA \in \R^{m \times n}$ of full column rank ($m \ge n$), sources as in Assumption 1.1假设 1.1 with unit variance, and noise independent of the sources. Isotropic noise is the standard assumption; noise with a known covariance $\bSigma_{\bn}$ is reduced to it by replacing $\bx$ with $\bSigma_{\bn}^{-1/2}\bx$. The covariance of the data is $\bC_{\bx} = \bA\bA^\top + \sigma^2\bI$. If $d_1 \ge \dots \ge d_n > 0$ are the nonzero eigenvalues of $\bA\bA^\top$, the eigenvalues of $\bC_{\bx}$ are

其中 $\bA \in \R^{m \times n}$ 列满秩（$m \ge n$），源满足Assumption 1.1假设 1.1且方差为一，噪声与源独立。各向同性噪声是标准假设；若噪声协方差 $\bSigma_{\bn}$ 已知，用 $\bSigma_{\bn}^{-1/2}\bx$ 代替 $\bx$ 即可化为这种情形。数据的协方差为 $\bC_{\bx} = \bA\bA^\top + \sigma^2\bI$。若 $d_1 \ge \dots \ge d_n > 0$ 是 $\bA\bA^\top$ 的非零特征值，则 $\bC_{\bx}$ 的特征值为

and the $n$ leading eigenvectors span the column space of $\bA$. This is why PCA reduction (Section 3.6第 3.6 节) keeps the signal, and it is the starting point of order selection. Four consequences of noise follow.

并且前 $n$ 个特征向量张成 $\bA$ 的列空间。这就是 PCA 降维（Section 3.6第 3.6 节）能保留信号的原因，也是选择阶数的出发点。噪声带来以下四个后果。

Whitening is biased白化有偏

After ordinary whitening, an algorithm that searches over rotations therefore cannot, in general, reach a separating matrix. A rotation can turn $\bV\bA$ into a scaled permutation matrix only if the columns of $\bV\bA$ are orthogonal; since $(\bV\bA)^\top(\bV\bA) = (\bA^\top\bA + \sigma^2\bI)^{-1}\bA^\top\bA$, this happens exactly when the columns of $\bA$ are orthogonal. The distortion is of relative size $\sigma^2/\lambda_k$ and is largest along the weakest signal direction. Bias correction requires $\sigma^2$, which can be estimated from the discarded eigenvalues when $m > n$ (see probabilistic PCA below). Unlike the noiseless case, equivariance is also lost: the outputs $\bW\bx = \bP\bs + \bW\bn$ contain a noise term that depends on $\bW$ and hence on $\bA$.

因此，在普通白化之后，在旋转中搜索的算法一般无法到达分离矩阵。只有当 $\bV\bA$ 的各列相互正交时，旋转才能把它变成带尺度的置换矩阵；而 $(\bV\bA)^\top(\bV\bA) = (\bA^\top\bA + \sigma^2\bI)^{-1}\bA^\top\bA$，所以这恰好发生在 $\bA$ 的各列相互正交的时候。畸变的相对大小为 $\sigma^2/\lambda_k$，沿最弱的信号方向最大。偏差校正需要知道 $\sigma^2$；当 $m > n$ 时，可以由被舍弃的特征值估计它（见下文的概率 PCA）。与无噪声情形不同，等变性也不再成立：输出 $\bW\bx = \bP\bs + \bW\bn$ 含有依赖于 $\bW$、从而依赖于 $\bA$ 的噪声项。

Higher-order cumulants ignore Gaussian noise高阶累积量对高斯噪声“视而不见”

Cumulants are additive for independent summands and vanish beyond order two for Gaussian variables (Proposition 2.8命题 2.8). Hence the fourth-order cumulants of noisy data are those of the noiseless mixture,

独立随机变量之和的累积量等于各自累积量之和，而高斯变量二阶以上的累积量全为零（Proposition 2.8命题 2.8）。因此含噪数据的四阶累积量与无噪声混合的完全相同：

After bias-corrected whitening $\bz = \bV_{\mathrm c}\bx$, a unit vector $\bw$ gives $\kurt(\bw^\top\bz) = \sum_p q_p^4\kappa_p$ with $\bq = (\bV_{\mathrm c}\bA)^\top\bw$ a unit vector, exactly as without noise; so the source directions remain the maxima of $|\kurt|$ on the sphere (Theorem 5.3定理 5.3), and cumulant methods such as JADE remain consistent once the whitening is corrected. Contrasts built from other nonlinearities, such as $\log\cosh$ or a likelihood, are biased by noise. Hyvärinen (1999) introduced “Gaussian moments”, expectations of Gaussian functions of the output whose noise bias can be removed in closed form, together with a modified FastICA.

经偏差校正的白化 $\bz = \bV_{\mathrm c}\bx$ 之后，对单位向量 $\bw$ 有 $\kurt(\bw^\top\bz) = \sum_p q_p^4\kappa_p$，其中 $\bq = (\bV_{\mathrm c}\bA)^\top\bw$ 也是单位向量，与无噪声时完全一样；所以源方向仍然是 $|\kurt|$ 在球面上的极大点（Theorem 5.3定理 5.3），白化一经校正，JADE 这类累积量方法仍然相合。基于其他非线性函数（如 $\log\cosh$ 或似然）的对比函数则会被噪声带偏。Hyvärinen（1999）提出了“高斯矩”，即输出的高斯函数的期望，其噪声偏差可以用闭式去除，并给出了相应修改的 FastICA。

The sources cannot be recovered exactly源无法精确恢复

Even with $\bA$ known, noise cannot be removed. For $m = n$ the unmixed outputs $\bA^{-1}\bx = \bs + \bA^{-1}\bn$ carry noise with covariance $\sigma^2(\bA^\top\bA)^{-1}$, which is large when $\bA$ is ill-conditioned. The linear estimator with the smallest mean squared error is $\hat\bs = \bA^\top(\bA\bA^\top + \sigma^2\bI)^{-1}\bx$, which accepts a bias (shrinkage toward zero) in exchange for less noise; nonlinear estimators that use the non-Gaussian source densities do better still (Hyvärinen, Karhunen & Oja, 2001). Noisy ICA thus splits into two tasks: estimating $\bA$, which can be done consistently, and estimating $\bs$, which can only be done approximately.

即使 $\bA$ 已知，噪声也无法去除。$m = n$ 时，解混输出 $\bA^{-1}\bx = \bs + \bA^{-1}\bn$ 所含噪声的协方差为 $\sigma^2(\bA^\top\bA)^{-1}$，$\bA$ 病态时它很大。均方误差最小的线性估计量是 $\hat\bs = \bA^\top(\bA\bA^\top + \sigma^2\bI)^{-1}\bx$，它以一定的偏差（向零收缩）换取较小的噪声；利用非高斯源密度的非线性估计量还能做得更好（Hyvärinen, Karhunen & Oja, 2001）。因此含噪 ICA 分为两个任务：估计 $\bA$，这可以相合地做到；估计 $\bs$，这只能近似地做到。

Probabilistic PCA概率 PCA

Probabilistic PCA (Tipping & Bishop, 1999) is the model $\bx = \bF\mathbf{f} + \bn$ with $k$ Gaussian factors $\mathbf{f} \sim \Ncal(\bzero, \bI)$ and $\bn \sim \Ncal(\bzero, \sigma^2\bI)$. Let $\ell_1 \ge \dots \ge \ell_m$ be the eigenvalues of the sample covariance, $\bU_k$ the corresponding $k$ leading eigenvectors and $\mathbf{L}_k = \diag(\ell_1, \dots, \ell_k)$. The maximum-likelihood estimates are

概率 PCA（Tipping & Bishop, 1999）是模型 $\bx = \bF\mathbf{f} + \bn$，其中 $k$ 个因子 $\mathbf{f} \sim \Ncal(\bzero, \bI)$ 为高斯分布，$\bn \sim \Ncal(\bzero, \sigma^2\bI)$。设 $\ell_1 \ge \dots \ge \ell_m$ 为样本协方差的特征值，$\bU_k$ 为相应的前 $k$ 个特征向量，$\mathbf{L}_k = \diag(\ell_1, \dots, \ell_k)$。最大似然估计为

where $\bR$ is an arbitrary orthogonal matrix. The noise variance is the average of the discarded eigenvalues, and the factor loadings are determined only up to the rotation $\bR$, because Gaussian factors can be rotated freely. That rotation is exactly what ICA determines when the factors are non-Gaussian. A noisy-ICA pipeline therefore estimates the signal subspace and $\sigma^2$ by probabilistic PCA, whitens with the bias-corrected matrix $(\mathbf{L}_k - \hat\sigma^2\bI)^{-1/2}\bU_k^\top$, and finds the rotation by ICA; probabilistic ICA for fMRI (Beckmann & Smith, 2004) follows this route (Chapter 14第 14 章).

其中 $\bR$ 是任意正交矩阵。噪声方差就是被舍弃特征值的平均；因子载荷只能确定到旋转 $\bR$，因为高斯因子可以自由旋转。而当因子非高斯时，这个旋转恰好由 ICA 确定。因此含噪 ICA 的流程是：用概率 PCA 估计信号子空间和 $\sigma^2$，用经偏差校正的矩阵 $(\mathbf{L}_k - \hat\sigma^2\bI)^{-1/2}\bU_k^\top$ 白化，再用 ICA 求出旋转。fMRI 中的概率 ICA（Beckmann & Smith, 2004）走的就是这条路（Chapter 14第 14 章）。

11.3 Choosing the number of components选择成分个数

When $m > n$, ICA is run on the $k$-dimensional principal subspace, and $k$, the model order, must be chosen. In the population, (11.2) shows $n$ eigenvalues above a flat floor $\sigma^2$. In a sample the noise eigenvalues spread around $\sigma^2$, the more so the larger $m/T$, and a weak signal eigenvalue can disappear among them. A principled choice compares, for each candidate $k$, how well the “$k$ signals plus white noise” covariance fits the data against how many parameters it uses.

当 $m > n$ 时，ICA 在 $k$ 维主子空间上运行，而 $k$，即模型阶数，必须选定。在总体层面，(11.2) 显示有 $n$ 个特征值高出平坦的噪声底 $\sigma^2$。在样本中，噪声特征值会在 $\sigma^2$ 周围散开，$m/T$ 越大散得越开，较弱的信号特征值可能淹没其中。有原则的选择方法是：对每个候选的 $k$，权衡“$k$ 个信号加白噪声”这一协方差模型对数据的拟合程度与它所用的参数个数。

The maximum-likelihood fit of that model is the probabilistic PCA solution, and its maximized Gaussian log-likelihood depends on the eigenvalues only through the ratio of the geometric to the arithmetic mean of the $m - k$ smallest ones (Exercise 11.5习题 11.5):

这个模型的最大似然拟合就是概率 PCA 的解，其最大化后的高斯对数似然只通过最小的 $m - k$ 个特征值的几何平均与算术平均之比依赖于特征值（Exercise 11.5习题 11.5）：

Here $L_k$ is the maximized likelihood of the model with $k$ signals and $L_m$ that of an unrestricted covariance, which does not depend on $k$. The logarithm on the right is at most zero and vanishes only when the $m - k$ smallest eigenvalues are equal. The model has $p(k) = mk - k(k-1)/2 + 1$ free parameters ($k$ signal eigenvalues, $\sigma^2$, and $k$ orthonormal eigenvectors in $\R^m$). The order is chosen by minimizing

其中 $L_k$ 是含 $k$ 个信号的模型的最大似然值，$L_m$ 是协方差不受限制时的最大似然值，后者与 $k$ 无关。右边的对数不大于零，且仅当最小的 $m - k$ 个特征值全部相等时为零。该模型有 $p(k) = mk - k(k-1)/2 + 1$ 个自由参数（$k$ 个信号特征值、$\sigma^2$，以及 $\R^m$ 中的 $k$ 个标准正交特征向量）。阶数通过最小化下列准则来选择：

the criteria of Akaike (1974) and of Rissanen (1978), written relative to $L_m$, which shifts them by a constant and does not change the minimizer; MDL coincides with half the Bayesian information criterion of Schwarz (1978). Wax and Kailath (1985) derived these eigenvalue forms for complex-valued array data, where the likelihood term carries $T$ instead of $T/2$ and the parameter count is $k(2m - k)$; the forms above are the real-valued analogues. Because its penalty grows with $\log T$, MDL is consistent for i.i.d. Gaussian data, whereas AIC tends to overestimate the order (Wax & Kailath, 1985). Minka (2000) proposed a Bayesian alternative, a Laplace approximation to the evidence of the probabilistic PCA model, which outperformed cross-validation and other methods in his simulations; it is used in probabilistic ICA (Beckmann & Smith, 2004).

这分别是 Akaike（1974）和 Rissanen（1978）的准则，这里写成相对于 $L_m$ 的形式，只差一个常数，不改变最小点；MDL 恰为 Schwarz（1978）的贝叶斯信息准则的一半。Wax 和 Kailath（1985）针对复值阵列数据推导了这些基于特征值的形式，那里似然项中是 $T$ 而不是 $T/2$，参数个数为 $k(2m - k)$；上面的形式是相应的实值版本。由于 MDL 的惩罚项随 $\log T$ 增长，它对独立同分布的高斯数据是相合的，而 AIC 往往高估阶数（Wax & Kailath, 1985）。Minka（2000）提出了一个贝叶斯替代方案：用 Laplace 近似计算概率 PCA 模型的证据（evidence），在他的模拟中优于交叉验证和其他方法；概率 ICA 采用了这一方法（Beckmann & Smith, 2004）。

Dependent samples相依样本

All of these criteria treat the $T$ samples as independent. Temporally or spatially correlated samples carry less information than their number suggests: the sample covariance fluctuates more, the noise eigenvalues spread more, and the criteria read the spread as signal and overestimate the order. Li, Adalı and Calhoun (2007) proposed to subsample the data so that the retained samples are effectively independent, choosing the subsampling depth from an entropy-rate measure of the dependence, and to apply the criteria to the subsampled data. In the GIFT toolbox, which implements this method, the eigenvalues of the subsampled data are further divided by the corresponding quantiles of the Marčenko–Pastur law (Section 3.6第 3.6 节) for the effective sample size, and the order is taken at the first local minimum of MDL rather than at its global minimum. The script below shows the effect with $n = 8$ Laplace sources in $m = 30$ sensors, where sources and noise are either white or first-order autoregressive with coefficient $\phi = 0.9$.

上述准则都把 $T$ 个样本当作相互独立。时间上或空间上相关的样本所含的信息比其个数所暗示的要少：样本协方差波动更大，噪声特征值散得更开，而准则会把这种散开误读为信号，从而高估阶数。Li、Adalı 和 Calhoun（2007）提出对数据降采样，使保留下来的样本近似独立（降采样的间隔由衡量相依程度的熵率指标确定），再对降采样后的数据应用这些准则。在实现了这一方法的 GIFT 工具箱中，降采样数据的特征值还要再除以按有效样本量计算的 Marčenko–Pastur 分布（Section 3.6第 3.6 节）的相应分位数，阶数取 MDL 的第一个局部极小点，而不是全局极小点。下面的脚本用 $m = 30$ 个传感器中的 $n = 8$ 个拉普拉斯源演示这一效应，源和噪声或者是白的，或者是系数 $\phi = 0.9$ 的一阶自回归过程。

import numpy as np

def order_criteria(X):
    """Orders chosen by AIC and MDL from the sample covariance eigenvalues."""
    m, T = X.shape
    ell = np.sort(np.linalg.eigvalsh(np.cov(X)))[::-1]
    aic, mdl = [], []
    for k in range(m):
        tail = ell[k:]                      # the m - k smallest eigenvalues
        loglr = (m - k) * (np.mean(np.log(tail)) - np.log(np.mean(tail)))
        p = m * k - k * (k - 1) / 2 + 1     # free parameters of the model
        aic.append(-T * loglr + 2 * p)
        mdl.append(-T / 2 * loglr + p / 2 * np.log(T))
    return int(np.argmin(aic)), int(np.argmin(mdl))

def ar1(E, phi):
    """Turn each row of white noise E into a unit-variance AR(1) series."""
    Y = np.empty_like(E)
    Y[:, 0] = E[:, 0]
    for t in range(1, E.shape[1]):
        Y[:, t] = phi * Y[:, t - 1] + np.sqrt(1 - phi ** 2) * E[:, t]
    return Y

rng = np.random.default_rng(1)
m, n, T = 30, 8, 5000
A = rng.normal(size=(m, n))
S = rng.laplace(0, 1 / np.sqrt(2), (n, T))
N = rng.normal(size=(m, T))
for phi in [0.0, 0.9]:
    X = A @ ar1(S, phi) + ar1(N, phi)
    print(f"phi = {phi}: all samples (AIC, MDL) = {order_criteria(X)}, "
          f"every 10th sample = {order_criteria(X[:, ::10])}")

The script prints

脚本输出

phi = 0.0: all samples (AIC, MDL) = (9, 8), every 10th sample = (8, 8)
phi = 0.9: all samples (AIC, MDL) = (29, 18), every 10th sample = (9, 8)

With independent samples MDL finds the eight sources and AIC overestimates by one. With dependent samples both fail, AIC choosing the largest admissible order. Keeping every tenth sample restores the behaviour seen with independent samples (MDL 8, AIC 9), although it discards 90% of the data; ten is close to the factor $(1 + \phi^2)/(1 - \phi^2) \approx 9.5$ by which the dependence inflates the variance of a sample covariance between two such AR(1) series. Figure 11.1图 11.1 shows why: the noise eigenvalues of the dependent data form a tilted floor rather than a flat one, and MDL reaches its minimum far beyond $k = 8$.

样本独立时，MDL 找到了全部八个源，AIC 多估了一个。样本相依时两者都失败了，AIC 甚至选了允许的最大阶数。每十个样本保留一个，尽管丢掉了 90% 的数据，却恢复了独立样本时的结果（MDL 为 8，AIC 为 9）；10 接近于这种相依性使两个 AR(1) 序列之间样本协方差的方差放大的倍数 $(1 + \phi^2)/(1 - \phi^2) \approx 9.5$。Figure 11.1图 11.1说明了原因：相依数据的噪声特征值构成倾斜的而不是平坦的“底”，MDL 的最小值远在 $k = 8$ 之后。

11.4 Too few or too many components: merged and split components成分过少或过多：合并与分裂

A wrong order has characteristic consequences. If $k < n$, PCA keeps the $k$ directions of largest variance and discards directions that still contain signal; the sources that live partly in the discarded directions cannot be separated and appear as merged components, mixtures of two or more sources. The typical victims are sources with nearly collinear mixing columns. Their sum direction has large variance and their difference direction small variance, proportional to $1 - \cos\theta$ for columns of equal norm at angle $\theta$ (Exercise 11.3习题 11.3), so the difference is the first thing to be discarded. In fMRI terms, two regions with similar time courses are found as one network.

阶数选错会带来典型的后果。若 $k < n$，PCA 只保留方差最大的 $k$ 个方向，丢掉了仍含信号的方向；部分落在被丢弃方向上的那些源无法分开，表现为合并的成分，即两个或更多源的混合。最典型的受害者是混合列几乎共线的源：它们的和方向方差大，差方向方差小，对夹角为 $\theta$、范数相等的两列，后者与 $1 - \cos\theta$ 成正比（Exercise 11.3习题 11.3），所以差方向最先被丢掉。用 fMRI 的话说，时间过程相近的两个脑区会被当成一个网络。

If $k > n$, the extra dimensions contain mostly noise. With Gaussian noise and enough samples, ICA leaves the sources intact and returns $k - n$ nearly Gaussian components, which are harmless once recognized. Two problems remain. First, components that are merged at a lower order may split at a higher one, and whether the parts are distinct sources or fragments of one process is a question about the data that ICA cannot answer. Second, when $k$ is large relative to $T$, ICA overfits: with too few samples per dimension it returns components concentrated on a single spike or bump, whose apparent non-Gaussianity is created by the fit itself (Särelä & Vigário, 2003). Overfitting is a reason to reduce the dimension before ICA even when the data contain many genuine sources. In practice the cost of overestimation is not negligible: on fMRI data, Li, Adalı and Calhoun (2007) found that at overestimated orders the estimated components become less stable and task-related activations are estimated less accurately.

若 $k > n$，多出来的维度主要是噪声。在高斯噪声且样本充足时，ICA 不会破坏源，只是多返回 $k - n$ 个近似高斯的成分，认出来就无害。但仍有两个问题。其一，低阶时被合并的成分在高阶时可能分裂，而分出来的部分是不同的源还是同一过程的碎片，是关于数据的问题，ICA 回答不了。其二，当 $k$ 相对于 $T$ 很大时，ICA 会过拟合：每个维度的样本太少，它会返回集中在单个尖峰或鼓包上的成分，其表面上的非高斯性是拟合本身造出来的（Särelä & Vigário, 2003）。即使数据中确实有很多源，这也是在 ICA 之前先降维的一个理由。实践中高估阶数的代价并不可忽略：Li、Adalı 和 Calhoun（2007）在 fMRI 数据上发现，阶数高估时估计出的成分变得不那么稳定，任务相关激活的估计也变差。

There is therefore no single correct order for a real data set: the order sets the scale at which the data are described. A sound practice is to run ICA at several orders, track how components merge and split between them (for example by correlating components across orders), and report the order and how it was chosen. Chapter 14第 14 章 discusses the consequences for brain networks.

因此，对一份真实数据并不存在唯一正确的阶数：阶数决定了描述数据的尺度。稳妥的做法是在几个阶数下分别运行 ICA，追踪成分在阶数之间如何合并与分裂（例如计算不同阶数下成分之间的相关），并报告所用的阶数及其选择方法。Chapter 14第 14 章讨论这对脑网络研究意味着什么。

11.5 Reliability across runs: ICASSO and RAICAR多次运行的可靠性：ICASSO 与 RAICAR

An ICA estimate depends on more than the model. The starting point decides which local optimum the algorithm reaches, and the finite sample perturbs every optimum (Chapter 10第 10 章). A component that appears in every run is more trustworthy than one that appears once. ICASSO (Himberg, Hyvärinen & Esposito, 2004) turns this observation into a procedure.

ICA 的估计结果不只取决于模型。起点决定了算法到达哪个局部最优，有限样本又会扰动每个最优点（Chapter 10第 10 章）。每次运行都出现的成分，比只出现一次的成分更可信。ICASSO（Himberg, Hyvärinen & Esposito, 2004）把这一观察变成了一套流程。

The quality index of a cluster $C$ with complement $C^c$ is the average similarity inside the cluster minus the average similarity to the estimates outside it:

簇 $C$（其补集为 $C^c$）的质量指标，是簇内的平均相似度减去它与簇外估计的平均相似度：

ICASSO also projects the estimates to two dimensions for visual inspection. RAICAR (Yang, LaConte, Weng & Hu, 2008) follows the same idea with a different matching step: it runs ICA many times, aligns the components across runs from the matrix of absolute cross-correlations between runs, ranks the aligned components by a reproducibility index computed from the absolute correlations among their aligned realizations, and averages the aligned estimates. Both methods reuse the same data. Split-half reproducibility asks a stronger question: split the data into two halves (two groups of subjects, two sessions, or the two halves of a long recording), run ICA on each, match the components between the halves by maximizing the total absolute correlation over permutations, and report the matched correlations. A component that reappears in independent data is closer to a scientific replication than one that reappears in reruns.

ICASSO 还把这些估计投影到二维平面上，便于目视检查。RAICAR（Yang, LaConte, Weng & Hu, 2008）遵循同样的思路，只是配对步骤不同：它多次运行 ICA，利用各次运行之间的相关系数绝对值矩阵把成分对齐，按可重复性指标给对齐后的成分排序（该指标由同一成分各次实现之间相关系数的绝对值算出），再对对齐后的估计取平均。这两种方法用的都是同一份数据。分半可重复性问的是更强的问题：把数据分成两半（两组被试、两次扫描，或长记录的前后两半），分别做 ICA，通过在所有置换中最大化总的相关绝对值来配对两半的成分，然后报告配对后的相关系数。在独立数据中重新出现的成分，比在重复运行中重新出现的成分，更接近科学意义上的重复验证。

The spread of a component across bootstrap runs is an empirical version of the sampling variability studied in Chapter 10第 10 章. It is large for nearly Gaussian sources, whose asymptotic variance is at least $1/(\mathcal{J} - 1)$ for one-unit estimators (Proposition 10.9命题 10.9), and it grows as $T$ decreases. A low stability index can therefore be predicted, in part, from the source distributions and the sample size.

一个成分在各次自助重抽样运行之间的离散程度，是Chapter 10第 10 章所研究的抽样变异性的经验版本。对接近高斯的源它很大，因为单单元估计量的渐近方差至少为 $1/(\mathcal{J} - 1)$（Proposition 10.9命题 10.9）；样本量 $T$ 越小，它也越大。所以较低的稳定性指标，在一定程度上可以由源的分布和样本量预先估计出来。

11.6 What a stability index does and does not certify稳定性指标能说明什么、不能说明什么

A high stability index certifies that the component is a reproducible optimum of the chosen contrast for these data: it is not an accident of one initialization and, when resampling is used, not an accident of a few samples. It does not certify much more.

较高的稳定性指标说明：对这份数据而言，这个成分是所选对比函数的一个可重复的最优点；它不是某次初始化的偶然结果，若用了重抽样，也不是少数几个样本造成的偶然结果。除此之外，它说明不了更多。

• Stable is not unbiased. A biased estimator is stable and wrong. Noise-biased whitening (Proposition 11.1命题 11.1) produces reproducible but distorted components, and an order that is too low produces perfectly stable merged components (Example 11.2例 11.2).
• Stable is not meaningful. In fMRI, head motion and physiological artifacts are among the most stable components of all; stability says nothing about their origin.
• Stable is not independent. ICA returns maximally independent components whether or not the data were generated by independent sources, and a dependent structure can be reproduced as reliably as an independent one.
• Unstable is not always bad. If two sources are nearly Gaussian, or more generally if the contrast is nearly flat within a subspace, each run returns a different rotation inside a subspace that is itself stable (Section 4.4第 4.4 节). The individual components then have low stability while the subspace they span is reliable; comparing subspaces across runs, rather than single components, reveals this (Exercise 11.4习题 11.4).

• 稳定不等于无偏。有偏的估计量可以既稳定又错误。受噪声影响的白化（Proposition 11.1命题 11.1）给出可重复但有畸变的成分，阶数过低则给出完全稳定的合并成分（Example 11.2例 11.2）。
• 稳定不等于有意义。在 fMRI 中，头动和生理伪迹恰恰是最稳定的成分之一；稳定性对它们的来源一无所知。
• 稳定不等于独立。无论数据是否由独立源生成，ICA 都会返回“尽可能独立”的成分；相依的结构也可以和独立的结构一样被稳定地重现。
• 不稳定未必是坏事。如果两个源接近高斯，或者更一般地，对比函数在某个子空间内几乎是平的，那么每次运行都会在这个本身稳定的子空间内给出不同的旋转（Section 4.4第 4.4 节）。此时单个成分的稳定性很低，而它们张成的子空间是可靠的；在各次运行之间比较子空间而不是单个成分，就能看出这一点（Exercise 11.4习题 11.4）。

Stability is thus necessary for a component to be interpreted, but not sufficient. Validity has to come from elsewhere: agreement with the noise and order diagnostics of this chapter, replication in independent data, and knowledge of the domain.

因此，稳定是对成分作解释的必要条件，而不是充分条件。有效性必须另有来源：与本章的噪声诊断和阶数诊断相互印证、在独立数据中得到重复，以及领域知识。

11.7 Exercises习题