Part II · Chapter 4第二部分 · 第 4 章

Identifiability可辨识性

The Darmois–Skitovich theorem and Comon's theorem: with at most one Gaussian source, the mixing matrix is determined up to scaling and permutation.

Darmois–Skitovich 定理与 Comon 定理：至多一个高斯源时，混合矩阵在尺度与置换意义下唯一。

4.1 What identifiability means可辨识性的含义

Chapter 1 showed that the order and the scale of the sources can never be recovered (Proposition 1.3命题 1.3), and Chapter 3第 3 章 showed that second-order statistics determine the mixing matrix only up to a rotation. This chapter answers the question that these two facts raise: is anything else lost? Suppose we knew the distribution of $\bx$ exactly, as if we had infinitely many noiseless samples. Would it determine $\bA$ up to the unavoidable order and scale? The answer is the central theorem of ICA: yes, provided at most one source is Gaussian. Without that proviso the answer is no, and we will see exactly what is lost.

第 1 章说明，源的顺序与尺度永远无法恢复（Proposition 1.3命题 1.3）；Chapter 3第 3 章说明，二阶统计量只能把混合矩阵确定到一个旋转。这两个事实自然引出本章要回答的问题：除此之外还会丢失什么？假设我们确切知道 $\bx$ 的分布，好比拥有无穷多个无噪声样本，它能否在顺序与尺度这两种不可避免的不确定性之外唯一确定 $\bA$ ？答案正是 ICA 的核心定理：只要至多有一个源是高斯的，就能。去掉这个条件，答案就是否定的，我们还会看清到底丢失了什么。

Identifiability is a property of the model, not of an algorithm. It asks whether two different parameter values can produce the same distribution of the observations. If they can, no amount of data and no estimator can tell them apart. If they cannot, the remaining tasks are to build contrast functions whose optima are the true parameters (Chapters 5 and 6), algorithms that find those optima (Part III), and a theory of how accurately this can be done from $T$ samples (Chapter 10第 10 章).

可辨识性是模型的性质，而不是算法的性质。它问的是：两个不同的参数值能否产生同一个观测分布？如果能，那么无论数据有多少、用什么估计量，都无法把它们区分开。如果不能，剩下的任务才是：构造以真实参数为最优点的对比函数（第 5、6 章），设计找到这些最优点的算法（第三部分），以及建立从 $T$ 个样本出发能估计到多准的理论（Chapter 10第 10 章）。

Definition 4.1 (identifiability)定义 4.1（可辨识性）

Consider models $\bx = \bA\bs$ in which $\bA \in \R^{m \times n}$ has full column rank and $\bs$ has mutually independent, non-degenerate components (a random variable is degenerate if it is almost surely constant). The mixing matrix is identifiable in a class of such models if, whenever two members $(\bA, \bs)$ and $(\bB, \bt)$ of the class produce observations with the same distribution, $\bA\bs \overset{d}{=} \bB\bt$ , there are a permutation matrix $\bPi$ and an invertible diagonal matrix $\bD$ with $\bB = \bA\bPi\bD$ .

考虑这样一类模型 $\bx = \bA\bs$ ： $\bA \in \R^{m \times n}$ 列满秩， $\bs$ 的各分量相互独立且非退化（随机变量退化是指它几乎必然为常数）。如果该类中任意两个成员 $(\bA, \bs)$ 与 $(\bB, \bt)$ 只要产生同分布的观测，即 $\bA\bs \overset{d}{=} \bB\bt$ ，就必然存在置换矩阵 $\bPi$ 与可逆对角矩阵 $\bD$ ，使 $\bB = \bA\bPi\bD$ ，则称混合矩阵在这一类模型中可辨识。

The condition $\bB = \bA\bPi\bD$ says that each column of $\bB$ is a nonzero multiple of a different column of $\bA$ . Since $\bD\bPi = \bPi(\bPi^\top\bD\bPi)$ and $\bPi^\top\bD\bPi$ is again diagonal, it does not matter on which side of the permutation the diagonal factor stands; we call any such product a scaled permutation matrix. Degenerate sources are excluded because a source that is identically zero could be attached to any column.

条件 $\bB = \bA\bPi\bD$ 的意思是： $\bB$ 的每一列都是 $\bA$ 中某一列（各不相同）的非零倍数。由于 $\bD\bPi = \bPi(\bPi^\top\bD\bPi)$ ，而 $\bPi^\top\bD\bPi$ 仍是对角矩阵，对角因子写在置换矩阵的哪一侧无关紧要；这样的乘积统称为带尺度的置换矩阵。之所以排除退化的源，是因为一个恒为零的源可以配上任意一列。

Two facts frame the problem. First, if all sources are Gaussian, identifiability fails completely: for $\bs \sim \Ncal(\bzero, \bI)$ and every orthogonal $\bQ$ , the pair $(\bA\bQ^\top, \bQ\bs)$ produces the same observations as $(\bA, \bs)$ (Section 3.5第 3.5 节). Second, for unit-variance sources the covariance $\bC_{\bx} = \bA\bA^\top$ determines $\bA$ up to a right orthogonal factor: if $\bB\bB^\top = \bA\bA^\top$ then $\bB = \bA\bQ$ with $\bQ$ orthogonal. The question is therefore whether the non-Gaussian structure of the sources forces $\bQ$ to be a signed permutation. The key tool is a classical characterization of the Gaussian law.

有两个事实界定了问题的范围。其一，如果所有源都是高斯的，可辨识性完全失效：对 $\bs \sim \Ncal(\bzero, \bI)$ 与任意正交矩阵 $\bQ$ ， $(\bA\bQ^\top, \bQ\bs)$ 与 $(\bA, \bs)$ 产生相同的观测（Section 3.5第 3.5 节）。其二，对单位方差的源，协方差 $\bC_{\bx} = \bA\bA^\top$ 把 $\bA$ 确定到右乘一个正交因子：若 $\bB\bB^\top = \bA\bA^\top$ ，则 $\bB = \bA\bQ$ ，其中 $\bQ$ 为正交矩阵。于是问题归结为：源的非高斯结构能否迫使 $\bQ$ 成为带符号的置换矩阵？关键工具是高斯分布的一个经典刻画。

4.2 The Darmois–Skitovich theoremDarmois–Skitovich 定理

When can two linear combinations of the same independent variables be independent of each other? Trivially, when they share no variable. For Gaussian variables sharing is allowed: if $s_1, s_2$ are independent $\Ncal(0, 1)$ , then $s_1 + s_2$ and $s_1 - s_2$ are jointly Gaussian and uncorrelated, hence independent. The theorem of Darmois (1953) and Skitovich (1953) says that the Gaussian is the only law with this property.

同一组独立变量的两个线性组合，什么时候彼此独立？不共享任何变量时显然独立。高斯变量则允许共享：若 $s_1, s_2$ 是独立的 $\Ncal(0, 1)$ 变量，则 $s_1 + s_2$ 与 $s_1 - s_2$ 联合高斯且不相关，因而独立。Darmois（1953）与 Skitovich（1953）的定理指出，具有这种性质的只有高斯分布。

Theorem 4.2 (Darmois–Skitovich)定理 4.2（Darmois–Skitovich）

Let $s_1, \dots, s_n$ be mutually independent random variables and $a_1, \dots, a_n$ , $b_1, \dots, b_n$ real numbers. If the linear forms $L_1 = \sum_{i=1}^n a_i s_i$ and $L_2 = \sum_{i=1}^n b_i s_i$ are independent, then every $s_i$ with $a_i b_i \neq 0$ is Gaussian.

设 $s_1, \dots, s_n$ 是相互独立的随机变量， $a_1, \dots, a_n$ 与 $b_1, \dots, b_n$ 是实数。若线性形式 $L_1 = \sum_{i=1}^n a_i s_i$ 与 $L_2 = \sum_{i=1}^n b_i s_i$ 相互独立，则凡满足 $a_i b_i \neq 0$ 的 $s_i$ 都是高斯的。

Here and below a constant counts as a degenerate Gaussian. The theorem restricts only the variables that appear in both forms. With $s_1$ Laplacian, $s_2$ uniform and $g_1, g_2$ independent standard Gaussians, the forms $L_1 = s_1 + g_1 + g_2$ and $L_2 = s_2 + g_1 - g_2$ are independent: $g_1 + g_2$ and $g_1 - g_2$ are independent Gaussians, and each form adds to one of them a variable that the other form does not contain. The shared variables $g_1$ and $g_2$ are Gaussian, as the theorem demands. Uncorrelatedness, by contrast, is cheap: for independent $s_1, s_2$ with unit variance and kurtosis $\kappa$ , the forms $y_1 = s_1 + s_2$ and $y_2 = s_1 - s_2$ are always uncorrelated, but their cross-cumulant $\cum(y_1, y_1, y_2, y_2)$ equals $2\kappa$ (Exercise 4.1习题 4.1), which is $-2.4$ for uniform and $6$ for Laplacian sources.

这里以及下文都把常数视为退化的高斯变量。定理只约束同时出现在两个线性形式中的变量。设 $s_1$ 服从拉普拉斯分布， $s_2$ 服从均匀分布， $g_1, g_2$ 是独立的标准高斯变量，则 $L_1 = s_1 + g_1 + g_2$ 与 $L_2 = s_2 + g_1 - g_2$ 相互独立： $g_1 + g_2$ 与 $g_1 - g_2$ 是独立的高斯变量，而两个形式各自在其中一个上加了一个另一形式不含的变量。被共享的 $g_1$ 与 $g_2$ 正是高斯的，与定理一致。相比之下，不相关是很廉价的性质：对方差为一、峭度为 $\kappa$ 的独立变量 $s_1, s_2$ ， $y_1 = s_1 + s_2$ 与 $y_2 = s_1 - s_2$ 总是不相关，但它们的交叉累积量 $\cum(y_1, y_1, y_2, y_2)$ 等于 $2\kappa$ （Exercise 4.1习题 4.1），对均匀源为 $-2.4$ ，对拉普拉斯源为 $6$ 。

Two facts about the Gaussian law关于高斯分布的两个事实

The proof rests on two classical results whose proofs use the theory of entire functions; we state them without proof.

证明依赖两个经典结果，它们的证明要用到整函数理论，这里只陈述、不证明。

Fact 4.3 (Marcinkiewicz, 1939)事实 4.3（Marcinkiewicz，1939）

If the characteristic function $\varphi_u(t) = \E[e^{\mathrm{i}tu}]$ of a random variable $u$ , where $\mathrm{i} = \sqrt{-1}$ , has the form $\varphi_u(t) = e^{P(t)}$ for all real $t$ , with $P$ a polynomial, then $P$ has degree at most two, so $u$ is Gaussian.

若随机变量 $u$ 的特征函数 $\varphi_u(t) = \E[e^{\mathrm{i}tu}]$ （ $\mathrm{i} = \sqrt{-1}$ ）对一切实数 $t$ 都具有 $\varphi_u(t) = e^{P(t)}$ 的形式，其中 $P$ 是多项式，则 $P$ 的次数不超过二，因而 $u$ 是高斯的。

Fact 4.4 (Cramér, 1936)事实 4.4（Cramér，1936）

If $u$ and $v$ are independent and $u + v$ is Gaussian, then $u$ and $v$ are both Gaussian. By induction, if a finite sum of independent variables is Gaussian, so is every summand.

若 $u$ 与 $v$ 独立且 $u + v$ 是高斯的，则 $u$ 与 $v$ 都是高斯的。由归纳法，若有限个独立变量之和是高斯的，则每个加项都是高斯的。

Recall from Chapter 2第 2 章 that when a variable $u$ has finite moments $m_r = \E[u^r]$ of every order, its cumulants $\kappa_r(u)$ are defined by the identity of formal power series $\sum_{r \ge 0} m_r z^r / r! = \exp\bigl(\sum_{r \ge 1} \kappa_r(u) z^r / r!\bigr)$ , that is, by the Taylor expansion of the log-characteristic function at the origin. The next lemma shows that a variable whose cumulants vanish beyond some order is Gaussian. It needs no assumption that the law is determined by its moments: the vanishing of the cumulants forces this by itself.

回忆Chapter 2第 2 章：若变量 $u$ 的各阶矩 $m_r = \E[u^r]$ 都有限，则其累积量 $\kappa_r(u)$ 由形式幂级数恒等式 $\sum_{r \ge 0} m_r z^r / r! = \exp\bigl(\sum_{r \ge 1} \kappa_r(u) z^r / r!\bigr)$ 定义，也就是对数特征函数在原点的泰勒展开。下面的引理说明，累积量从某阶起全部为零的变量必是高斯的。它不需要“分布由矩唯一确定”这一假设：累积量的消失本身就保证了这一点。

Lemma 4.5 (finitely many nonzero cumulants)引理 4.5（只有有限个非零累积量）

Let $u$ have finite moments of all orders, and suppose that $\kappa_r(u) = 0$ for every $r > K$ . Then $u$ is Gaussian.

设 $u$ 的各阶矩都有限，且对一切 $r > K$ 有 $\kappa_r(u) = 0$ ，则 $u$ 是高斯的。

Proof.证明

Let $P(z) = \sum_{r=1}^{K} \kappa_r(u) z^r / r!$ . By assumption, $\sum_r m_r z^r / r!$ is the Taylor series of the entire function $e^{P(z)}$ , so it converges absolutely for every complex $z$ , and so does its derivative $\sum_r m_{r+1} z^r / r!$ . For real $x > 0$ we have $\E|u|^r = m_r$ for even $r$ and $\E|u|^r \le 1 + m_{r+1}$ for odd $r$ (because $|v|^r \le 1 + v^{r+1}$ when $r + 1$ is even). Hence $\E\, e^{x|u|} = \sum_r x^r \E|u|^r / r! < \infty$ for every $x$ . This domination lets us expand the characteristic function term by term: $\varphi_u(t) = \E\, e^{\mathrm{i}tu} = \sum_r m_r (\mathrm{i}t)^r / r! = e^{P(\mathrm{i}t)}$ for every real $t$ . Since $t \mapsto P(\mathrm{i}t)$ is a polynomial, Fact 4.3事实 4.3 shows that it has degree at most two, so $u$ is Gaussian.

令 $P(z) = \sum_{r=1}^{K} \kappa_r(u) z^r / r!$ 。由假设， $\sum_r m_r z^r / r!$ 正是整函数 $e^{P(z)}$ 的泰勒级数，因此对任意复数 $z$ 绝对收敛，其导数级数 $\sum_r m_{r+1} z^r / r!$ 也是如此。对实数 $x > 0$ ， $r$ 为偶数时 $\E|u|^r = m_r$ ， $r$ 为奇数时 $\E|u|^r \le 1 + m_{r+1}$ （因为 $r + 1$ 为偶数时 $|v|^r \le 1 + v^{r+1}$ ）。于是对任意 $x$ 都有 $\E\, e^{x|u|} = \sum_r x^r \E|u|^r / r! < \infty$ 。有了这个控制，特征函数就可以逐项展开：对一切实数 $t$ ， $\varphi_u(t) = \E\, e^{\mathrm{i}tu} = \sum_r m_r (\mathrm{i}t)^r / r! = e^{P(\mathrm{i}t)}$ 。由于 $t \mapsto P(\mathrm{i}t)$ 是多项式，Fact 4.3事实 4.3表明其次数不超过二，故 $u$ 是高斯的。

Proof when all moments are finite各阶矩有限时的证明

We prove Theorem 4.2定理 4.2 under the additional assumption that every $s_i$ has finite moments of all orders. The general case, which needs no moments at all, requires a more delicate analysis of characteristic functions; see Kagan, Linnik & Rao (1973).

我们在附加假设“每个 $s_i$ 的各阶矩都有限”之下证明Theorem 4.2定理 4.2。完全不需要矩条件的一般情形，要对特征函数做更精细的分析，见 Kagan, Linnik & Rao（1973）。

Proof (all moments finite).证明（各阶矩有限的情形）

Step 1: cross-cumulants vanish. Let $\Phi_i(t) = \log \E\, e^{\mathrm{i}ts_i}$ , defined near $t = 0$ ; the upright $\mathrm{i}$ is the imaginary unit and the italic $i$ an index. Because all moments are finite, $\Phi_i$ is infinitely differentiable there, with $\Phi_i^{(r)}(0) = \mathrm{i}^r \kappa_r(s_i)$ . By independence of the $s_i$ , the log-characteristic function of the pair $(L_1, L_2)$ is $\sum_i \Phi_i(a_i t_1 + b_i t_2)$ . Independence of $L_1$ and $L_2$ means that this function splits as $\sum_i \Phi_i(a_i t_1) + \sum_i \Phi_i(b_i t_2)$ near the origin. Differentiate $p \ge 1$ times in $t_1$ and $q \ge 1$ times in $t_2$ and set $t_1 = t_2 = 0$ : the split side gives zero, and the other side gives $\mathrm{i}^{p+q} \sum_i a_i^p b_i^q \kappa_{p+q}(s_i)$ . Hence

第一步：交叉累积量为零。令 $\Phi_i(t) = \log \E\, e^{\mathrm{i}ts_i}$ ，它在 $t = 0$ 附近有定义；正体 $\mathrm{i}$ 表示虚数单位，斜体 $i$ 表示下标。由于各阶矩有限， $\Phi_i$ 在该处无穷次可微，且 $\Phi_i^{(r)}(0) = \mathrm{i}^r \kappa_r(s_i)$ 。由 $s_i$ 的独立性， $(L_1, L_2)$ 的对数特征函数为 $\sum_i \Phi_i(a_i t_1 + b_i t_2)$ 。 $L_1$ 与 $L_2$ 独立，意味着该函数在原点附近可以拆成 $\sum_i \Phi_i(a_i t_1) + \sum_i \Phi_i(b_i t_2)$ 。对 $t_1$ 求 $p \ge 1$ 阶、对 $t_2$ 求 $q \ge 1$ 阶导数，再令 $t_1 = t_2 = 0$ ：拆开的一侧给出零，另一侧给出 $\mathrm{i}^{p+q} \sum_i a_i^p b_i^q \kappa_{p+q}(s_i)$ 。因此

\sum_{i=1}^{n} a_i^{p}\, b_i^{q}\, \kappa_{p+q}(s_i) = 0 \qquad \text{for all } p, q \ge 1.\tag{4.1}

The left side is the cross-cumulant $\cum(L_1, \dots, L_1, L_2, \dots, L_2)$ with $p$ copies of $L_1$ and $q$ copies of $L_2$ , expanded by multilinearity; cross-cumulants of independent variables vanish (Proposition 2.8命题 2.8).

左边正是含 $p$ 个 $L_1$ 、 $q$ 个 $L_2$ 的交叉累积量 $\cum(L_1, \dots, L_1, L_2, \dots, L_2)$ 按多重线性展开的结果；独立变量的交叉累积量为零（Proposition 2.8命题 2.8）。

Step 2: a Vandermonde argument. Let $S = \{i : a_i b_i \neq 0\}$ ; the terms with $i \notin S$ vanish in (4.1). For $i \in S$ put $c_i = b_i / a_i \neq 0$ , let $c^{(1)}, \dots, c^{(K)}$ be the distinct values among the $c_i$ , and group the indices accordingly: $G_k = \{i \in S : c_i = c^{(k)}\}$ . Let $u_k = \sum_{i \in G_k} a_i s_i$ . Cumulants are additive over independent summands and satisfy $\kappa_r(a s) = a^r \kappa_r(s)$ , so $\kappa_r(u_k) = \sum_{i \in G_k} a_i^r \kappa_r(s_i)$ . Fix an order $r \ge 2$ and write $a_i^{r-q} b_i^{q} = a_i^r c_i^{q}$ . Then (4.1) with $p = r - q$ becomes

第二步：Vandermonde 论证。令 $S = \{i : a_i b_i \neq 0\}$ ；在 (4.1) 中 $i \notin S$ 的项均为零。对 $i \in S$ 令 $c_i = b_i / a_i \neq 0$ ，记 $c^{(1)}, \dots, c^{(K)}$ 为这些 $c_i$ 中互不相同的值，并据此把下标分组： $G_k = \{i \in S : c_i = c^{(k)}\}$ 。令 $u_k = \sum_{i \in G_k} a_i s_i$ 。累积量对独立加项可加，且满足 $\kappa_r(a s) = a^r \kappa_r(s)$ ，所以 $\kappa_r(u_k) = \sum_{i \in G_k} a_i^r \kappa_r(s_i)$ 。固定阶数 $r \ge 2$ ，把 $a_i^{r-q} b_i^{q}$ 写成 $a_i^r c_i^{q}$ 。于是取 $p = r - q$ 时，(4.1) 变成

\sum_{k=1}^{K} \bigl(c^{(k)}\bigr)^{q}\, \kappa_r(u_k) = 0, \qquad q = 1, \dots, r-1.

When $r \ge K + 1$ , the equations with $q = 1, \dots, K$ form a square linear system in the unknowns $\kappa_r(u_1), \dots, \kappa_r(u_K)$ . Its matrix $\bigl[(c^{(k)})^q\bigr]_{q,k}$ is a Vandermonde matrix in the distinct numbers $c^{(k)}$ multiplied on the right by $\diag(c^{(1)}, \dots, c^{(K)})$ , so it is invertible. Therefore $\kappa_r(u_k) = 0$ for every $k$ and every $r \ge K + 1$ .

当 $r \ge K + 1$ 时，取 $q = 1, \dots, K$ 的那些方程构成关于未知量 $\kappa_r(u_1), \dots, \kappa_r(u_K)$ 的方阵线性方程组。其系数矩阵 $\bigl[(c^{(k)})^q\bigr]_{q,k}$ 等于由互异数 $c^{(k)}$ 构成的 Vandermonde 矩阵右乘 $\diag(c^{(1)}, \dots, c^{(K)})$ ，因而可逆。所以对每个 $k$ 和每个 $r \ge K + 1$ 都有 $\kappa_r(u_k) = 0$ 。

Step 3: conclusion. Each $u_k$ has finite moments of all orders, so by Lemma 4.5引理 4.5 it is Gaussian. It is a sum of the independent variables $a_i s_i$ , $i \in G_k$ , so by Fact 4.4事实 4.4 each $a_i s_i$ is Gaussian, and so is $s_i$ because $a_i \neq 0$ . Every $i \in S$ lies in some $G_k$ .

第三步：结论。每个 $u_k$ 的各阶矩都有限，由Lemma 4.5引理 4.5知它是高斯的。 $u_k$ 是独立变量 $a_i s_i$ （ $i \in G_k$ ）之和，由Fact 4.4事实 4.4，每个 $a_i s_i$ 都是高斯的；又因 $a_i \neq 0$ ， $s_i$ 也是高斯的。而每个 $i \in S$ 都属于某个 $G_k$ 。

The two classical ingredients play different roles. The Vandermonde step needs no property of the Gaussian at all; it shows that independence kills all but finitely many cumulants of each group $u_k$ . Marcinkiewicz's theorem turns "finitely many cumulants" into "Gaussian", and Cramér's theorem is needed only when several shared variables enter both forms in the same proportion, so that the cumulant equations see only their sum.

两个经典结果各司其职。Vandermonde 一步完全没有用到高斯分布的性质，它说明独立性使每组 $u_k$ 只剩下有限个非零累积量。Marcinkiewicz 定理把“只有有限个累积量”变成“是高斯的”；Cramér 定理只在若干共享变量以相同比例进入两个线性形式时才需要，因为此时累积量方程只能看到它们的和。

4.3 Comon's theoremComon 定理

Comon (1994) observed that the Darmois–Skitovich theorem settles the identifiability of ICA. The argument works directly with the global matrix $\bP = \bW\bA$ , which maps the sources to the outputs, $\by = \bP\bs$ .

Comon（1994）注意到，Darmois–Skitovich 定理解决了 ICA 的可辨识性问题。论证直接针对把源映射到输出的全局矩阵 $\bP = \bW\bA$ 进行，即 $\by = \bP\bs$ 。

Theorem 4.6 (Comon)定理 4.6（Comon）

Let $\bs = (s_1, \dots, s_n)^\top$ have mutually independent, non-degenerate components with finite variances, at most one of which is Gaussian, and let $\bP \in \R^{n \times n}$ be invertible. If the components of $\by = \bP\bs$ are pairwise independent, then $\bP = \bPi\bD$ for some permutation matrix $\bPi$ and invertible diagonal matrix $\bD$ . In particular, the components of $\by$ are then mutually independent.

设 $\bs = (s_1, \dots, s_n)^\top$ 的各分量相互独立、非退化且方差有限，其中至多一个是高斯的；设 $\bP \in \R^{n \times n}$ 可逆。若 $\by = \bP\bs$ 的各分量两两独立，则存在置换矩阵 $\bPi$ 与可逆对角矩阵 $\bD$ ，使 $\bP = \bPi\bD$ 。特别地，此时 $\by$ 的各分量相互独立。

Proof.证明

Take two rows $i \neq j$ . The outputs $y_i = \sum_k P_{ik} s_k$ and $y_j = \sum_k P_{jk} s_k$ are independent, so by Theorem 4.2定理 4.2 every $s_k$ with $P_{ik} P_{jk} \neq 0$ is Gaussian. Hence each column $k$ that belongs to a non-Gaussian source has at most one nonzero entry, and at least one because $\bP$ is invertible. These columns are multiples of standard basis vectors and are linearly independent, so their nonzero entries lie in distinct rows. If no source is Gaussian, this already says $\bP = \bPi\bD$ .

任取两行 $i \neq j$ 。输出 $y_i = \sum_k P_{ik} s_k$ 与 $y_j = \sum_k P_{jk} s_k$ 独立，由Theorem 4.2定理 4.2，凡满足 $P_{ik} P_{jk} \neq 0$ 的 $s_k$ 都是高斯的。因此，对应非高斯源的每一列 $k$ 至多有一个非零元；又因 $\bP$ 可逆，至少有一个。这些列都是标准基向量的倍数，且线性无关，所以它们的非零元位于不同的行。若没有高斯源，这已经说明 $\bP = \bPi\bD$ 。

Otherwise let $s_g$ be the Gaussian source. The $n - 1$ non-Gaussian columns occupy $n - 1$ distinct rows; let $r$ be the remaining row. Row $r$ is zero outside column $g$ , so $y_r = P_{rg} s_g$ , and $P_{rg} \neq 0$ because $\bP$ is invertible. Any other row $i$ has the form $y_i = P_{il} s_l + P_{ig} s_g$ , where $l$ is the non-Gaussian column whose nonzero entry sits in row $i$ . Independent variables with finite variance are uncorrelated, so $0 = \Cov(y_i, y_r) = P_{ig} P_{rg} \Var(s_g)$ , which forces $P_{ig} = 0$ . Thus column $g$ also has a single nonzero entry, and $\bP = \bPi\bD$ . Finally, $\bPi\bD\bs$ has mutually independent components (Proposition 1.3命题 1.3).

否则，设 $s_g$ 是那个高斯源。 $n - 1$ 个非高斯列占据了 $n - 1$ 个不同的行，记剩下的一行为 $r$ 。第 $r$ 行在第 $g$ 列以外全为零，所以 $y_r = P_{rg} s_g$ ；又因 $\bP$ 可逆， $P_{rg} \neq 0$ 。其余任一行 $i$ 都形如 $y_i = P_{il} s_l + P_{ig} s_g$ ，其中 $l$ 是非零元恰在第 $i$ 行的那个非高斯列。方差有限的独立变量互不相关，故 $0 = \Cov(y_i, y_r) = P_{ig} P_{rg} \Var(s_g)$ ，这迫使 $P_{ig} = 0$ 。于是第 $g$ 列也只有一个非零元，即 $\bP = \bPi\bD$ 。最后， $\bPi\bD\bs$ 的各分量相互独立（Proposition 1.3命题 1.3）。

The Gaussian column needs the separate covariance step because the Darmois–Skitovich theorem places no restriction on a Gaussian variable shared by two forms. Applied to the ICA model, the theorem gives identifiability.

高斯列之所以需要单独的协方差论证，是因为 Darmois–Skitovich 定理对两个线性形式共享的高斯变量没有任何限制。把这个定理用到 ICA 模型上，就得到可辨识性。

Corollary 4.7 (identifiability with at most one Gaussian source)推论 4.7（至多一个高斯源时的可辨识性）

Let $\bx = \bA\bs$ satisfy Assumption 1.1假设 1.1, with non-degenerate sources of finite variance.

If $\bW \in \R^{n \times m}$ makes $\bW\bA$ invertible and the components of $\by = \bW\bx$ pairwise independent, then $\bW\bA = \bPi\bD$ : the outputs are the sources up to order and scale.
The mixing matrix is identifiable: if $\bA\bs \overset{d}{=} \bB\bt$ where $(\bB, \bt)$ also satisfies these assumptions, then $\bB = \bA\bPi\bD$ .

设 $\bx = \bA\bs$ 满足Assumption 1.1假设 1.1，且各源非退化、方差有限。

若 $\bW \in \R^{n \times m}$ 使 $\bW\bA$ 可逆，并使 $\by = \bW\bx$ 的各分量两两独立，则 $\bW\bA = \bPi\bD$ ：输出就是在顺序与尺度意义下的源。
混合矩阵可辨识：若 $\bA\bs \overset{d}{=} \bB\bt$ ，且 $(\bB, \bt)$ 也满足上述假设，则 $\bB = \bA\bPi\bD$ 。

Proof.证明

Part 1 is Theorem 4.6定理 4.6 applied to $\bP = \bW\bA$ . For part 2, the covariance of $\bx$ is $\bA\bSigma_{\bs}\bA^\top = \bB\bSigma_{\bt}\bB^\top$ with positive definite diagonal $\bSigma_{\bs}$ and $\bSigma_{\bt}$ , so $\bA$ and $\bB$ have the same column space. Let $\bA^{+}$ be the pseudo-inverse, so that $\bA^{+}\bA = \bI$ and $\bA\bA^{+}$ is the orthogonal projector onto that column space; then $\bA\bM = \bB$ for $\bM = \bA^{+}\bB$ , and $\bM$ is invertible because $\bB$ has rank $n$ . Applying $\bA^{+}$ to both sides of $\bA\bs \overset{d}{=} \bB\bt$ gives $\bs \overset{d}{=} \bM\bt$ , so $\bM\bt$ has mutually independent components. By Theorem 4.6定理 4.6 applied to $\bt$ , $\bM = \bPi\bD$ , and $\bB = \bA\bPi\bD$ .

第 1 条就是把Theorem 4.6定理 4.6用于 $\bP = \bW\bA$ 。对第 2 条， $\bx$ 的协方差为 $\bA\bSigma_{\bs}\bA^\top = \bB\bSigma_{\bt}\bB^\top$ ，其中 $\bSigma_{\bs}$ 与 $\bSigma_{\bt}$ 是正定对角矩阵，所以 $\bA$ 与 $\bB$ 的列空间相同。记 $\bA^{+}$ 为伪逆，则 $\bA^{+}\bA = \bI$ ，且 $\bA\bA^{+}$ 是到该列空间的正交投影；于是对 $\bM = \bA^{+}\bB$ 有 $\bA\bM = \bB$ ，又因 $\bB$ 的秩为 $n$ ， $\bM$ 可逆。对 $\bA\bs \overset{d}{=} \bB\bt$ 两边作用 $\bA^{+}$ ，得 $\bs \overset{d}{=} \bM\bt$ ，所以 $\bM\bt$ 的各分量相互独立。对 $\bt$ 应用Theorem 4.6定理 4.6，得 $\bM = \bPi\bD$ ，从而 $\bB = \bA\bPi\bD$ 。

Two features of Theorem 4.6定理 4.6 deserve emphasis. First, pairwise independence of the outputs is enough. For general random vectors pairwise independence is much weaker than mutual independence (Exercise 4.2习题 4.2), but for invertible linear transforms of independent sources the two coincide. It is therefore enough, in principle, to make the outputs independent pair by pair. The algorithm of Comon (1994) is built on this: it sweeps repeatedly over the pairs of outputs, rotating each pair to optimize a contrast, and JADE uses the same pairwise Jacobi sweeps (Chapter 9第 9 章). Second, the theorem justifies the notion of a contrast function (Comon, 1994): any criterion that is optimized only when the outputs are independent is optimized only at separating matrices. The mutual information of the outputs in Chapter 6第 6 章 is the canonical example.

Theorem 4.6定理 4.6有两点值得强调。第一，输出两两独立就足够了。对一般的随机向量，两两独立远弱于相互独立（Exercise 4.2习题 4.2）；但对独立源的可逆线性变换，两者是一回事。因此，原则上只需让输出两两独立即可。Comon（1994）的算法正是建立在这一点之上：它反复扫描各对输出，每次旋转一对输出以优化对比函数；JADE 也采用同样的成对 Jacobi 扫描（Chapter 9第 9 章）。第二，该定理为对比函数的概念提供了依据（Comon, 1994）：凡是只在输出独立时才取到最优的准则，都只在分离矩阵处取到最优。Chapter 6第 6 章中输出的互信息就是最典型的例子。

4.4 Several Gaussian sources多个高斯源

The condition "at most one Gaussian source" cannot be dropped, but its failure is confined to the Gaussian sources. The next example shows the obstruction; the theorem after it shows that it does not spread to the non-Gaussian sources.

“至多一个高斯源”这一条件不能去掉，但它失效时，影响只局限于高斯源本身。下面的例子展示这一障碍，随后的定理说明它不会波及非高斯源。

Example 4.8 (a continuum of solutions)例 4.8（一族连续的解）

Let $s_1, s_2$ be independent $\Ncal(0, 1)$ and let $s_3, \dots, s_n$ be independent non-Gaussian sources with unit variance, independent of $s_1, s_2$ . For an angle $\theta$ , let $\bQ_\theta$ rotate the $(1, 2)$ coordinate plane by $\theta$ and leave the other coordinates fixed. The first two components of $\bQ_\theta\bs$ are jointly Gaussian with identity covariance, hence independent standard Gaussians, and they remain independent of $s_3, \dots, s_n$ ; so $\bQ_\theta\bs \overset{d}{=} \bs$ . Consequently $(\bA\bQ_\theta^\top, \bQ_\theta\bs)$ produces the same observations as $(\bA, \bs)$ for every $\theta$ , while $\bA\bQ_\theta^\top$ is of the form $\bA\bPi\bD$ only when $\theta$ is a multiple of $\pi/2$ . The data are compatible with a one-parameter family of mixing matrices.

设 $s_1, s_2$ 是独立的 $\Ncal(0, 1)$ 变量， $s_3, \dots, s_n$ 是方差为一、相互独立的非高斯源，且与 $s_1, s_2$ 独立。对角度 $\theta$ ，令 $\bQ_\theta$ 在 $(1, 2)$ 坐标平面内旋转 $\theta$ ，其余坐标不动。 $\bQ_\theta\bs$ 的前两个分量联合高斯且协方差为单位阵，因此是独立的标准高斯变量，并且仍与 $s_3, \dots, s_n$ 独立；所以 $\bQ_\theta\bs \overset{d}{=} \bs$ 。于是对任意 $\theta$ ， $(\bA\bQ_\theta^\top, \bQ_\theta\bs)$ 与 $(\bA, \bs)$ 产生相同的观测，而 $\bA\bQ_\theta^\top$ 只有在 $\theta$ 为 $\pi/2$ 的整数倍时才具有 $\bA\bPi\bD$ 的形式。与数据相容的混合矩阵构成一个单参数族。

Theorem 4.9 (partial identifiability)定理 4.9（部分可辨识性）

Let $\bs$ have mutually independent, non-degenerate components with finite variances; let $s_1, \dots, s_{n-k}$ be non-Gaussian and $s_{n-k+1}, \dots, s_n$ Gaussian, with diagonal covariance $\bSigma_G$ . Let $\bP \in \R^{n \times n}$ be invertible and suppose the components of $\by = \bP\bs$ are pairwise independent. Then there is a permutation matrix $\bPi$ such that

设 $\bs$ 的各分量相互独立、非退化且方差有限；其中 $s_1, \dots, s_{n-k}$ 非高斯， $s_{n-k+1}, \dots, s_n$ 是高斯的，其协方差为对角矩阵 $\bSigma_G$ 。设 $\bP \in \R^{n \times n}$ 可逆，且 $\by = \bP\bs$ 的各分量两两独立。则存在置换矩阵 $\bPi$ ，使

\bPi\bP = \begin{pmatrix} \bD & \bzero \\ \bzero & \bM \end{pmatrix},\tag{4.2}

where $\bD$ is an invertible $(n-k) \times (n-k)$ diagonal matrix and $\bM$ is an invertible $k \times k$ matrix with $\bM\bSigma_G\bM^\top$ diagonal. Conversely, every such $\bP$ makes the components of $\bP\bs$ mutually independent.

其中 $\bD$ 是 $(n-k) \times (n-k)$ 可逆对角矩阵， $\bM$ 是使 $\bM\bSigma_G\bM^\top$ 为对角矩阵的 $k \times k$ 可逆矩阵。反之，任何这样的 $\bP$ 都使 $\bP\bs$ 的各分量相互独立。

Proof.证明

As in the proof of Theorem 4.6定理 4.6, the Darmois–Skitovich theorem shows that each non-Gaussian column of $\bP$ has exactly one nonzero entry, and that these $n - k$ entries lie in distinct rows. Let $R$ be the set of the other $k$ rows and $\bM$ the $k \times k$ block of $\bP$ formed by the rows in $R$ and the Gaussian columns. The rows in $R$ vanish outside the Gaussian columns, so $\by_R = \bM\bs_G$ , where $\bs_G = (s_{n-k+1}, \dots, s_n)^\top$ ; these $k$ rows of the invertible $\bP$ are linearly independent and supported on $k$ columns, so $\bM$ is invertible.

与Theorem 4.6定理 4.6的证明一样，由 Darmois–Skitovich 定理知， $\bP$ 的每个非高斯列恰有一个非零元，且这 $n - k$ 个非零元位于不同的行。记其余 $k$ 行构成的集合为 $R$ ，记 $\bM$ 为 $\bP$ 中由 $R$ 中各行与高斯列构成的 $k \times k$ 子块。 $R$ 中各行在高斯列以外全为零，所以 $\by_R = \bM\bs_G$ ，其中 $\bs_G = (s_{n-k+1}, \dots, s_n)^\top$ ；这 $k$ 行是可逆矩阵 $\bP$ 的行，线性无关，又只在 $k$ 个列上取非零值，所以 $\bM$ 可逆。

Now take a row $i \notin R$ ; it has the form $y_i = P_{il} s_l + \mathbf{p}_i^\top\bs_G$ , where $l$ is its non-Gaussian column and $\mathbf{p}_i$ collects its entries in the Gaussian columns. For each $r \in R$ , independence of $y_i$ and $y_r$ gives $0 = \Cov(y_i, y_r)$ ; stacking these $k$ equations gives $\bM\bSigma_G\mathbf{p}_i = \bzero$ , and since $\bM\bSigma_G$ is invertible, $\mathbf{p}_i = \bzero$ . Ordering the rows as the non-Gaussian rows followed by $R$ produces (4.2). Finally, $\by_R = \bM\bs_G$ is a Gaussian vector with covariance $\bM\bSigma_G\bM^\top$ ; its components are pairwise independent, hence uncorrelated, so this matrix is diagonal.

再取一行 $i \notin R$ ，它形如 $y_i = P_{il} s_l + \mathbf{p}_i^\top\bs_G$ ，其中 $l$ 是该行对应的非高斯列， $\mathbf{p}_i$ 汇集了该行在高斯列上的元素。对每个 $r \in R$ ，由 $y_i$ 与 $y_r$ 独立得 $0 = \Cov(y_i, y_r)$ ；把这 $k$ 个方程合在一起，就是 $\bM\bSigma_G\mathbf{p}_i = \bzero$ ，而 $\bM\bSigma_G$ 可逆，故 $\mathbf{p}_i = \bzero$ 。把各行排成“先非高斯行、后 $R$ 中各行”的顺序，就得到 (4.2)。最后， $\by_R = \bM\bs_G$ 是协方差为 $\bM\bSigma_G\bM^\top$ 的高斯向量；其分量两两独立，因而不相关，所以这个矩阵是对角的。

For the converse, the first $n - k$ outputs of $\bPi\bP\bs$ are $d_l s_l$ , functions of distinct non-Gaussian sources, and the remaining outputs $\bM\bs_G$ are a function of the Gaussian sources only. The latter are jointly Gaussian and uncorrelated, hence mutually independent, so all $n$ outputs are mutually independent.

反过来， $\bPi\bP\bs$ 的前 $n - k$ 个输出是 $d_l s_l$ ，分别只依赖于不同的非高斯源；其余输出 $\bM\bs_G$ 只依赖于高斯源。后者联合高斯且互不相关，因而相互独立，所以全部 $n$ 个输出相互独立。

In terms of the mixing matrix, write $\bA = [\bA_N \ \bA_G]$ with the columns of the non-Gaussian and the Gaussian sources. A matrix $\bW$ that produces pairwise independent outputs with $\bP = \bW\bA$ invertible implies the mixing matrix $\bA\bP^{-1} = [\bA_N\bD^{-1} \ \ \bA_G\bM^{-1}]\,\bPi$ , because $\bx = \bA\bP^{-1}\by$ . The non-Gaussian columns are recovered individually, up to scale and order, exactly as in Corollary 4.7推论 4.7. The Gaussian columns are replaced by $\bA_G\bM^{-1}$ , which spans the same subspace as $\bA_G$ but is otherwise arbitrary. Only the Gaussian subspace $\mathrm{span}(\bA_G)$ is identified. After whitening, with unit-variance outputs, $\bM$ is orthogonal: any rotation inside the Gaussian subspace is as good as any other. The case $k = 1$ is Theorem 4.6定理 4.6 again, since a $1 \times 1$ block is diagonal.

用混合矩阵的语言来说，把 $\bA$ 按非高斯源与高斯源分块为 $\bA = [\bA_N \ \bA_G]$ 。若 $\bW$ 使输出两两独立且 $\bP = \bW\bA$ 可逆，则由 $\bx = \bA\bP^{-1}\by$ ，它所对应的混合矩阵为 $\bA\bP^{-1} = [\bA_N\bD^{-1} \ \ \bA_G\bM^{-1}]\,\bPi$ 。非高斯列被逐个恢复，只差尺度与顺序，与Corollary 4.7推论 4.7完全相同。高斯列则被替换成 $\bA_G\bM^{-1}$ ：它与 $\bA_G$ 张成同一个子空间，除此之外是任意的。能被辨识的只有高斯子空间 $\mathrm{span}(\bA_G)$ 。白化并把输出归一化为单位方差之后， $\bM$ 是正交矩阵：高斯子空间内的任何旋转都同样合格。 $k = 1$ 时， $1 \times 1$ 的块自然是对角的，结论又回到Theorem 4.6定理 4.6。

In practice this means that an ICA algorithm applied to data with several Gaussian sources returns some basis of the Gaussian subspace, chosen by its initialization and by sampling noise. Such components change from run to run, while the non-Gaussian components stay put. This situation arises, for instance, when the model order is overestimated and several noise dimensions, often close to Gaussian, enter the decomposition; Chapter 11第 11 章 discusses how to detect it (Section 11.5第 11.5 节).

在实践中，这意味着：对含多个高斯源的数据运行 ICA，算法会返回高斯子空间的某一组基，具体是哪一组取决于初始化和抽样噪声。这些成分每次运行都会变化，而非高斯成分保持不变。例如，当模型阶数估计过高、若干往往接近高斯的噪声维度进入分解时，就会出现这种情况；Chapter 11第 11 章讨论如何发现它（Section 11.5第 11.5 节）。

4.5 Identifiability, separability and uniqueness可辨识性、可分离性与唯一性

Eriksson & Koivunen (2004) distinguished three properties of the model $\bx = \bA\bs$ that coincide in the square case but come apart in general:

Eriksson & Koivunen（2004）区分了模型 $\bx = \bA\bs$ 的三种性质，它们在方阵情形下是一回事，一般情形下则各不相同：

Identifiability. The columns of $\bA$ are determined by the distribution of $\bx$ up to scale and order (Definition 4.1定义 4.1; when $m < n$ , full column rank is replaced by the requirement that no column be zero).
Separability. A matrix $\bW$ exists for which $\bW\bx$ has independent, non-degenerate components, and every such $\bW$ recovers the sources: $\bW\bx = \bPi\bD\bs$ .
Uniqueness. The model is identifiable and, in addition, the distributions of the sources are determined up to scale and order.

可辨识性。 $\bA$ 的各列由 $\bx$ 的分布在尺度与顺序意义下唯一确定（Definition 4.1定义 4.1；当 $m < n$ 时，把列满秩条件换成没有零列）。
可分离性。存在使 $\bW\bx$ 各分量独立且非退化的矩阵 $\bW$ ，并且任何这样的 $\bW$ 都能恢复源： $\bW\bx = \bPi\bD\bs$ 。
唯一性。模型可辨识，并且源的分布也在尺度与顺序意义下唯一确定。

Under Assumption 1.1假设 1.1, with non-degenerate sources of finite variance, all three hold. Corollary 4.7推论 4.7 gives identifiability, and also separability, because independent non-degenerate outputs force $\bW\bA$ to be invertible: a linear relation among the outputs would make one of them a function of the others, hence independent of itself and therefore constant. Since the sources $\bs = \bA^{+}\bx$ are then determined up to scale and order, so are their distributions. With more sources than sensors ( $m < n$ ) the properties separate. The model is never separable: $\bW\bA$ has rank at most $m < n$ , so the $n$ outputs satisfy a linear relation and cannot be independent and non-degenerate. Yet the mixing matrix can still be identifiable: if no source is Gaussian and no two columns of $\bA$ are collinear, then $\bA$ is identifiable within the class of models whose sources are all non-Gaussian and whose mixing matrices have no two collinear columns (Eriksson & Koivunen, 2004, building on Kagan, Linnik & Rao, 1973). Uniqueness can fail even then: if every source contains an independent Gaussian component, Gaussian variance can sometimes be moved from one source to another without changing the distribution of $\bx$ (Exercise 4.5习题 4.5). Chapter 12第 12 章 develops the overcomplete case (Theorem 12.1定理 12.1).

在Assumption 1.1假设 1.1之下（源非退化且方差有限），三者都成立。Corollary 4.7推论 4.7给出可辨识性，也给出可分离性，因为相互独立且非退化的输出迫使 $\bW\bA$ 可逆：若各输出之间存在线性关系，其中某个输出就是其余输出的函数，从而与自身独立，只能是常数。既然源 $\bs = \bA^{+}\bx$ 在尺度与顺序意义下已被确定，它们的分布自然也被确定。源多于传感器（ $m < n$ ）时，三者就分开了。此时模型永远不可分离： $\bW\bA$ 的秩至多为 $m < n$ ，所以 $n$ 个输出之间存在线性关系，不可能既相互独立又非退化。但混合矩阵仍可能可辨识：若没有高斯源、且 $\bA$ 没有两列共线，则在“源都非高斯、混合矩阵没有共线列”的模型类中， $\bA$ 是可辨识的（Eriksson & Koivunen, 2004，其基础是 Kagan, Linnik & Rao, 1973 的定理）。即便如此，唯一性仍可能不成立：如果每个源都含有一个独立的高斯成分，有时可以把高斯方差从一个源挪到另一个源，而不改变 $\bx$ 的分布（Exercise 4.5习题 4.5）。Chapter 12第 12 章专门讨论过完备情形（Theorem 12.1定理 12.1）。

Other models, other answers换一个模型，换一个答案

Identifiability always refers to a model class, and changing the class changes the answer. The standard model treats the samples as independent draws (Assumption 1.1假设 1.1, item 4), and the Gaussian obstruction is a consequence of ignoring time. If the sources are stationary time series with unit variance and autocorrelations $\rho_i(\tau) = \E[s_i(t)s_i(t-\tau)]$ , the lagged covariances $\E[\bx(t)\bx(t-\tau)^\top] = \bA\,\diag(\rho_1(\tau), \dots, \rho_n(\tau))\,\bA^\top$ identify $\bA$ even when every source is Gaussian, provided that for each pair of sources some lag distinguishes their autocorrelations (Theorem 9.6定理 9.6). In the other direction, a general nonlinear mixture $\bx = \mathbf{f}(\bs)$ is not identifiable at all: independent components can be manufactured from any random vector by a suitable nonlinear map (Hyvärinen & Pajunen, 1999; Theorem 13.2定理 13.2). Identifiability returns, up to componentwise transformations, when the sources are modulated by an observed auxiliary variable and suitable variability conditions hold (Theorem 13.5定理 13.5).

可辨识性总是相对于某一类模型而言的，换一类模型，答案就会改变。标准模型把样本视为独立抽样（Assumption 1.1假设 1.1第 4 条），高斯源造成的障碍正是忽略时间顺序的后果。如果源是方差为一的平稳时间序列，自相关为 $\rho_i(\tau) = \E[s_i(t)s_i(t-\tau)]$ ，那么滞后协方差 $\E[\bx(t)\bx(t-\tau)^\top] = \bA\,\diag(\rho_1(\tau), \dots, \rho_n(\tau))\,\bA^\top$ 即使在所有源都是高斯时也能辨识 $\bA$ ，只要对每一对源都有某个滞后能区分它们的自相关（Theorem 9.6定理 9.6）。反方向上，一般的非线性混合 $\bx = \mathbf{f}(\bs)$ 完全不可辨识：从任何随机向量出发，都可以用适当的非线性映射造出独立成分（Hyvärinen & Pajunen, 1999；Theorem 13.2定理 13.2）。当源受一个可观测的辅助变量调制、且满足适当的变化性条件时，可辨识性又在逐分量变换的意义下恢复（Theorem 13.5定理 13.5）。

4.6 Identifiable is not the same as easy to estimate可辨识不等于容易估计

Identifiability is a statement about infinite data and says nothing about how many samples are needed. A source that is non-Gaussian but close to Gaussian is identifiable, yet the statistics that tell it apart from a Gaussian are small and must be estimated from data. The experiment below makes this visible. Each source is a normalized sum of $k$ independent uniform variables; it has unit variance and kurtosis $-1.2/k$ (cumulants add over independent summands), and it approaches a Gaussian as $k$ grows. Two such sources are rotated by a random angle, and the angle is recovered by maximizing the absolute kurtosis of a projection over a grid of directions, as in Section 1.5第 1.5 节; the fourth moments of all projections are computed from the five fourth-order moments of $\bz$ .

可辨识性是关于无穷多数据的论断，对需要多少样本只字未提。一个非高斯但接近高斯的源是可辨识的，但把它与高斯区分开的那些统计量很小，而且必须从数据中估计。下面的实验让这一点一目了然。每个源是 $k$ 个独立均匀变量的归一化和：方差为一，峭度为 $-1.2/k$ （累积量对独立加项可加）， $k$ 越大越接近高斯。把两个这样的源旋转一个随机角度，再像Section 1.5第 1.5 节那样，在一组方向上搜索投影峭度绝对值的最大值来恢复这个角度；所有投影的四阶矩都由 $\bz$ 的五个四阶矩算出。

import numpy as np

rng = np.random.default_rng(0)
T, runs = 5000, 500
th = np.linspace(0, np.pi, 1800, endpoint=False)    # all directions (w ~ -w)
c, s = np.cos(th), np.sin(th)

def source(k):
    """Normalized sum of k unit-variance uniforms: kurtosis -1.2 / k."""
    u = rng.uniform(-np.sqrt(3), np.sqrt(3), (k, T))
    return u.sum(axis=0) / np.sqrt(k)

for k in [1, 2, 4, 8]:
    err = []
    for _ in range(runs):
        a = rng.uniform(0, np.pi / 2)                   # true rotation angle
        Q = np.array([[np.cos(a), -np.sin(a)], [np.sin(a), np.cos(a)]])
        z = Q @ np.vstack([source(k), source(k)])       # whitened mixture
        m = [np.mean(z[0]**(4 - j) * z[1]**j) for j in range(5)]
        m4 = (c**4 * m[0] + 4 * c**3 * s * m[1] + 6 * c**2 * s**2 * m[2]
              + 4 * c * s**3 * m[3] + s**4 * m[4])     # E[(w^T z)^4]
        m2 = (c**2 * np.mean(z[0]**2) + 2 * c * s * np.mean(z[0] * z[1])
              + s**2 * np.mean(z[1]**2))                # E[(w^T z)^2]
        d = (th[np.argmax(np.abs(m4 - 3 * m2**2))] - a) % (np.pi / 2)
        err.append(min(d, np.pi / 2 - d))               # error modulo pi/2
    rms = np.degrees(np.sqrt(np.mean(np.square(err))))
    print(f"k = {k}: kurtosis {-1.2 / k:+.2f}, rms error {rms:5.2f} deg")

The script prints root-mean-square angle errors of 0.95, 2.09, 5.51 and 13.33 degrees for $k = 1, 2, 4, 8$ . Each halving of the kurtosis multiplies the error by a little more than two, and at $k = 8$ the error is already half of what a random guess gives: an angle error uniform on $[0, \pi/4]$ has root-mean-square $(\pi/4)/\sqrt{3} \approx 26$ degrees. Better contrasts than kurtosis (Chapter 6第 6 章) change the numbers but cannot remove the effect. Chapter 10第 10 章 makes this precise: the Cramér–Rao bound on the separation error of two sources grows without bound as their Fisher informations approach the Gaussian value (Theorem 10.10定理 10.10). It is the finite-sample counterpart of Example 4.8例 4.8: exactly Gaussian sources cannot be separated at all, and nearly Gaussian sources can be separated only with many samples.

脚本对 $k = 1, 2, 4, 8$ 分别输出 0.95、2.09、5.51 和 13.33 度的均方根角度误差。峭度每减半，误差就变为原来的两倍多；到 $k = 8$ 时，误差已达随机猜测水平的一半：在 $[0, \pi/4]$ 上均匀分布的角度误差，其均方根为 $(\pi/4)/\sqrt{3} \approx 26$ 度。换用比峭度更好的对比函数（Chapter 6第 6 章）会改变这些数字，但消除不了这一效应。Chapter 10第 10 章将把这一点说得更精确：当两个源的 Fisher 信息趋于高斯分布的取值时，分离误差的 Cramér–Rao 界趋于无穷（Theorem 10.10定理 10.10）。这正是Example 4.8例 4.8在有限样本下的对应：严格高斯的源根本无法分离，接近高斯的源只有在样本很多时才能分离。

Key points

本章要点

Identifiability is a property of the model: the distribution of $\bx$ must determine $\bA$ up to a scaled permutation of its columns. Without it, no algorithm can succeed.
Darmois–Skitovich: two independent linear forms in independent variables can share only Gaussian variables. With all moments finite, the proof combines vanishing cross-cumulants, a Vandermonde system, and the theorems of Marcinkiewicz and Cramér.
Comon: if at most one source is Gaussian and $\bP$ is invertible, pairwise independence of the components of $\bP\bs$ forces $\bP = \bPi\bD$ . Hence ICA is identifiable, and pairwise independence of the outputs already implies separation.
With $k \ge 2$ Gaussian sources, the non-Gaussian sources are still identified individually, but the Gaussian ones only through the subspace they span, with a free $k \times k$ rotation after whitening.
Identifiability, separability and uniqueness differ beyond the square model; temporal structure and auxiliary variables change what is identifiable; and identifiable sources that are nearly Gaussian need many samples.

可辨识性是模型的性质： $\bx$ 的分布必须在列的带尺度置换意义下确定 $\bA$ 。缺少这一性质，任何算法都无法成功。
Darmois–Skitovich 定理：独立变量的两个相互独立的线性形式只能共享高斯变量。在各阶矩有限时，证明由交叉累积量为零、Vandermonde 方程组以及 Marcinkiewicz 与 Cramér 的定理组合而成。
Comon 定理：若至多一个源是高斯的且 $\bP$ 可逆，则 $\bP\bs$ 各分量两两独立迫使 $\bP = \bPi\bD$ 。因此 ICA 可辨识，且输出两两独立就已意味着分离成功。
有 $k \ge 2$ 个高斯源时，非高斯源仍能逐个辨识，而高斯源只能通过它们张成的子空间来辨识，白化后还剩一个任意的 $k \times k$ 旋转。
超出方阵模型后，可辨识性、可分离性与唯一性各不相同；时间结构与辅助变量会改变哪些量可辨识；接近高斯的源虽然可辨识，却需要大量样本。

4.7 Exercises习题

Exercise 4.1习题 4.1 basic基础

Let $s_1, s_2$ be independent with zero mean, unit variance and kurtosis $\kappa$ , and let $y_1 = s_1 + s_2$ , $y_2 = s_1 - s_2$ . (a) Show that $y_1$ and $y_2$ are uncorrelated. (b) Show that $\cum(y_1, y_1, y_2, y_2) = \Cov(y_1^2, y_2^2) = 2\kappa$ . (c) Evaluate this for uniform and Laplacian sources. What does the Darmois–Skitovich theorem add when $\kappa = 0$ ?

设 $s_1, s_2$ 独立，均值为零、方差为一、峭度为 $\kappa$ ，令 $y_1 = s_1 + s_2$ ， $y_2 = s_1 - s_2$ 。（a）证明 $y_1$ 与 $y_2$ 不相关。（b）证明 $\cum(y_1, y_1, y_2, y_2) = \Cov(y_1^2, y_2^2) = 2\kappa$ 。（c）对均匀源和拉普拉斯源计算这个量。当 $\kappa = 0$ 时，Darmois–Skitovich 定理还能告诉我们什么？

Show solution查看解答

(a) $\E[y_1 y_2] = \E[s_1^2] - \E[s_2^2] = 0$ . (b) By multilinearity, and because cross-cumulants of independent variables vanish, $\cum(y_1, y_1, y_2, y_2) = \sum_i a_i^2 b_i^2 \kappa_4(s_i)$ with $\ba = (1, 1)$ and $\bb = (1, -1)$ , which is $2\kappa$ . Directly: $y_1 y_2 = s_1^2 - s_2^2$ , so $\E[y_1^2 y_2^2] = \E[(s_1^2 - s_2^2)^2] = 2\E[s^4] - 2 = 2(\kappa + 3) - 2 = 2\kappa + 4$ , while $\E[y_1^2]\,\E[y_2^2] = 4$ ; hence $\Cov(y_1^2, y_2^2) = 2\kappa$ . For zero-mean variables $\cum(y_1, y_1, y_2, y_2) = \E[y_1^2 y_2^2] - \E[y_1^2]\E[y_2^2] - 2(\E[y_1 y_2])^2$ , which gives the same value because $\E[y_1 y_2] = 0$ . (c) Uniform sources ( $\kappa = -1.2$ ) give $-2.4$ and Laplacian sources ( $\kappa = 3$ ) give $6$ , so in both cases $y_1$ and $y_2$ are dependent although uncorrelated. When $\kappa = 0$ this statistic vanishes, but the theorem still says that $y_1$ and $y_2$ are independent only if $s_1$ and $s_2$ are Gaussian; for a non-Gaussian law with $\kappa = 0$ and all moments finite, some other cross-cumulant in (4.1) must be nonzero.

（a） $\E[y_1 y_2] = \E[s_1^2] - \E[s_2^2] = 0$ 。（b）由多重线性，并注意到独立变量的交叉累积量为零， $\cum(y_1, y_1, y_2, y_2) = \sum_i a_i^2 b_i^2 \kappa_4(s_i)$ ，其中 $\ba = (1, 1)$ ， $\bb = (1, -1)$ ，结果为 $2\kappa$ 。也可以直接计算： $y_1 y_2 = s_1^2 - s_2^2$ ，故 $\E[y_1^2 y_2^2] = \E[(s_1^2 - s_2^2)^2] = 2\E[s^4] - 2 = 2(\kappa + 3) - 2 = 2\kappa + 4$ ，而 $\E[y_1^2]\,\E[y_2^2] = 4$ ，所以 $\Cov(y_1^2, y_2^2) = 2\kappa$ 。对零均值变量， $\cum(y_1, y_1, y_2, y_2) = \E[y_1^2 y_2^2] - \E[y_1^2]\E[y_2^2] - 2(\E[y_1 y_2])^2$ ，由于 $\E[y_1 y_2] = 0$ ，结果相同。（c）均匀源（ $\kappa = -1.2$ ）给出 $-2.4$ ，拉普拉斯源（ $\kappa = 3$ ）给出 $6$ ，因此两种情形下 $y_1$ 与 $y_2$ 虽不相关却不独立。当 $\kappa = 0$ 时这个统计量为零，但定理仍然断言：只有当 $s_1$ 与 $s_2$ 是高斯时， $y_1$ 与 $y_2$ 才独立；对 $\kappa = 0$ 、各阶矩有限的非高斯分布，(4.1) 中必有另一个交叉累积量不为零。

Exercise 4.2习题 4.2 basic基础

Let $y_1, y_2$ be independent random signs, equal to $\pm 1$ with probability $1/2$ each, and let $y_3 = y_1 y_2$ . Show that $y_1, y_2, y_3$ are pairwise independent but not mutually independent. Why does this not contradict Theorem 4.6定理 4.6?

设 $y_1, y_2$ 是独立的随机符号，各以 $1/2$ 的概率取 $\pm 1$ ，令 $y_3 = y_1 y_2$ 。证明 $y_1, y_2, y_3$ 两两独立，但不相互独立。为什么这与Theorem 4.6定理 4.6不矛盾？

Show solution查看解答

$y_3$ is again a random sign. For signs $a, b$ : $P(y_1 = a, y_3 = b) = P(y_1 = a, y_2 = ab) = 1/4 = P(y_1 = a)\,P(y_3 = b)$ , and likewise for the pair $(y_2, y_3)$ ; the pair $(y_1, y_2)$ is independent by assumption. But $P(y_1 = 1, y_2 = 1, y_3 = -1) = 0 \neq 1/8$ , so the three are not mutually independent. There is no contradiction because $\by$ is not of the form $\bP\bs$ with $\bP$ invertible and $\bs$ independent: if it were, $\bs = \bP^{-1}\by$ would take only the four values that $\by$ takes, whereas a vector with three independent non-degenerate components takes at least $2^3 = 8$ values with positive probability.

$y_3$ 也是随机符号。对符号 $a, b$ ，有 $P(y_1 = a, y_3 = b) = P(y_1 = a, y_2 = ab) = 1/4 = P(y_1 = a)\,P(y_3 = b)$ ； $(y_2, y_3)$ 同理； $(y_1, y_2)$ 按假设独立。但 $P(y_1 = 1, y_2 = 1, y_3 = -1) = 0 \neq 1/8$ ，所以三者不相互独立。这与定理并不矛盾，因为 $\by$ 不能写成 $\bP\bs$ （ $\bP$ 可逆、 $\bs$ 各分量独立）的形式：否则 $\bs = \bP^{-1}\by$ 只能取 $\by$ 所取的四个值，而三个独立非退化分量构成的向量至少以正概率取 $2^3 = 8$ 个不同的值。

Exercise 4.3习题 4.3 intermediate进阶

Let $s_1, s_2$ be independent with finite moments of all orders, and suppose that $s_1 + s_2$ and $s_1 - s_2$ are independent. Using (4.1) directly, show that $s_1$ and $s_2$ are Gaussian with equal variances, without appealing to Cramér's theorem. (This special case of the Darmois–Skitovich theorem is known as the Kac–Bernstein theorem; see Kac (1939) and Kagan, Linnik & Rao (1973).)

设 $s_1, s_2$ 独立且各阶矩都有限，并设 $s_1 + s_2$ 与 $s_1 - s_2$ 独立。直接利用 (4.1)，在不借助 Cramér 定理的情况下证明 $s_1$ 与 $s_2$ 都是高斯的且方差相等。（Darmois–Skitovich 定理的这一特例称为 Kac–Bernstein 定理，见 Kac（1939）以及 Kagan, Linnik & Rao（1973）。）

Show solution查看解答

Here $\ba = (1, 1)$ and $\bb = (1, -1)$ , so (4.1) reads $\kappa_r(s_1) + (-1)^q \kappa_r(s_2) = 0$ whenever $r = p + q$ with $p, q \ge 1$ . For $r = 2$ the only choice is $p = q = 1$ , giving $\kappa_2(s_1) = \kappa_2(s_2)$ : equal variances. For $r \ge 3$ both $q = 1$ and $q = 2$ are allowed, giving $\kappa_r(s_1) - \kappa_r(s_2) = 0$ and $\kappa_r(s_1) + \kappa_r(s_2) = 0$ , hence $\kappa_r(s_1) = \kappa_r(s_2) = 0$ for every $r \ge 3$ . By Lemma 4.5引理 4.5 with $K = 2$ , both variables are Gaussian. Cramér's theorem is not needed because the ratios $b_i / a_i = 1$ and $-1$ are distinct, so every group $G_k$ in the proof contains a single variable.

此时 $\ba = (1, 1)$ ， $\bb = (1, -1)$ ，所以只要 $r = p + q$ 且 $p, q \ge 1$ ，(4.1) 就给出 $\kappa_r(s_1) + (-1)^q \kappa_r(s_2) = 0$ 。 $r = 2$ 时只能取 $p = q = 1$ ，得 $\kappa_2(s_1) = \kappa_2(s_2)$ ，即方差相等。 $r \ge 3$ 时 $q = 1$ 与 $q = 2$ 都可以取，得到 $\kappa_r(s_1) - \kappa_r(s_2) = 0$ 与 $\kappa_r(s_1) + \kappa_r(s_2) = 0$ ，因此对每个 $r \ge 3$ 都有 $\kappa_r(s_1) = \kappa_r(s_2) = 0$ 。由Lemma 4.5引理 4.5（取 $K = 2$ ），两个变量都是高斯的。这里不需要 Cramér 定理，因为比值 $b_i / a_i = 1$ 与 $-1$ 互不相同，证明中的每一组 $G_k$ 都只含一个变量。

Exercise 4.4习题 4.4 intermediate进阶

Let $n = 3$ and let $\bz = \tilde\bA\bs$ be whitened data with $\tilde\bA$ orthogonal, where $s_1$ is Laplacian and $s_2, s_3$ are standard Gaussian, all independent with unit variance. Describe all orthogonal $\bU$ for which $\by = \bU\bz$ has pairwise independent components. Which source is recovered, and how large is the set of solutions?

设 $n = 3$ ， $\bz = \tilde\bA\bs$ 是白化数据， $\tilde\bA$ 为正交矩阵，其中 $s_1$ 服从拉普拉斯分布， $s_2, s_3$ 是标准高斯变量，三者独立且方差为一。求出所有使 $\by = \bU\bz$ 各分量两两独立的正交矩阵 $\bU$ 。哪个源被恢复了？解的集合有多大？

Show solution查看解答

Apply Theorem 4.9定理 4.9 to the orthogonal, hence invertible, matrix $\bP = \bU\tilde\bA$ with $k = 2$ and $\bSigma_G = \bI$ : there is a permutation $\bPi$ with $\bPi\bU\tilde\bA = \bigl(\begin{smallmatrix} d & \bzero \\ \bzero & \bM \end{smallmatrix}\bigr)$ . This matrix is orthogonal, so its diagonal blocks are orthogonal: $d = \pm 1$ and $\bM \in O(2)$ , for which $\bM\bM^\top = \bI$ is automatically diagonal. Conversely every such matrix gives independent outputs. Hence $\bU = \bPi^\top \bigl(\begin{smallmatrix} \pm 1 & \bzero \\ \bzero & \bM \end{smallmatrix}\bigr) \tilde\bA^\top$ with $\bM$ any $2 \times 2$ rotation or reflection. One output equals $\pm s_1$ : the Laplacian source is recovered up to sign and position. The other two outputs are $\bM(s_2, s_3)^\top$ , an arbitrary orthonormal recombination of the Gaussian sources. The solution set is a union of finitely many circles (three positions for $\pm s_1$ , two signs, and the two components of $O(2)$ ), so it is one-dimensional: one free angle, as in Example 4.8例 4.8.

对正交（因而可逆）矩阵 $\bP = \bU\tilde\bA$ 应用Theorem 4.9定理 4.9，其中 $k = 2$ ， $\bSigma_G = \bI$ ：存在置换矩阵 $\bPi$ ，使 $\bPi\bU\tilde\bA = \bigl(\begin{smallmatrix} d & \bzero \\ \bzero & \bM \end{smallmatrix}\bigr)$ 。这个矩阵是正交的，所以它的对角块也正交： $d = \pm 1$ ， $\bM \in O(2)$ ，而此时 $\bM\bM^\top = \bI$ 自然是对角的。反之，任何这样的矩阵都给出独立的输出。因此 $\bU = \bPi^\top \bigl(\begin{smallmatrix} \pm 1 & \bzero \\ \bzero & \bM \end{smallmatrix}\bigr) \tilde\bA^\top$ ，其中 $\bM$ 是任意 $2 \times 2$ 旋转或反射。有一个输出等于 $\pm s_1$ ：拉普拉斯源在符号与位置意义下被恢复。另外两个输出是 $\bM(s_2, s_3)^\top$ ，即高斯源的任意标准正交重组。解集是有限个圆的并（ $\pm s_1$ 的三个位置、两种符号，以及 $O(2)$ 的两个连通分支），因此是一维的：恰有一个自由角度，与Example 4.8例 4.8一致。

Exercise 4.5习题 4.5 advanced硬核

Let $m = 2$ , $n = 4$ and $\ba_i = (\cos\alpha_i, \sin\alpha_i)^\top$ with $\alpha_i = (i - 1)\pi/4$ . Let $s_i = u_i + g_i$ , where $u_1, \dots, u_4, g_1, \dots, g_4$ are independent, each $u_i$ is uniform with unit variance and $g_i \sim \Ncal(0, \sigma_i^2)$ . (a) Find numbers $\delta_1, \dots, \delta_4$ , not all zero, with $\sum_i \delta_i \ba_i\ba_i^\top = \bzero$ . (b) Show that replacing $\sigma_i^2 = 1$ by $\sigma_i^2 = 1 + \varepsilon\delta_i$ , with $|\varepsilon| < 1$ , does not change the distribution of $\bx = \sum_i s_i \ba_i$ . (c) Conclude that the source distributions are not unique, even though no source is Gaussian and no two columns of $\bA$ are collinear.

设 $m = 2$ ， $n = 4$ ， $\ba_i = (\cos\alpha_i, \sin\alpha_i)^\top$ ，其中 $\alpha_i = (i - 1)\pi/4$ 。令 $s_i = u_i + g_i$ ，其中 $u_1, \dots, u_4, g_1, \dots, g_4$ 相互独立，每个 $u_i$ 服从方差为一的均匀分布， $g_i \sim \Ncal(0, \sigma_i^2)$ 。（a）求不全为零的数 $\delta_1, \dots, \delta_4$ ，使 $\sum_i \delta_i \ba_i\ba_i^\top = \bzero$ 。（b）证明把 $\sigma_i^2 = 1$ 换成 $\sigma_i^2 = 1 + \varepsilon\delta_i$ （ $|\varepsilon| < 1$ ）不改变 $\bx = \sum_i s_i \ba_i$ 的分布。（c）由此说明：尽管没有高斯源、 $\bA$ 也没有两列共线，源的分布却不唯一。

Show solution查看解答

(a) The four matrices $\ba_i\ba_i^\top$ are $\bigl(\begin{smallmatrix} 1 & 0 \\ 0 & 0 \end{smallmatrix}\bigr)$ , $\tfrac12\bigl(\begin{smallmatrix} 1 & 1 \\ 1 & 1 \end{smallmatrix}\bigr)$ , $\bigl(\begin{smallmatrix} 0 & 0 \\ 0 & 1 \end{smallmatrix}\bigr)$ and $\tfrac12\bigl(\begin{smallmatrix} 1 & -1 \\ -1 & 1 \end{smallmatrix}\bigr)$ ; they must be linearly dependent because symmetric $2 \times 2$ matrices form a three-dimensional space. Indeed $\delta = (1, -1, 1, -1)$ works: the $(1,1)$ entry is $1 - \tfrac12 + 0 - \tfrac12 = 0$ , the $(1,2)$ entry is $0 - \tfrac12 + 0 + \tfrac12 = 0$ , and the $(2,2)$ entry is $0 - \tfrac12 + 1 - \tfrac12 = 0$ . (b) Write $\bx = \sum_i u_i \ba_i + \mathbf{g}$ with $\mathbf{g} = \sum_i g_i \ba_i$ . The two terms are independent, and $\mathbf{g}$ is Gaussian with covariance $\sum_i \sigma_i^2 \ba_i\ba_i^\top$ , which does not change when $\sigma_i^2 = 1$ is replaced by $1 + \varepsilon\delta_i$ (these are positive for $|\varepsilon| < 1$ ). The first term does not change either, and the law of a sum of independent vectors is determined by the laws of the summands, so the law of $\bx$ is unchanged. (c) Each source $s_i = u_i + g_i$ has kurtosis $-1.2$ (the Gaussian part adds none) and variance $1 + \sigma_i^2$ , so its normalized kurtosis $-1.2/(1 + \sigma_i^2)^2$ is nonzero, and it depends on $\sigma_i^2$ . For $\varepsilon \neq 0$ the multiset of normalized kurtoses changes from four copies of $-0.3$ to two copies each of $-1.2/(2 + \varepsilon)^2$ and $-1.2/(2 - \varepsilon)^2$ , so the new sources are not a rescaled permutation of the old ones. The columns are pairwise non-collinear and the sources are non-Gaussian, so $\bA$ is identifiable within the class of models described in Section 4.5第 4.5 节 (Theorem 12.1定理 12.1), yet the source distributions are not determined by the distribution of $\bx$ .

（a）四个矩阵 $\ba_i\ba_i^\top$ 分别为 $\bigl(\begin{smallmatrix} 1 & 0 \\ 0 & 0 \end{smallmatrix}\bigr)$ 、 $\tfrac12\bigl(\begin{smallmatrix} 1 & 1 \\ 1 & 1 \end{smallmatrix}\bigr)$ 、 $\bigl(\begin{smallmatrix} 0 & 0 \\ 0 & 1 \end{smallmatrix}\bigr)$ 与 $\tfrac12\bigl(\begin{smallmatrix} 1 & -1 \\ -1 & 1 \end{smallmatrix}\bigr)$ ； $2 \times 2$ 对称矩阵构成三维空间，所以它们必然线性相关。事实上 $\delta = (1, -1, 1, -1)$ 即满足要求： $(1,1)$ 元为 $1 - \tfrac12 + 0 - \tfrac12 = 0$ ， $(1,2)$ 元为 $0 - \tfrac12 + 0 + \tfrac12 = 0$ ， $(2,2)$ 元为 $0 - \tfrac12 + 1 - \tfrac12 = 0$ 。（b）把 $\bx$ 写成 $\bx = \sum_i u_i \ba_i + \mathbf{g}$ ，其中 $\mathbf{g} = \sum_i g_i \ba_i$ 。两项相互独立， $\mathbf{g}$ 是协方差为 $\sum_i \sigma_i^2 \ba_i\ba_i^\top$ 的高斯向量；把 $\sigma_i^2 = 1$ 换成 $1 + \varepsilon\delta_i$ （ $|\varepsilon| < 1$ 时为正）不改变这个协方差。第一项也不变，而独立向量之和的分布由各加项的分布决定，所以 $\bx$ 的分布不变。（c）每个源 $s_i = u_i + g_i$ 的峭度为 $-1.2$ （高斯部分不贡献峭度），方差为 $1 + \sigma_i^2$ ，所以其归一化峭度 $-1.2/(1 + \sigma_i^2)^2$ 不为零，且随 $\sigma_i^2$ 变化。当 $\varepsilon \neq 0$ 时，归一化峭度构成的多重集由四个 $-0.3$ 变为各两个 $-1.2/(2 + \varepsilon)^2$ 与 $-1.2/(2 - \varepsilon)^2$ ，因此新的源不是旧的源经过尺度变换和置换得到的。各列两两不共线，源都非高斯，所以在Section 4.5第 4.5 节所述的模型类中 $\bA$ 可辨识（Theorem 12.1定理 12.1）；然而源的分布并不由 $\bx$ 的分布唯一确定。

4.8 References参考文献

Comon, P. (1994). Independent component analysis, a new concept? Signal Processing, 36(3), 287–314.
Cramér, H. (1936). Über eine Eigenschaft der normalen Verteilungsfunktion. Mathematische Zeitschrift, 41(1), 405–414.
Darmois, G. (1953). Analyse générale des liaisons stochastiques: étude particulière de l'analyse factorielle linéaire. Revue de l'Institut International de Statistique, 21(1/2), 2–8.
Eriksson, J., & Koivunen, V. (2004). Identifiability, separability, and uniqueness of linear ICA models. IEEE Signal Processing Letters, 11(7), 601–604.
Hyvärinen, A., & Pajunen, P. (1999). Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks, 12(3), 429–439.
Kac, M. (1939). On a characterization of the normal distribution. American Journal of Mathematics, 61(3), 726–728.
Kagan, A. M., Linnik, Yu. V., & Rao, C. R. (1973). Characterization Problems in Mathematical Statistics. Wiley.
Marcinkiewicz, J. (1939). Sur une propriété de la loi de Gauss. Mathematische Zeitschrift, 44(1), 612–618.
Skitovich, V. P. (1953). On a property of the normal distribution. Doklady Akademii Nauk SSSR, 89, 217–219 (in Russian).