The ICA BookICA 学习书 Yuda Bi

An open book · bilingual开放教材 · 中英双语

Independent Component Analysis: From Foundations to Frontiers独立成分分析：从基础到硬核

Yuda Bi

A bilingual open book on independent component analysis, from first principles to identifiability, contrasts, algorithms, Cramér–Rao bounds, overcomplete and nonlinear ICA, and ICA for fMRI, with exercises and solutions.

一本中英双语的独立成分分析开放教材：从基本原理到可辨识性、对比函数、算法、Cramér–Rao 界、过完备与非线性 ICA，以及 fMRI 中的 ICA，每章配习题与解答。

Start reading开始阅读 →

About this book关于本书

Independent component analysis recovers hidden signals from their linear mixtures using two assumptions: the signals are statistically independent, and at most one of them is Gaussian. These two assumptions lead to a remarkably complete theory, with a sharp identifiability theorem, a family of contrast functions linked by information theory, algorithms with provable convergence, exact asymptotic error formulas and a Cramér–Rao bound, and clear limits once the model is stretched to more sources than sensors or to nonlinear mixing. It is also one of the workhorses of brain imaging.

独立成分分析只依靠两个假设，即各信号相互统计独立、且其中至多一个服从高斯分布，就能从线性混合中恢复隐藏的信号。这两个假设引出了一套相当完整的理论：一个精确的可辨识性定理，一族由信息论联系起来的对比函数，可证明收敛的算法，精确的渐近误差公式与 Cramér–Rao 界，以及当模型推广到源多于传感器或非线性混合时清晰的边界。它也是脑影像分析的主力工具之一。

This book develops the subject from first principles to research level. Every chapter states its results precisely, proves the central ones or points to where the proofs can be found, and checks the theory against small numerical experiments that readers can run themselves. Each chapter closes with exercises, from routine checks to derivations of published results, with worked solutions.

本书从基本原理一直讲到研究前沿。每一章都精确陈述结论，对核心结果给出证明或指出证明的出处，并用读者可以自己运行的小型数值实验来检验理论。每章末尾配有习题，从常规练习到已发表结果的推导，都附有完整解答。

How to read it如何阅读

The prerequisites are linear algebra (eigendecompositions, orthogonal matrices) and probability at the level of an undergraduate course. The chapters on statistical theory also use basic asymptotic statistics; Appendix A collects what is needed. Two paths through the book work well:

预备知识是线性代数（特征分解、正交矩阵）和本科程度的概率论。统计理论各章还会用到基本的渐近统计，所需内容汇集在附录 A。推荐两条阅读路线：

Use the switch at the top of every page to read in English, in Chinese, or in both languages paragraph by paragraph; the choice is remembered. Links jump between chapters, theorems, equations and exercises, and the contents panel on the left (behind the Contents button on a phone) follows you through the book.

每页顶部的切换按钮可以选择英文、中文，或者中英逐段对照阅读，选择会被记住。链接可以在章节、定理、公式和习题之间跳转；左侧的目录栏（手机上点目录按钮打开）在全书中随时可用。

Further reading延伸阅读

Four books cover the field from complementary angles:

以下四本书从不同角度覆盖了这一领域：

  1. Hyvärinen, A., Karhunen, J., & Oja, E. (2001). Independent Component Analysis. Wiley.
  2. Comon, P., & Jutten, C. (Eds.) (2010). Handbook of Blind Source Separation: Independent Component Analysis and Applications. Academic Press.
  3. Cichocki, A., & Amari, S. (2002). Adaptive Blind Signal and Image Processing: Learning Algorithms and Applications. Wiley.
  4. Stone, J. V. (2004). Independent Component Analysis: A Tutorial Introduction. MIT Press.

Contents目录

Part I · Foundations第一部分 · 基础

  1. Chapter 1第 1 章The ICA ProblemICA 问题

    The cocktail-party problem, the linear mixing model, what can and cannot be recovered, and how ICA differs from PCA and factor analysis.

    鸡尾酒会问题、线性混合模型、哪些量可以恢复哪些不能，以及 ICA 与 PCA、因子分析的区别。

  2. Chapter 2第 2 章Independence, Cumulants and Entropy独立性、累积量与熵

    The probability toolkit of ICA: independence versus uncorrelatedness, characteristic functions, moments and cumulants, differential entropy, mutual information and negentropy.

    ICA 的概率工具：独立与不相关、特征函数、矩与累积量、微分熵、互信息与负熵。

  3. Chapter 3第 3 章Whitening and the Geometry of ICA白化与 ICA 的几何

    Second-order statistics determine the mixing matrix up to a rotation. Whitening, PCA reduction, and why a rotation is exactly what higher-order statistics must find.

    二阶统计量只能把混合矩阵确定到一个旋转。白化、PCA 降维，以及为什么高阶统计量要找的恰好是这个旋转。

Part II · Principles第二部分 · 原理

  1. Chapter 4第 4 章Identifiability可辨识性

    The Darmois–Skitovich theorem and Comon's theorem: with at most one Gaussian source, the mixing matrix is determined up to scaling and permutation.

    Darmois–Skitovich 定理与 Comon 定理：至多一个高斯源时，混合矩阵在尺度与置换意义下唯一。

  2. Chapter 5第 5 章Non-Gaussianity and Kurtosis非高斯性与峭度

    Why mixing makes signals more Gaussian, kurtosis as a contrast, the landscape of kurtosis on the sphere, and the price of its sensitivity to outliers.

    为什么混合会让信号更接近高斯、峭度作为对比函数、峭度在球面上的地形，以及它对离群值敏感的代价。

  3. Chapter 6第 6 章Negentropy, Mutual Information and Likelihood负熵、互信息与似然

    Information-theoretic contrasts and maximum likelihood: their equivalences, the score function, Infomax, robust negentropy approximations, and what happens when the source model is wrong.

    信息论对比函数与最大似然：它们的等价关系、得分函数、Infomax、稳健的负熵近似，以及源模型设错时会发生什么。

Part III · Algorithms第三部分 · 算法

  1. Chapter 7第 7 章Natural Gradient, Infomax and Stability自然梯度、Infomax 与稳定性

    The relative (natural) gradient, equivariance, local stability of the separating solutions, and the Infomax and extended Infomax algorithms.

    相对（自然）梯度、等变性、分离解的局部稳定性，以及 Infomax 与扩展 Infomax 算法。

  2. Chapter 8第 8 章FastICAFastICA 算法

    The fixed-point algorithm as an approximate Newton method: one-unit, deflation and symmetric versions, convergence rates, and the choice of nonlinearity.

    作为近似牛顿法的不动点算法：单单元、逐次提取与对称版本，收敛速度，以及非线性函数的选择。

  3. Chapter 9第 9 章Tensor and Second-Order Methods: JADE and SOBI张量方法与二阶方法：JADE 与 SOBI

    Fourth-order cumulant tensors and joint diagonalization (JADE), Jacobi rotations, and separation from temporal structure alone (AMUSE, SOBI).

    四阶累积量张量与联合对角化（JADE）、Jacobi 旋转，以及仅利用时间结构的分离（AMUSE、SOBI）。

Part IV · Statistical Theory第四部分 · 统计理论

  1. Chapter 10第 10 章Asymptotic Performance and the Cramér–Rao Bound渐近性能与 Cramér–Rao 界

    The gain matrix and interference-to-signal ratio, asymptotic variance of contrast-based estimators, the optimal nonlinearity, and the Cramér–Rao bound for ICA.

    增益矩阵与干扰信号比、基于对比函数的估计量的渐近方差、最优非线性函数，以及 ICA 的 Cramér–Rao 界。

  2. Chapter 11第 11 章Noise, Model Order and Reliability噪声、模型阶数与可靠性

    Noisy ICA, PCA reduction and model-order selection, split and unstable components, and how to assess the reliability of an ICA decomposition.

    含噪 ICA、PCA 降维与模型阶数选择、分裂与不稳定的成分，以及如何评估一次 ICA 分解的可靠性。

Part V · Beyond the Square Linear Model第五部分 · 超越方阵线性模型

  1. Chapter 12第 12 章Overcomplete ICA and Sparse Component Analysis过完备 ICA 与稀疏成分分析

    More sources than sensors: what is still identifiable, why sources cannot simply be inverted, tensor rank, and sparsity-based recovery.

    源多于传感器时：哪些仍可辨识、为什么源不能直接求逆、张量秩，以及基于稀疏性的恢复。

  2. Chapter 13第 13 章Nonlinear ICA非线性 ICA

    Why general nonlinear ICA is not identifiable, and how temporal structure or auxiliary variables restore identifiability (TCL, PCL, iVAE).

    为什么一般的非线性 ICA 不可辨识，以及时间结构或辅助变量如何恢复可辨识性（TCL、PCL、iVAE）。

Part VI · ICA for Brain Imaging第六部分 · 脑影像中的 ICA

  1. Chapter 14第 14 章Spatial ICA of fMRIfMRI 的空间 ICA

    Spatial versus temporal ICA, the fMRI data model, preprocessing and dimension reduction, and how to read components as networks or artifacts.

    空间 ICA 与时间 ICA、fMRI 数据模型、预处理与降维，以及如何把成分解读为网络或伪迹。

  2. Chapter 15第 15 章Group ICA, Dual Regression and IVA组 ICA、双回归与独立向量分析

    Multi-subject ICA: temporal concatenation, back-reconstruction, dual regression, spatially constrained ICA, independent vector analysis, network connectivity and multimodal fusion.

    多被试 ICA：时间拼接、反重建、双回归、空间约束 ICA、独立向量分析、网络连接以及多模态融合。

Appendix附录

  1. Appendix A附录 AMathematical Background数学基础

    Facts used throughout: orthogonal and permutation matrices, matrix square roots, the polar decomposition, Kronecker products and tensors, matrix calculus, optimization on the sphere and the orthogonal group, and the asymptotic statistics of Part IV (delta method, sandwich covariance, Fisher information).

    全书用到的数学工具：正交矩阵与置换矩阵、矩阵平方根、极分解、Kronecker 积与张量、矩阵微积分、球面与正交群上的优化，以及第四部分用到的渐近统计（delta 方法、三明治协方差、Fisher 信息）。