Part.31_Principal_Component_Analysis(ML_Andrew.Ng.)

Last updated Nov 11, 2021 Edit Source

• All
• Machine learning
• Dimensionality reduction
• PC a

Tags: #MachineLearning #DimensionalityReduction #PCA

• Normalization 归一化
• Standardization 标准化

PCA依赖于欧氏距离, 所以预处理数据可以让降维效果更好.

$$\Sigma=\frac{1}{m} \sum_{i=1}^{n}\left(x^{(i)}\right)\left(x^{(i)}\right)^{T}$$

• Using Singular Value Decomposition
• [U,S,V] = svd(Sigma);
• We need to use $$U=\left[\begin{array}{cccc} \mid & \mid & & \mid \\ u^{(1)} & u^{(2)} & \ldots & u^{(n)} \\ \mid & \mid & & \mid \end{array}\right] \in \mathbb{R}^{n \times n}$$

• 取出$U$里面的前$k$个特征向量: $$\left[\begin{array}{cccc} \mid & \mid & & \mid \\ u^{(1)} & u^{(2)} & \ldots & u^{(k)} \\ \mid & \mid & & \mid \end{array}\right] \in \mathbb{R}^{n \times k}$$

  • 这就是我们的投影矩阵

• 单个数据点的投影: $$z^{(i)}=\left[\begin{array}{cccc} \mid & \mid & & \mid \\ u^{(1)} & u^{(2)} & \ldots & u^{(k)} \\ \mid & \mid & & \mid \end{array}\right]^{T} x^{(i)}{n \times 1}=\left[\begin{array}{c} -\left(u^{(1)}\right)^T- \\ \vdots \\ -\left(u^{(k)}\right)^T- \end{array}\right]{k \times n}x^{(i)}_{n \times 1}$$