JS散度
# Jensen–Shannon divergence
2022-06-08
Tags: #JSDivergence
- KL 散度是不对称的, 这使得在训练过程中可能出现一些问题，所以我们在 KL 散度基础上引入 JS 散度
# 定义
- The Jensen–Shannon divergence (JSD) $M_+^1( A ) × M_+^1(A) → [ 0 , ∞ )$ 1is a symmetrized and smoothed version of the Kullback–Leibler divergence $D ( P ∥ Q )$ .
- It is defined by $$ \operatorname{JSD}(P | Q)=\frac{1}{2} D(P | M)+\frac{1}{2} D(Q | M) $$ where $M=\frac{1}{2}(P+Q)$.
看不懂这个符号: Consider the set $M_+^1( A )$ of probability distributions where $A$ is a set provided with some σ-algebra of measurable subsets. In particular we can take $A$ to be a finite or countable set with all subsets being measurable. Jensen–Shannon divergence - Wikipedia ↩︎