正定値実対称行列の諸性質
機械学習やより一般の情報科学の文脈で頻出する正定値実対称行列の性質についてまとめる。（分散共分散行列とかで出てくるンゴ。）より広くエルミート行列とか半正定値行列とかの性質について考えるのが見通しが良くていいのだが、実際使う上で正定値実対称に絞って覚えて（？）しまった方が楽。
定義
A ∈ R n × n A\in \mathbb{R}^{n\times n} A ∈ R n × n
A が対称行列 ⟺ d e f A T = A A ≻ 0 ⟺ d e f ∀ x ≠ 0 , x T A x > 0 \begin{aligned}
A\text{が対称行列} \;\; & \overset{\mathrm{def}}{\Longleftrightarrow} \; &&A^T = A \\
A\succ 0\quad & \overset{\mathrm{def}}{\Longleftrightarrow} &&\forall_{\bm{x}\neq 0},\; \bm{x}^T A \bm{x}>0
\end{aligned} A が対称行列 A ≻ 0 ⟺ def ⟺ def A T = A ∀ x = 0 , x T A x > 0 ただし A ≻ 0 A\succ 0 A ≻ 0 A A A
性質
この時、実対称行列 A A A
A が正定値 ⟺ A の固有値はすべて正 ( ∃ U , U T U = I A = U D U T , D = diag ( λ 1 , λ 2 , … ) , λ i > 0 ) ⟺ A はグラム行列 ( ∃ B ; 正則 A = B T B ) ⟺ A のコレスキー分解が存在する ( ∃ L ; 下三角、対角成分が正 A = L L T ) \begin{aligned}
& A\text{が正定値} \\
& \\
& \qquad \Longleftrightarrow A\text{の固有値はすべて正}\\
& \qquad\qquad \qquad \; \left(^{^\exists U,\; U^T U = I}\; A = UDU^T ,\; D=\text{diag}(\lambda_1,\lambda_2,\ldots),\;\lambda_i > 0\right)\\
& \qquad \Longleftrightarrow A\text{はグラム行列}\\
& \qquad\qquad \qquad \;\left(^{^\exists B;\;\text{正則}}\; A=B^{T}B\right)\\
& \qquad \Longleftrightarrow A\text{のコレスキー分解が存在する}\\
& \qquad\qquad \qquad \;\left(^{^\exists L;\;\text{下三角、対角成分が正}}\; A=LL^T\right)
\end{aligned} A が正定値 ⟺ A の固有値はすべて正 ( ∃ U , U T U = I A = U D U T , D = diag ( λ 1 , λ 2 , … ) , λ i > 0 ) ⟺ A はグラム行列 ( ∃ B ; 正則 A = B T B ) ⟺ A のコレスキー分解が存在する ( ∃ L ; 下三角、対角成分が正 A = L L T ) また、よく使うのは次。
実対称 A A A ⟹ \Longrightarrow ⟹ A A A
A A A ⟹ \Longrightarrow ⟹ A A A
（ A = U T D U A=U^T D U A = U T D U A − 1 = U T D − 1 U A^{-1} = U^T D^{-1} U A − 1 = U T D − 1 U
A A A f ( x ) = 1 2 x T A x − b T x f(\bm{x}) = \dfrac{1}{2}\bm{x}^T A \bm{x} - \bm{b}^T\bm{x} f ( x ) = 2 1 x T A x − b T x A x min = b A\bm{x}_\text{min}=\bm{b} A x min = b
( A A A A x = b Ax=b A x = b
A A A ∃ B : 正定値実対称 ; A = B 2 ^\exists B:\text{正定値実対称};\; A=B^2 ∃ B : 正定値実対称 ; A = B 2
（正定値平方根 A 1 / 2 A^{1/2} A 1/2
正則行列の極分解は直交行列＋正定値実対称行列で、一意
（ F = S U = V T , U T U = V T V = I , S , T ; 実対称正定値 F=SU=VT\; , \; U^T U = V^T V = I \;,\; S,T;\text{実対称正定値} F = S U = V T , U T U = V T V = I , S , T ; 実対称正定値
証明
実対称正定値 <-> 固有値はすべて正
A A A A A A
U A U − 1 = D = diag ( λ 1 , λ 2 , … ) ⇝ A = U T D U UAU^{-1} = D = \text{diag}\left(\lambda_1,\lambda_2,\ldots\right)\leadsto A = U^{T}D U U A U − 1 = D = diag ( λ 1 , λ 2 , … ) ⇝ A = U T D U よって、
x T A x = x T U T D U x = ( U x ) T D ( U x ) = ∑ i λ i ( U x ) i 2 \begin{aligned}
\bm{x}^TAx &= \bm{x}^TU^{T}D U\bm{x}\\
& = \left(U\bm{x}\right)^T D \left(U\bm{x}\right)\\
& = \sum_i \lambda_i \left(U\bm{x}\right)_i^2
\end{aligned} x T A x = x T U T D U x = ( U x ) T D ( U x ) = i ∑ λ i ( U x ) i 2 となるので、固有値 λ i \lambda_i λ i x T A x > 0 \bm{x}^T A \bm{x}>0 x T A x > 0 ∀ x ≠ 0 \forall \bm{x}\neq 0 ∀ x = 0 x \bm{x} x U x U\bm{x} U x R n \ 0 \mathbb{R}^n\backslash \bm{0} R n \ 0 ∀ x ≠ 0 ; x T A x > 0 ⟹ ∀ i ; λ i > 0 \forall {\bm{x}\neq 0} ; \bm{x}^T A \bm{x}>0 \Longrightarrow \forall i ; \lambda_i >0 ∀ x = 0 ; x T A x > 0 ⟹ ∀ i ; λ i > 0
グラム行列・正定値実対称平方根
A A A D = diag ( λ 1 , λ 2 , … ) D = \text{diag}(\lambda_1,\lambda_2,\ldots) D = diag ( λ 1 , λ 2 , … ) λ i > 0 \lambda_i>0 λ i > 0
D − 1 = diag ( λ 1 − 1 , λ 2 − 1 … ) D = diag ( λ 1 , λ 2 … ) \begin{aligned}
D^{-1} = \;\text{diag}\left(\lambda_1^{-1},\lambda_2^{-1}\ldots\right)\\[5pt]
\sqrt{D} = \; \text{diag}\left(\sqrt{\lambda_1},\sqrt{\lambda_2}\ldots\right)
\end{aligned} D − 1 = diag ( λ 1 − 1 , λ 2 − 1 … ) D = diag ( λ 1 , λ 2 … ) よって、 A A A
A = U T D U = U T D D U = U T D T Q Q T D U = ( Q T D U ) T ( Q T D U ) = ( U T D U ) ( U T D U ) \begin{aligned}
A &= U^T D U = U^T \sqrt{D}\sqrt{D} U = U^T \sqrt{D}^T Q Q^T\sqrt{D} U \\
&= \left(Q^T\sqrt{D}U\right)^T \left(Q^T\sqrt{D}U\right)\\
&= \left(U^T \sqrt{D}U\right)\left(U^T \sqrt{D}U\right)
\end{aligned} A = U T D U = U T D D U = U T D T Q Q T D U = ( Q T D U ) T ( Q T D U ) = ( U T D U ) ( U T D U ) ただし Q Q Q B = Q T D U B=Q^T\sqrt{D}U B = Q T D U A A A A = B T B A = B^T B A = B T B Q = U Q = U Q = U B T = B B^T = B B T = B A 1 / 2 = U T D U A^{1/2} = U^T \sqrt{D}U A 1/2 = U T D U A A A A = A 1 / 2 A 1 / 2 A = A^{1/2}A^{1/2} A = A 1/2 A 1/2 A A A A A A
x T U T D U x = ∑ i λ i ( U x ) i 2 > 0 ( ∵ x ≠ 0 , U正則 ⇒ U x ≠ 0 , ∀ i λ i > 0 ) \begin{aligned}
\bm{x}^T U^T \sqrt{D} U \bm{x} = \displaystyle\sum_i \sqrt{\lambda_i}\left(U\bm{x}\right)_i^2>0\\
\left(\; \because\;\bm{x}\neq 0,\; \text{U正則}\Rightarrow U\bm{x}\neq 0\;,^{\forall i} \lambda_i > 0\right)
\end{aligned} x T U T D U x = i ∑ λ i ( U x ) i 2 > 0 ( ∵ x = 0 , U 正則 ⇒ U x = 0 , ∀ i λ i > 0 ) から従う。
正定値実対称平方根の一意性
実対称正定値行列 A A A R 2 = A R^2 =A R 2 = A R R R R R R A A A λ i \lambda_i λ i E λ i A E_{\lambda_i}^A E λ i A x \bm{x} x
A ( R x ) = R 3 x = R A x = λ i ( R x ) A\left(R\bm{x}\right) = R^3\bm{x} = RA\bm{x} = \lambda_i\left(R\bm{x}\right) A ( R x ) = R 3 x = R A x = λ i ( R x ) となっているので、 R x ∈ E λ i A R\bm{x}\in E_{\lambda_i}^A R x ∈ E λ i A R R R A A A R R R E λ i A E_{\lambda_i}^A E λ i A R ∣ E λ i A R_{|E_{\lambda_i}^A} R ∣ E λ i A R R R R ∣ E λ i A R_{|E_{\lambda_i}^A} R ∣ E λ i A R ∣ E λ i A R_{|E_{\lambda_i}^A} R ∣ E λ i A μ j \mu_j μ j E μ j R E_{\mu_j}^R E μ j R x \bm{x} x
( R ∣ E λ i A ) 2 x = μ j 2 x ( R ∣ E λ i A ) 2 x = A x = λ i x \begin{aligned}
&\left(R_{|E_{\lambda_i}^A}\right)^2\bm{x} = \mu_j^2\bm{x}\\
&\left(R_{|E_{\lambda_i}^A}\right)^2\bm{x} = A\bm{x} = \lambda_i\bm{x}
\end{aligned} ( R ∣ E λ i A ) 2 x = μ j 2 x ( R ∣ E λ i A ) 2 x = A x = λ i x となる。特に、 R R R R ∣ E λ i A R_{|E_{\lambda_i}^A} R ∣ E λ i A μ j > 0 \mu_j>0 μ j > 0 μ j = λ i \mu_j = \sqrt{\lambda_i} μ j = λ i R ∣ E λ i A R_{|E_{\lambda_i}^A} R ∣ E λ i A
⨁ j E μ j R = E λ i A ⇝ E λ i R = E λ i A \bigoplus_j E_{\mu_j}^R = E_{\lambda_i}^A\;\; \leadsto \;\; E_{\sqrt{\lambda_i}}^R = E_{\lambda_i}^A j ⨁ E μ j R = E λ i A ⇝ E λ i R = E λ i A よって A A A λ i \lambda_i λ i R R R λ i \sqrt{\lambda_i} λ i R R R
R x = λ i x for λ i s.t x ∈ E λ i A R\bm{x}=\sqrt{\lambda_i}\bm{x}\quad \text{for}\;\; \lambda_i \;\; \text{s.t}\;\; \bm{x}\in E_{\lambda_i}^A R x = λ i x for λ i s.t x ∈ E λ i A よって、 R 2 = A R^2 = A R 2 = A R R R
正則行列の極分解は直交行列＋正定値実対称行列で一意に定まる
A A A Q , P Q,P Q , P J , K J,K J , K
A = Q J = K P A = QJ = KP A = Q J = K P と極分解することができ、直交行列 Q , P Q,P Q , P J , K J,K J , K
まず、 A A A A T A A^TA A T A A T A \sqrt{A^TA} A T A A A A
A = ( A A T A − 1 ) A T A A = \left(A\sqrt{A^T A}^{-1}\right)\sqrt{A^T A} A = ( A A T A − 1 ) A T A 特に今、 Q = A A T A − 1 Q = A\sqrt{A^T A}^{-1} Q = A A T A − 1
Q T Q = ( A T A − 1 ) T A T A A T A − 1 = A T A − 1 A T A A T A − 1 = I \begin{aligned}
Q^TQ &= \left(\sqrt{A^T A}^{-1}\right)^TA^TA\sqrt{A^T A}^{-1}\\
&= \sqrt{A^T A}^{-1}A^TA\sqrt{A^T A}^{-1}\\
&= I
\end{aligned} Q T Q = ( A T A − 1 ) T A T A A T A − 1 = A T A − 1 A T A A T A − 1 = I となるので、 Q Q Q A T A = J \sqrt{A^TA}=J A T A = J
A = ( A A T A − 1 ) A T A = Q J A = \left(A\sqrt{A^T A}^{-1}\right)\sqrt{A^T A} = QJ A = ( A A T A − 1 ) A T A = Q J であり、確かに A A A A A T AA^T A A T
A = A A T ( A A T − 1 A ) A = \sqrt{AA^T}\left(\sqrt{AA^T}^{-1}A\right) A = A A T ( A A T − 1 A ) と分解すれば、同様にして P = A A T − 1 A P=\sqrt{AA^T}^{-1}A P = A A T − 1 A K = A A T K = \sqrt{AA^T} K = A A T A = K P A=KP A = K P
次に分解の一意性について考える。今 A = Q J A = QJ A = Q J Q Q Q J J J A T A = J Q T Q J = J 2 A^T A = J Q^TQJ = J^2 A T A = J Q T Q J = J 2 J = A T A J = \sqrt{A^TA} J = A T A J J J Q = A J − 1 Q = AJ^{-1} Q = A J − 1 A = K P A=KP A = K P