STK 500. Pengantar Teori Statistika. Eigenvalues and Eigenvectors

Transkripsi

1 STK 5 Pengantar Teori Statistika Eigenvalues and Eigenvectors

2 Eigenvalues and Eigenvectors Example : if we have a matrix A: then A or A I which implies there are two roots or eigenvalues : =-6 and =4.

3 Eigenvalues and Eigenvectors For a square matrix A, let I be a conformable identity matrix. Then the scalars satisfying the polynomial equation A - I = are called the eigenvalues (or characteristic roots) of A. The equation A - I = is called the characteristic equation or the determinantal equation.

4 Eigenvalues and Eigenvectors For a matrix A with eigenvectors, a nonzero vector x such that Ax = x is called an eigenvector (or characteristic vector) of A associated with.

5 Example if we have a matrix A: 4 A 4-4 with eigenvalues = -6 and = 4, the eigenvector of A associated with = -6 is Ax and x 4x 6 x 4-4 x x x 4x 6x 8x 4x 4x 4x 6x 4x x Fixing x = yields a solution for x of.

6 Example Note that eigenvectors are usually normalized so they have unit length, i.e., e For our previous example we have: e x x'x x x'x Thus our arbitrary choice to fix x = has no impact on the eigenvector associated with = -6.

7 Example For matrix A and eigenvalue = 4, we have Ax and x 4x 4 x 4-4 x x x 4x 4x x 4x 4x 4x 4x 4x 8x We again arbitrarily fix x =, which now yields a solution for x of /.

8 Normalization of Eigenvectors Normalization to unit length yields e x 5 x'x Again our arbitrary choice to fix x = has no impact on the eigenvector associated with = 4.

9 Example Find the eigenvalues and corresponding eigenvectors for the matrix A. What is the dimension of the eigenspace of each eigenvalue? Characteristic equation: I A Eigen value : A 3 ( )

10 Example The eigenspace of λ = : x ( I A) x x x x x 3 s t x 3 s t, s, t s t s, t R: the eigenspace of A corresponding to Thus, the dimension of its eigenspace is.

11 Eigenvalues and Eigenvectors () If an eigenvalue occurs as a multiple root (k times) for the characteristic polynominal, then has multiplicity k. () The multiplicity of an eigenvalue is greater than or equal to the dimension of its eigen space.

12 Example 3 Find the eigenvalues and corresponding eigenspaces for 3 A 3 3 I A 3 ( ) ( 4) eigenvalues 4, The eigenspaces for these two eigenvalues are as follows. B B {(,, )} Basis for 4 {(,, ),(,, )} Basis for

13 Example 4 Find the eigenvalues of the matrix A and find a basis for each of the corresponding eigenspaces A Characteristic equation: I A 3 ( ) ( )( 3) Eigenvalues:,, 3 3 According to the note on the previous slide, the dimension of the eigenspace of λ = is at most to be For λ = and λ 3 = 3, the dimensions of their eigenspaces are at most to be

14 Example 4 () x 5 x ( I A) x x 3 x4 x t x s s t, s, t x 3 t x4 t, is a basis for the eigenspace corresponding to The dimension of the eigenspace of λ = is

15 Example 4 () x 5 x ( I A) x x 3 x4 x x 5t 5 t, t x 3 t x4 5 is a basis for the eigenspace corresponding to The dimension of the eigenspace of λ = is

16 Example 4 (3) 3 3 x 5 x ( 3I A) x x 3 x4 x x 5t 5 t, t x 3 x4 t 5 is a basis for the eigenspace corresponding to 3 3 The dimension of the eigenspace of λ 3 = 3 is

17 Eigenvalues and Eigenvectors Theorem. Eigenvalues for triangular matrices If A is an nn triangular matrix, then its eigenvalues are the entries on its main diagonal Finding eigenvalues for triangular and diagonal matrices (a) A (a) I A ( )( )( 3) 5 3 3,, 3 3

18 Eigenvalues and Eigenvectors Finding eigenvalues for triangular and diagonal matrices (b) A 4 3 (b),,, 4,

19 Diagonalization Definition : A square matrix A is called diagonalizable if there exists an invertible matrix P such that P AP is a diagonal matrix (i.e., P diagonalizes A) Definition : A square matrix A is called diagonalizable if A is similar to a diagonal matrix

20 Diagonalization Theorem : Similar matrices have the same eigenvalues If A and B are similar nn matrices, then they have the same eigenvalues A and B are similar B P AP For any diagonal matrix in the form of D = λi, P DP = D Considering the characteristic equation of B: I B I P AP P IP P AP P ( I A) P P I A P P P I A P P I A I A Since A and B have the same characteristic equation, they are with the same eigenvalues

21 Example 5 Eigenvalue problems and diagonalization programs Characteristic equation: 3 A 3 3 I A 3 ( 4)( ) The eigenvalues : 4,, 3 () 4 the eigenvector p

22 Example 5, 4 P [ p p p3], and P AP P [ p p p ] () the eigenvector p p3 If 3 P AP 4 The above example can prove Thm. numerically since the eigenvalues for both A and P AP are the same to be 4,, and The reason why the matrix P is constructed with eigenvectors of A is demonstrated in Thm. 3 on the next slide

23 Diagonalization Theorem 3: An nn matrix A is diagonalizable if and only if it has n linearly independent eigenvectors Note that if there are n linearly independent eigenvectors, it does not imply that there are n distinct eigenvalues. It is possible to have only one eigenvalue with multiplicity n, and there are n linearly independent eigenvectors for this eigenvalue However, if there are n distinct eigenvalues, then there are n linearly independent eigenvectors and thus A must be diagonalizable Since is diagonalizable, there exists an invertible s.t. A P D P AP is diagonal. Let P [ p p p ] and D diag(,,, ), then ( ) n PD [ p p pn] n [ p p p ] n n n

24 ( ) Diagonalization AP PD D P AP (since ) [ Ap Ap Ap ] [ p p p ] n n n Ap p, i,,, n i i i (The above equations imply the column vectors p of P are eigenvectors of A, and the diagonal entries in D are eigenvalues of A) Because Ais diagonalizable P is invertible Columns in P, i.e., p, p,, p, are linearly independent (see p. 4. in the lecture note or p. 46 in the text book) Thus, Ahas n linearly independent eigenvectors Since Ahas nlinearly independent eigenvectors p, p, i n i with corresponding eigenvalues,, n, then Ap p, i,,, n Let P [ p p p n ] i i i p n

25 Diagonalization AP A[ p p p ] [ Ap Ap Ap ] n [ p p p ] n n [ p p pn] PD n Since p, p,, p are linearly independent P is invertible (see p. 4. in the lecture note or p. 46 in the text book) AP PDP AP D n A is diagonalizable (according to the definition of the diagonalizable matrix on Slide 7.7) Note that p i n 's are linearly independent eigenvectors and the diagonal entries i in the resulting diagonalized D are eigenvalues of A

26 Example 6 A matrix that is not diagonalizable Show that the following matrix is not diagonalizable A Characteristic equation: I A ( ) The eigenvalue, and then solve ( I A) x for eigenvectors I A I A eigenvector p Since A does not have two linearly independent eigenvectors, A is not diagonalizable.

27 Diagonalization Steps for diagonalizing an nn square matrix: Step : Find n linearly independent eigenvectors p, p, pn Step : Let P [ p p p n ] Step 3: for A with corresponding eigenvalues P AP D n where Ap p, i,,, n i i i,,, n

28 Example 6 Diagonalizing a matrix A 3 3 Find a matrix P such that P AP is diagonal. Characteristic equation: I A 3 ( )( )( 3) 3 The eigenvalues :,, 3 3

29 Example G.-J. E. I A x t x eigenvector p x 3 t 3 4 G.-J. E. I A x 4 t x 4 t eigenvector p x 3 t 4

30 Example G.-J. E. 3I A x t x t eigenvector p 3 x 3 t P [ p p p3] and it follows that 4 P AP 3

31 Diagonalization Note: a quick way to calculate A k based on the diagonalization technique () () k k k D D k n n k k D P AP D P AP P AP P AP P A P repeat k times k k k k k A PD P, where D k n

32 Diagonalization Theorem 4: Sufficient conditions for diagonalization If an nn matrix A has n distinct eigenvalues, then the corresponding eigenvectors are linearly independent and thus A is diagonalizable. Proof : Let λ, λ,, λ n be distinct eigenvalues and corresponding eigenvectors be x, x,, x n. In addition, consider that the first m eigenvalues are linearly independent, but the first m+ eigenvalues are linearly dependent, i.e., x m c x c x c x m m, () where c i s are not all zero. Multiplying both sides of Eq. () by A yields Axm Ac x Acx Acmx m x c x c x c x m m m m m ()

33 Diagonalization On the other hand, multiplying both sides of Eq. () by λ m+ yields mxm c mx c mx cm m xm (3) Now, subtracting Eq. () from Eq. (3) produces c ( )x c ( )x c ( )x = m m m m m m Since the first m eigenvectors are linearly independent, we can infer that all coefficients of this equation should be zero, i.e., c ( ) c ( ) c ( ) = m m m m m Because all the eigenvalues are distinct, it follows all c i s equal to, which contradicts our assumption that x m+ can be expressed as a linear combination of the first m eigenvectors. So, the set of n eigenvectors is linearly independent given n distinct eigenvalues, and according to Thm. 3, we can conclude that A is diagonalizable.

34 Example 7 Determining whether a matrix is diagonalizable A 3 Because A is a triangular matrix, its eigenvalues are,, 3. 3 According to Thm. 4, because these three values are distinct, A is diagonalizable.

35 Symmetric Matrices and Orthogonal Diagonalization A square matrix A is symmetric if it is equal to its transpose: T A A Example symmetric matrices and nonsymetric matrices 5 3 A B C (symmetric) (symmetric) (nonsymmetric)

36 Symmetric Matrices and Orthogonal Diagonalization Thm 5: Eigenvalues of symmetric matrices If A is an nn symmetric matrix, then the following properties are true. a) A is diagonalizable (symmetric matrices are guaranteed to has n linearly independent eigenvectors and thus be diagonalizable). b) All eigenvalues of A are real numbers c) If is an eigenvalue of A with multiplicity k, then has k linearly independent eigenvectors. That is, the eigenspace of has dimension k. The above theorem is called the Real Spectral Theorem, and the set of eigenvalues of A is called the spectrum of A.

37 Prove that a symmetric matrix is diagonalizable. b c c a A Proof: Characteristic equation: ) ( c ab b a b c c a A I 4 ) ( ) 4( ) ( c b a c b ab a c ab b ab a c ab b a As a function in, this quadratic polynomial function has a nonnegative discriminant as follows Example 8

38 Example 8 () ( a b) 4c a b, c A a c a c b a itself is a diagonal matrix. () ( a b) 4c The characteristic polynomial of A has two distinct real roots, which implies that A has two distinct real eigenvalues. According to Thm. 5, A is diagonalizable.

39 Symmetric Matrices and Orthogonal Diagonalization Orthogonal matrix : A square matrix P is called orthogonal if it is invertible and T T T P P (or PP P P I) Thm. 6: Properties of orthogonal matrices An nn matrix P is orthogonal if and only if its column vectors form an orthonormal set. Proof: Suppose the column vectors of P form an orthonormal set, i.e., T P P P p p p, where p p for i j and p p. n i j i i p p p p p p p p p p p p T T T n n T T T p p p p p p p p p p p p T T T p n p pn p pn pn pn p pn p pn pn It implies that P = P T and thus P is orthogonal. I n

40 Example 9 Show that P is an orthogonal matrix. P T T If P is a orthogonal matrix, then P P PP I T 4 PP I

41 Symmetric Matrices and Orthogonal Diagonalization Moreover, let p, p 5, and p 5 3, we can produce p p p p. 3 3 p p p p p p and p p 3 3 So, { p, p, p } is an orthonormal set. (Thm. 7.8 can be 3 illustrated by this example.)

42 Symmetric Matrices and Orthogonal Diagonalization Thm. 7: Properties of symmetric matrices Let A be an nn symmetric matrix. If and are distinct eigenvalues of A, then their corresponding eigenvectors x and x are orthogonal. (Thm. 6 only states that eigenvectors corresponding to distinct eigenvalues are linearly independent Proof : because A is symmetric ( x x ) ( x ) x ( Ax ) x ( Ax ) x ( x A ) x T T T ( x A) x x ( Ax ) x ( x ) x ( x ) (x x ) T T T The above equation implies ( )( x x ), and because, it follows that x x. So, x and x are orthogonal. For distinct eigenvalues of a symmetric matrix, their corresponding eigenvectors are orthogonal and thus linearly independent to each other Note that there may be multiple x and x corresponding to and

43 Symmetric Matrices and Orthogonal Diagonalization A matrix A is orthogonally diagonalizable if there exists an orthogonal matrix P such that P AP = D is diagonal. Thm.8: Fundamental theorem of symmetric matrices Let A be an nn matrix. Then A is orthogonally diagonalizable and has real eigenvalues if and only if A is symmetric. Proof: ( ) A is orthogonally diagonalizable ( ) T D P AP is diagonal, and P is an orthogonal matrix s.t. P P T T T T T T T T T A PDP PDP A ( PDP ) ( P ) D P PDP A See the next two slides

44 Symmetric Matrices and Orthogonal Diagonalization Let A be an nn symmetric matrix. () Find all eigenvalues of A and determine the multiplicity of each. According to Thm. 7, eigenvectors corresponding to distinct eigenvalues are orthognoal () For each eigenvalue of multiplicity, choose a unit eigenvector. (3) For each eigenvalue of multiplicity k, find a set of k linearly independent eigenvectors. If this set {v, v,, v k } is not orthonormal, apply the Gram-Schmidt orthonormalization process.

45 Symmetric Matrices and Orthogonal Diagonalization (4) The composite of steps () and (3) produces an orthonormal set of n eigenvectors. Use these orthonormal and thus linearly independent eigenvectors to form the columns of P. i. According to Thm. 7, the matrix P is orthogonal ii. Following the diagonalization process, D = P AP is diagonal iii. therefore, the matrix A is orthogonally diagonalizable

46 Determining whether a matrix is orthogonally diagonalizable A A 3 A 3 A 4 Orthogonally diagonalizable Symmetric matrix Example

47 Example Orthogonal diagonalization Find an orthogonal matrix Pthat diagonalizes A. A 4 4 Sol: () I A ( 3) ( 6) 6, 3 (has a multiplicity of ) v () 6, v (,, ) u ( 3, 3, 3) v (3) 3, v (,, ), v3 (,, ) Verify Thm. 7 that v v = v v 3 = Linearly independent but not orthogonal

48 Example Gram-Schmidt Process: v w w v (,, ), w v w (,, ) u ww w w (,, ), u (,, ) w w3 Verify Thm. 7 that after the Gram-Schmidt orthonormalization process, i) w and w 3 are eigenvectors of A corresponding to the eigenvalue of 3, and ii) v w = v w 3 = P u u u 3 6 P AP 3 3

49 Beberapa Teorema Akar Ciri dan Vektor Ciri Jika λ adalah akar ciri matriks A dengan vektor ciri padanannya x, maka untuk fungsi tertentu g(a) akan mempunyai akar ciri g(λ) dengan x vektor ciri padanannya. Kasus khusus :. Jika λ adalah akar ciri matriks A, maka גc adalah akar ciri matriks ca dengan c sebarang skalar Bukti : c A x = c λ x x vektor ciri padanan λ dari matriks A x vektor ciri padanan c λ dari matriks ca

50 Beberapa Teorema Akar Ciri dan Vektor Ciri. Jika ג adalah akar ciri matriks A, dengan x vektor ciri padanannya, maka cג+k adalah akar ciri matriks (ca+ki) dengan x vektor ciri padanannya. Bukti : c A x + k x = c λ x + k x (c A + k I)x = (c λ + k) x (tidak dapat diperluas untuk A + B, dengan A, B sebarang matriks n x n ) 3. λ adalah akar ciri dari matriks A (dapat diperluas untuk A k ) Bukti : A x = λ x A(A x) = A(λ x ) A x= λ Ax= λ(λ x )= λ x

51 Beberapa Teorema Akar Ciri dan Vektor Ciri 4. / λ adalah akar dari matriks A - Bukti : A x = λ x A - (A x) = A - (λ x ) x= λ A - x A - x = λ - x 5. Kasus () dan () dapat digunakan untuk mencari akar ciri dan vektor ciri dari polinomial A Contoh : (A A -3 A + 5 I ) x = A 3 x + 4 A x -3 Ax + 5 x = λ 3 x + 4 λ x -3 λ x + 5 x =(λ 3 + 4λ -3 λ + 5)x λ λ -3 λ + 5 adalah akar ciri dari A A -3 A + 5 I dan x vektor ciri padanannya

52 Beberapa Teorema Akar Ciri dan Vektor Ciri Sifat (5) dapat diperluas untuk deret tak hingga. Misal : akar ciri A adalah λ, maka (- λ) adalah akar ciri dari (I-A). Jika (I-A) nonsingular, maka (- λ) - adalah akar ciri dari (I-A) -. Jika -< λ <, maka (- λ) - =+ λ + λ +... Jika akar ciri A memenuhi -< λ <, maka (I-A) -.= I +A + A +A

53 Beberapa Teorema Akar Ciri dan Vektor Ciri 6. Jika matriks A berukuran (n x n) dengan akar ciri λ,..., λ n maka a. ǀAǀ= λ i b. tr(a)= λ i Bukti : a a a 3 a a a 3 a3 a3 a 33 (-λ) 3 + (-λ) tr (A) +(-λ) tr (A) + ǀAǀ= Dengan tr i (A)= jumlah minor utama, tr (A)= tr (A) tr (A )=ǀa a ǀ+ ǀa a 33 ǀ + ǀa a 33 ǀ tr 3 (A )=ǀa a a 33 ǀ

54 Beberapa Teorema Akar Ciri dan Vektor Ciri Bukti (lanjutan): Secara umum (-λ) n +(-λ) n- tr (A) +(-λ) n- tr (A) (-λ)tr n- (A) +ǀAǀ= Jika λ,..., λ n akar ciri dari persamaan tersebut maka (λ -λ) (λ -λ)... (λ n -λ)= (-λ) n +(-λ) n- λ i +(-λ) n- i j λ i λ j λ i = Sehingga λ i = tr(a) dan λ i =ǀAǀ

55 Beberapa Teorema Akar Ciri dan Vektor Ciri Jika A dan B berukuran (n x n) atau A berukuran (n x p) dan B berukuran (p x n), maka akar ciri (tak nol) AB sama dengan akar ciri BA. Jika x vektor ciri AB, maka Bx vektor ciri BA Jika A berukuran (n x n), maka. Jika P (n x n) nonsingular, maka A dan P - AP mempunyai akar ciri yang sama. Jika C (n x n) matriks ortogonal, A dan C AC mempunyai akar ciri yang sama

56 Teorema Matriks Simetrik. a. Akar ciri λ,..., λ n adalah real b. Vektor ciri x,..., x n bersifat ortogonal Bukti (a): Ambil λ bilangan kompleks dengan x vektor ciri padanannya Jika λ= a+ib dan λ*= a-ib, x={x i }= a+ib dan x*={x i *}= a-ib Maka A x = λ x x* A x =x* λx= λ x* x dan A x* = λ*x* x* Ax = (Ax *) x =(λ*x*) x=λ* x* x Sehingga λ* x* x = λ x* x,dan x* x adalah jumlah kuadrat λ* = λ atau a+ib= a-ib berarti b=

57 Teorema Matriks Simetrik Bukti (b): Misalkan λ λ dengan vektor ciri x x dan A=A, serta Ax k = λ x k λ x x = x λ x = x Ax = x A x = x Ax = x λ x = λ x x λ, λ, maka x x = (ortogonal). A dapat dinyatakan sebagai A=CDC (dekomposisi spektral) dengan D adalah matriks diagonal dengan unsur diagonalnya λ i dan C adalah matriks dengan unsur pada kolomnya x padanan akar ciri λ i

58 Teorema Matriks Simetrik 3. Matriks (semi) definit positif a. Jika A definit positif, maka λ i > untuk i=,...,n b. Jika A semi definit positif, maka λ i untuk i=,...,n. Banyaknya akar ciri λ i > sama dengan rank(a) Catatan : Jika A definit positif dapat ditentukan A ½. Karena λ i > maka pada dekomposisi spektral A= A ½ A ½ =(A ½ ) A = (A ½ ) = C D C=(C D ½ C) (C D ½ C)

59 Teorema Matriks Simetrik 4. Jika A singular, idempoten, dan simetrik, maka A semi definit positif Bukti : A= A dan A =A maka A =A =A A= A A (semi definit positif) 5. Jika A simetrik idempoten dengan rank (A)=r maka A mempunyai r akar ciri bernilai dan (n-r) akar ciri bernilai Bukti : Ax = λ x dan A x = λ x karena A =A A x = λ x Ax = λ x λx= λ x (λ- λ )x=. Karena x maka (λ- λ ) = λ bernilai atau.berdasarkan teorema (4), maka A semi definit positif dengan r menyatakan banyaknya λ>

60 Teorema Matriks Simetrik 6. Jika A idempoten dan simetrik dengan pangkat r, maka rank(a)=tr(a)=r Bukti : Berdasarkan teorema (5), maka λ i = tr(a)

61 Teorema Jika A (n x n) matriks idempoten, P matriks nonsingular (n xn) dan C matriks ortogonal (n xn) maka : a. I-A idempoten b. A(I-A)= dan (I-A) A= c. P - A P idempoten d. C A C idempoten (jika A simetrik maka C A C idempoten simetrik Jika A (n x p) dengan rank(a)=r, A - adalah kebalikan umum A dan (A A) - adalah kebalikan umum (A A), maka A - A, A A - dan A(A A) - A idempoten

62 Quadratic Forms A Quadratic From is a function Q(x) = x Ax in k variables x,,x k where x x x and A is a k x k symmetric matrix. x k

63 Note that a quadratic form has only squared terms and crossproducts, and so can be written then Quadratic Forms Suppose we have x Q x i j x and A 4 x Q( x) = x'ax = x + 4x x - x k k a x x ij i j

64 Spectral Decomposition and Quadratic Forms Any k x k symmetric matrix can be expressed in terms of its k eigenvalueeigenvector pairs ( i, e i ) as A k i e e ' i i i This is referred to as the spectral decomposition of A.

65 Spectral Decomposition and Quadratic Forms For our previous example on eigenvalues and eigenvectors we showed that A has eigenvalues = -6 and = -4, with corresponding (normalized) eigenvectors e 5, e 5, - 5 5

66 A Can we reconstruct A? k i Spectral Decomposition and e e ' i i i Quadratic Forms A

67 Spectral Decomposition and Quadratic Forms Spectral decomposition can be used to develop/illustrate many statistical results/ concepts. We start with a few basic concepts: - Nonnegative Definite Matrix when any k x k matrix A such that x Ax x =[x, x,, x k ] the matrix A and the quadratic form are said to be nonnegative definite.

68 Spectral Decomposition and Quadratic Forms - Positive Definite Matrix when any k x k matrix A such that < x Ax x =[x, x,, x k ] [,,, ] the matrix A and the quadratic form are said to be positive definite.

69 Spectral Decomposition and Quadratic Forms Example - Show that the following quadratic form is positive definite: 6x + 4x - 4 xx We first rewrite the quadratic form in matrix notation: Q( x) = x x 6 - x = x' Ax - 4 x

70 Spectral Decomposition and Quadratic Forms Now identify the eigenvalues of the resulting matrix A (they are = and = 8). A I or

71 Spectral Decomposition and Quadratic Forms Next, using spectral decomposition we can write: k ' ' ' ' ' i i i 8 i A e e e e e e e e e e where again, the vectors e i are the normalized and orthogonal eigenvectors associated with the eigenvalues = and = 8.

72 A k i Spectral Decomposition and e e ' i i i Quadratic Forms Sidebar - Note again that we can recreate the original matrix A from the spectral decomposition: A

73 Spectral Decomposition and Because and are scalars, premultiplication and postmultiplication by x and x, respectively, yield: ' ' ' ' ' x Ax x e e x 8x e e x y + 8y where Quadratic Forms ' ' ' ' y x e e x and y x e e x At this point it is obvious that x Ax is at least nonnegative definite!

74 Spectral Decomposition and Quadratic Forms We now show that x Ax is positive definite, i.e. x Ax y + 8y ' From our definitions of y and y we have ' y e x ' y e x or y Ex

75 Since E is an orthogonal matrix, E exists. Thus, Spectral Decomposition and Quadratic Forms x E ' y But x = E y implies y. At this point it is obvious that x Ax is positive definite!

76 Spectral Decomposition and Quadratic Forms This suggests rules for determining if a k x k symmetric matrix A (or equivalently, its quadratic form x Ax) is nonegative definite or positive definite: - A is a nonegative definite matrix iff i, i =,,rank(a) - A is a positive definite matrix iff i >, i =,,rank(a)

77 Square Root Matrices Because spectral decomposition allows us to express the inverse of a square matrix in terms of its eigenvalues and eigenvectors, it enables us to conveniently create a square root matrix. Let A be a p x p positive definite matrix with the spectral decomposition A k i e e ' i i i

78 Also let P be a matrix whose columns are the normalized eigenvectors e, e,, e p of A, i.e., Then Square Root Matrices P e e e p k ' ' i i i i A e e P P where P P = PP = I and p

79 Now since (P - P )PP =PP (P - P )=PP =I we have Next let Square Root Matrices A P P e e - ' k ' i i i i p

80 Square Root Matrices The matrix k ' ' P P ieiei A i is called the square root of A.

81 Square Root Matrices The square root of A has the following properties: ' A A A A A - - A A A A I A A A where A A -

82 Next let - denote the matrix matrix whose columns are the normalized eigenvectors e, e,, e p of A, i.e., Then Square Root Matrices P e e e p k ' ' i i i i A e e P P where P P = PP = I and p