Linear Algebra (Math 2890) Solution to Final Review Problems

1. Let A be the matrix

A = [\begin{matrix}2 & 1 & 1 \\ 1 & 2 & 1 \\ 1 & 1 & 2 \end{matrix}] .

.

(a) Prove that det(A − λI) = (1 − λ)^{2}(4 − λ).

Solution: Compute A − λI = [\begin{matrix}2 − λ & 1 & 1 \\ 1 & 2 − λ & 1 \\ 1 & 1 & 2 − λ \end{matrix}] and

det(A − λI) = (2 − λ)^{3} + 1 + 1 − (2 − λ) − (2 − λ) − (2 − λ) = 8−12 λ+6 λ^{2}−λ^{3}+2−6+3λ = −λ^{3}+6λ^{2}−9λ+4 = (1−λ)^{2}(4−λ).

(b) Orthogonally diagonalizes the matrix A, giving an orthogonal matrix P and a diagonal matrix D such that A = P DP^{t} Solution: We know that the eigenvalues are 1 ,1 and 4.

When λ = 1, A − (1)I = [ \begin{matrix}1 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 1 \end{matrix} ] [ \begin{matrix}1 & 1 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ]

x ∈ Null(A − I) if x_{1} + x_{2} + x_{3} = 0. So x_{1} = −x_{2} − x_{3} and

x = [\begin{matrix}−x_{2} − x_{3} \\ x_{2} \\ x_{3} \end{matrix}] = x_{2} [\begin{matrix}−1 \\ 1 \\ 0 \end{matrix}]+x_{3} [\begin{matrix}−1 \\ 0 \\ 1 \end{matrix}]. Thus \{w_{1} = [\begin{matrix}−1 \\ 1 \\ 0 \end{matrix}] , w_{2} =

[\begin{matrix}−1 \\ 0 \\ 1 \end{matrix}]\} is a basis of eigenvectors when λ = −1. Now we use Gram-Schmidt process to find an orthogonal basis for Null(A − I).

Let v_{1} = w_{1} = [\begin{matrix}−1 \\ 1 \\ 0 \end{matrix}] and v_{2} = w_{2} − \frac{w_{2}·v_{1}}{v_{1}·v_{1}} v_{1}. Compute w_{2} · v_{1} =

[\begin{matrix}−1 \\ 0 \\ 1 \end{matrix}] · [\begin{matrix}−1 \\ 1 \\ 0 \end{matrix}] = 1 and v_{1} · v_{1} = [\begin{matrix}−1 \\ 1 \\ 0 \end{matrix}] · [\begin{matrix}−1 \\ 1 \\ 0 \end{matrix}] = 2.

So v_{2} = [\begin{matrix}−1 \\ 0 \\ 1 \end{matrix}] − (\frac{1}{2}) [\begin{matrix}−1 \\ 1 \\ 0 \end{matrix}] = [\begin{matrix}−\frac{1}{2} \\ −\frac{1}{2} \\ 1 \end{matrix}].

Hence \{v_{1} = [\begin{matrix}−1 \\ 1 \\ 0 \end{matrix}] , v_{2} = [\begin{matrix}−\frac{1}{2} \\ −\frac{1}{2} \\ 1 \end{matrix}]\}is an orthogonal basis for Null(A− I).

When λ = 4, A − 4I = [ \begin{matrix}−2 & 1 & 1 \\ 1 & −2 & 1 \\ 1 & 1 & −2 \end{matrix} ] { \atop interchange r_{1} and r_{2},}

[ \begin{matrix}1 & −2 & 1 \\ −2 & 1 & 1 \\ 1 & 1 & −2 \end{matrix} ]
{ \atop −2 r_{1} + r_{2}, −r_{1} + r_{3}} [ \begin{matrix}1 & −2 & 1 \\ 0 & −3 & 3 \\ 0 & 3 & −3 \end{matrix} ]

{ \atop r_{2} + r_{3}, r_{2}/(−3)} [ \begin{matrix}1 & −2 & 1 \\ 0 & 1 & −1 \\ 0 & 0 & 0 \end{matrix} ] { \atop 2r_{2} + r_{1}} [ \begin{matrix}1 & 0 & −1 \\ 0 & 1 & −1 \\ 0 & 0 & 0 \end{matrix} ] x ∈ Null(A−

4I) if x_{1} − x_{3} = 0 and x_{2} − x_{3} = 0. So x = [\begin{matrix}x_{3} \\ x_{3} \\ x_{3} \end{matrix}] = x_{3} [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}]. Thus

\{v_{3} = [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}]\} is a basis for Null(A − 4I).

So \{v_{1} = [\begin{matrix}−1 \\ 1 \\ 0 \end{matrix}] , v_{2} = [\begin{matrix}−\frac{1}{2} \\ −\frac{1}{2} \\ 1 \end{matrix}] , v_{3} = [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}]\} is an orthogonal basis

for R^{3} which are eigenvectors corresponding to λ = 1, λ = 1 and λ = 4. Compute ||v_{1}|| = \sqrt{2}, ||v_{2}|| = \sqrt{\frac{1}{4} + \frac{1}{4} + 1} = \sqrt{\frac{6}{4}} = \sqrt{\frac{3}{2}} and ||v_{3}|| = \sqrt{3}.

Thus \{\frac{v_{1}}{||v_{1}||} = [\frac{−1}{\sqrt{2}} ; \frac{1}{\sqrt{2}} ; 0] , \frac{v_{2}}{||v_{2}||} = [−\frac{1}{\sqrt{6}} ; −\frac{1}{\sqrt{6}} ; \frac{2}{\sqrt{6}} ] , \frac{v_{3}}{||v_{3}||} = [\frac{1}{\sqrt{3}} ; \frac{1}{\sqrt{3}} ; \frac{1}{\sqrt{3}} ]\} is an or-

thonormal basis for R^{3} which are eigenvectors corresponding to λ = 1, λ = 1 and λ = 4.

Finally, we have A = P [\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 4 \end{matrix}] P^{T} where P = [\frac{v_{1}}{||v_{1}||} \frac{v_{2}}{||v_{2}||} \frac{v_{3}}{||v_{3}||}] =

[\begin{matrix}\frac{−1}{\sqrt{2}} & −\frac{1}{\sqrt{6}} & \frac{1}{\sqrt{3}} \\ \frac{1}{\sqrt{2}} & −\frac{1}{\sqrt{6}} & \frac{1}{\sqrt{3}} \\ 0 & \frac{2}{\sqrt{6}} & \frac{1}{\sqrt{3}} \end{matrix} ].

(c) Write the quadratic form associated with A using variables x_{1}, x_{2}, and x_{3}? Solution: Recall that the quadratic form in x_{1}, x_{2} and x_{3} is Q_{A}(x) = x^{T} Ax = 2x_{1}^{2} + 2x_{2}^{2} + 2x_{3}^{2} + 2x_{1}x_{2} + 2x_{1}x_{3} + 2x_{2}x_{3}.

(d) Find A^{−1}, A^{10} and e^{A}. Solution:Recall that A = P DP^{T}. Then A^{−1} = P D^{−1}P^{T} =

P [\begin{matrix}1^{−1} & 0 & 0 \\ 0 & 1^{−1} & 0 \\ 0 & 0 & 4^{−1} \end{matrix}] P^{T} = P [\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & \frac{1}{4} \end{matrix}] P^{T}.

A^{10} = P [\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 4^{10} \end{matrix}] P^{T} and e^{A} = P [\begin{matrix}e & 0 & 0 \\ 0 & e & 0 \\ 0 & 0 & e^{4} \end{matrix}] P^{T}

(e) What’s A^{−5}([\begin{matrix}1 \\ 1 \\ 1 \end{matrix}])?

Solution: Note that v_{3} = [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] is an eigenvector with eigenvalue

4. So we have A([\begin{matrix}1 \\ 1 \\ 1 \end{matrix}]) = 4 [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] and A^{k}([\begin{matrix}1 \\ 1 \\ 1 \end{matrix}]) = 4^{k} [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}]. Hence

A^{−5}([\begin{matrix}1 \\ 1 \\ 1 \end{matrix}]) = 4^{−5} [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}].

(f) What is lim_{n→∞}A^{−n}? Recall that A = P DP^{T} and A^{−n} = P D^{−n}P^{T} =

P [\begin{matrix}1^{−n} & 0 & 0 \\ 0 & 1^{−n} & 0 \\ 0 & 0 & 4^{−n} \end{matrix}] P^{T} = P [\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 4^{−n} \end{matrix}] P^{T}. Note that lim_{n→∞} 4^{−n} =

0. So we have lim_{n→∞} A^{−n} = P [\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{matrix}] P^{T}.

2. Classify the quadratic forms for the following quadratic forms. Make a change of variable x = P y, that transforms the quadratic form into one with no cross term. Also write the new quadratic form.

(a) 9x_{1}^{2} − 8x_{1}x_{2} + 3x_{2}^{2}.

Let Q(x_{1}, x_{2}) = 9x_{1}^{2} − 8x_{1}x_{2} + 3x_{2}^{2} = x^{T} [\begin{matrix}9 & −4 \\ −4 & 3 \end{matrix}] x and A =

[\begin{matrix}9 & −4 \\ −4 & 3 \end{matrix}]. We want to orthogonally diagonalizes A.

Compute A − λI = [\begin{matrix}9 − λ & −4 \\ −4 & 3 − λ \end{matrix}] and det(A − λI) = (9 − λ)(3 −

λ) − 16 = λ^{2} − 12λ + 27 − 16 = λ^{2} − 12λ + 11 = (λ − 1)(λ − 11). So λ = 1 or λ = 11. Since the eigenvalues of A are all positive, we know that the quadratic form is positive definite. Now we diagonalize A.

λ = 1: A − 1 · I = [\begin{matrix}9 − 1 & −4 \\ −4 & 3 − 1 \end{matrix}] = [\begin{matrix}8 & −4 \\ −4 & 2 \end{matrix}] [\begin{matrix}2 & −1 \\ 0 & 0 \end{matrix}]. So

x ∈ Null(A − 1 · I) iff 2x_{1} − x_{2} = 0. So x_{2} = 2x_{1} and x = [\begin{matrix}x_{1} \\ 2x_{1} \end{matrix}] =

x_{1} [\begin{matrix}1 \\ 2 \end{matrix}]. So [\begin{matrix}1 \\ 2 \end{matrix}] is an eigenvector corresponding to eigenvalue λ =

1.

λ = 11: A − 11 · I = [\begin{matrix}9 − 11 & −4 \\ −4 & 3 − 11 \end{matrix}] = [\begin{matrix}−2 & −4 \\ −4 & −8 \end{matrix}] [\begin{matrix}1 & 2 \\ 0 & 0 \end{matrix}].

So x ∈ Null(A − 11 · I) iff x_{1} + 2x_{2} = 0. So x_{1} = −2x_{2} and

x = [\begin{matrix}−2x_{2} \\ x_{2} \end{matrix}] = x_{2} [\begin{matrix}−2 \\ 1 \end{matrix}]. So [\begin{matrix}−2 \\ 1 \end{matrix}] is an eigenvector corresponding

to eigenvalue λ = 11.

Now \{v_{1} = [\begin{matrix}1 \\ 2 \end{matrix}] , v_{2} = [\begin{matrix}−2 \\ 1 \end{matrix}]\} is an orthogonal basis. Compute

||v_{1}|| = \sqrt{5} and ||v_{2}|| = \sqrt{5}. Thus \{\frac{v_{1}}{||v_{1}||} = [\frac{1}{\sqrt{5}} ; \frac{2}{\sqrt{5}}] , \frac{v_{2}}{||v_{2}||} = [\frac{−2}{\sqrt{5}} ; \frac{1}{\sqrt{5}}]\} is

an orthonormal basis of eigenvectors. So we have A = P [\begin{matrix}1 & 0 \\ 0 & 11 \end{matrix}] P^{T}

where P = [\begin{matrix}\frac{1}{\sqrt{5}} & \frac{−2}{\sqrt{5}} \\ \frac{2}{\sqrt{5}} & \frac{1}{\sqrt{5}} \end{matrix}].

Now Q(x) = x^{T} Ax = x^{T} P [\begin{matrix}1 & 0 \\ 0 & 11 \end{matrix}] P^{T} x = y^{T} [\begin{matrix}1 & 0 \\ 0 & 11 \end{matrix}] y = y_{1}^{2} +

11y_{2}^{2} if y = P^{T} x. So P y = P P^{T} x, x = P y and P = [\begin{matrix}\frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}} \\ \frac{−2}{\sqrt{5}} & \frac{1}{\sqrt{5}} \end{matrix}].

Note that we have used the fact that P P^{T} = I.

(b) −5x_{1}^{2} + 4x_{1}x_{2} − 2x_{2}^{2}.

Let Q(x_{1}, x_{2}) = −5x_{1}^{2} + 4x_{1}x_{2} − 2x_{2}^{2} = x^{T} [\begin{matrix}−5 & 2 \\ 2 & −2 \end{matrix}] x and A =

[\begin{matrix}−5 & 2 \\ 2 & −2 \end{matrix}]. We want to orthogonally diagonalizes A.

Compute A − λI = [\begin{matrix}−5 − λ & 2 \\ 2 & −2 − λ \end{matrix}] and det(A − λI) = (−5 −

λ)(−2 − λ) − 4 = λ^{2} + 7λ + 10 − 4 = λ^{2} + 7λ + 6 = (λ + 1)(λ + 6). So λ = −1 or λ = −6. Since the eigenvalues of A are all negative, we know that the quadratic form is negative definite. Now we diagonalize A.

λ = −1: A−(−1)·I = [\begin{matrix}−5 − (−1) & 2 \\ 2 & −2 − (−1) \end{matrix}] = [\begin{matrix}−4 & 2 \\ 2 & −1 \end{matrix}] [\begin{matrix}2 & −1 \\ 0 & 0 \end{matrix}].

So x ∈ Null(A − 1 · I) iff 2x_{1} − x_{2} = 0. So x_{2} = 2x_{1} and x = [\begin{matrix}x_{1} \\ 2x_{1} \end{matrix}] = x_{1} [\begin{matrix}1 \\ 2 \end{matrix}]. So [\begin{matrix}1 \\ 2 \end{matrix}] is an eigenvector corresponding to eigenvalue λ = −1.

λ = −6: A−(−6)·I = [\begin{matrix}−5 − (−6) & 2 \\ 2 & (−2) − (−6) \end{matrix}] = [\begin{matrix}1 & 2 \\ 2 & 4 \end{matrix}] [\begin{matrix}1 & 2 \\ 0 & 0 \end{matrix}].

So x ∈ Null(A − 11 · I) iff x_{1} + 2x_{2} = 0. So x_{1} = −2x_{2} and x = [\begin{matrix}−2x_{2} \\ x_{2} \end{matrix}] = x_{2} [\begin{matrix}−2 \\ 1 \end{matrix}]. So [\begin{matrix}−2 \\ 1 \end{matrix}] is an eigenvector corresponding to eigenvalue λ = −6. Now \{v_{1} = [\begin{matrix}1 \\ 2 \end{matrix}] , v_{2} = [\begin{matrix}−2 \\ 1 \end{matrix}]\} is an orthogonal basis. Compute

||v_{1}|| = \sqrt{5} and ||v_{2}|| = \sqrt{5}. Thus \{\frac{v_{1}}{||v_{1}||} = [\frac{1}{\sqrt{5}} ; \frac{2}{\sqrt{5}}] , \frac{v_{2}}{||v_{2}||} = [\frac{−2}{\sqrt{5}} ; \frac{1}{\sqrt{5}}]\} is

an orthonormal basis of eigenvectors. So we have A = P [\begin{matrix}−1 & 0 \\ 0 & −6 \end{matrix}] P^{T}

where P = [\begin{matrix}\frac{1}{\sqrt{5}} & \frac{−2}{\sqrt{5}} \\ \frac{2}{\sqrt{5}} & \frac{1}{\sqrt{5}} \end{matrix}].

Now Q(x) = x^{T} Ax = x^{T} P [\begin{matrix}−1 & 0 \\ 0 & −6 \end{matrix}] P^{T} x = y^{T} [\begin{matrix}−1 & 0 \\ 0 & −6 \end{matrix}] y =

−y_{1}^{2} − 6y_{2}^{2} if y = P^{T} x. So P y = P P^{T} x, x = P y and P =

[\begin{matrix}\frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}} \\ \frac{−2}{\sqrt{5}} & \frac{1}{\sqrt{5}} \end{matrix}].

(c) 8x_{1}^{2} + 6x_{1}x_{2}.

Let Q(x_{1}, x_{2}) = 8x_{1}^{2} + 6x_{1}x_{2} = x^{T} [\begin{matrix}8 & 3 \\ 3 & 0 \end{matrix}] x and A = [\begin{matrix}8 & 3 \\ 3 & 0 \end{matrix}]. We

want to orthogonally diagonalizes A.

Compute A − λI = [\begin{matrix}8 − λ & 3 \\ 3 & 0 − λ \end{matrix}] and det(A − λI) = (8 − λ) ·

(−λ) − 9 = λ^{2} − 8λ − 9 = (λ + 1)(λ − 9). So λ = −1 or λ = 9. Since A has positive and negative eigenvalues, we know that the quadratic form is indefinite. Now we diagonalize A.

λ = −1: A − (−1) · I = [\begin{matrix}8 − (−1) & 3 \\ 3 & 0 − (−1) \end{matrix}] = [\begin{matrix}9 & 3 \\ 3 & 1 \end{matrix}] [\begin{matrix}3 & 1 \\ 0 & 0 \end{matrix}].

So x ∈ Null(A − 1 · I) iff 3x_{1} + x_{2} = 0. So x_{2} = −3x_{1} and x = [\begin{matrix}x_{1} \\ −3x_{1} \end{matrix}] = x_{1} [\begin{matrix}1 \\ −3 \end{matrix}]. So [\begin{matrix}1 \\ −3 \end{matrix}] is an eigenvector corresponding to eigenvalue λ = −1.

λ = 9: A − 9 · I = [\begin{matrix}8 − 9 & 3 \\ 3 & 0 − 9 \end{matrix}] = [\begin{matrix}−1 & 3 \\ 3 & −9 \end{matrix}] [\begin{matrix}1 & −3 \\ 0 & 0 \end{matrix}]. So

x ∈ Null(A − 9 · I) iff x_{1} − 3x_{2} = 0. So x_{1} = 3x_{2} and x = [\begin{matrix}3x_{2} \\ x_{2} \end{matrix}] =

x_{2} [\begin{matrix}3 \\ 1 \end{matrix}]. So [\begin{matrix}3 \\ 1 \end{matrix}] is an eigenvector corresponding to eigenvalue λ = 9. Now \{v_{1} = [\begin{matrix}1 \\ −3 \end{matrix}] , v_{2} = [\begin{matrix}3 \\ 1 \end{matrix}]\} is an orthogonal basis. Compute

||v_{1}|| = \sqrt{10} and ||v_{2}|| = \sqrt{10}. Thus \{\frac{v_{1}}{||v_{1}||} = [\frac{1}{\sqrt{10}} ; \frac{−3}{\sqrt{10}}] , \frac{v_{2}}{||v_{2}||} =

[\frac{3}{\sqrt{10}} ; \frac{1}{\sqrt{10}}]\} is an orthonormal basis of eigenvectors. So we have A =

P [\begin{matrix}−1 & 0 \\ 0 & 9 \end{matrix}] P^{T} where P = [\begin{matrix}\frac{1}{\sqrt{10}} & \frac{−3}{\sqrt{10}} \\ \frac{3}{\sqrt{10}} & \frac{1}{\sqrt{10}} \end{matrix}].

Now Q(x) = x^{T} Ax = x^{T} P [\begin{matrix}−1 & 0 \\ 0 & 9 \end{matrix}] P^{T} x = y^{T} [\begin{matrix}−1 & 0 \\ 0 & 9 \end{matrix}] y = −y_{1}^{2} +

9y_{2}^{2} if y = P^{T} x. So P y = P P^{T} x, x = P y and P [\begin{matrix}\frac{1}{\sqrt{10}} & \frac{−3}{\sqrt{10}} \\ \frac{3}{\sqrt{10}} & \frac{1}{\sqrt{10}} \end{matrix}].

3. (a) Find a 3 × 3 matrix A which is not diagonalizable?

Solution: Let A = [\begin{matrix}0 & 1 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{matrix}]. Then det(A − λI) = −λ^{3} and the

eigenvalues of A are zero.

A−0·I = [\begin{matrix}0 & 1 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{matrix}] ∼ [\begin{matrix}0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{matrix}]. The eigenvector x = [\begin{matrix}x_{1} \\ x_{2} \\ x_{3} \end{matrix}] satisfies

x_{2} = 0 and x_{3} = 0. The eigenvector is x == [\begin{matrix}x_{1} \\ 0 \\ 0 \end{matrix}] = x_{1} [\begin{matrix}1 \\ 0 \\ 0 \end{matrix}]. So there

is only one eigenvector for A and A is not diagonalizable.

(b) Give an example of a 2 × 2 matrix which is diagonalizable but not orthogonally diagonalizable?

Solution: Let A = [\begin{matrix}1 & 4 \\ 1 & 1 \end{matrix}]. Then det(A − λI) == [\begin{matrix}1 − λ & 4 \\ 1 & 1 − λ \end{matrix}] =

(1 − λ)^{2} − 4 = (1 − λ)^{2} − 2^{2} = (1 − λ − 2)(1 − λ + 2) = (−λ − 1)(3 − λ). So A has two distinct eigenvalues and A is diagonalizable. But A is not symmetric. So A is not orthogonally diagonalizable.

4. Let A = [ \begin{matrix}1 & 2 & 2 \\ 1 & 1 & 0 \\ 0 & 1 & 2 \\ −1 & 0 & −1 \end{matrix} ].

(a) Find the condition on b = [\begin{matrix}b_{1} \\ b_{2} \\ b_{3} \\ b_{4} \end{matrix}] such that Ax = b is solvable. Solution: Consider the augmented matrix [A b] = [ \begin{matrix}1 & 2 & 2 & b_{1} \\ 1 & 1 & 0 & b_{2} \\ 0 & 1 & 2 & b_{3} \\ −1 & 0 & −1 & b_{4} \end{matrix} ]

{ \atop a_{2} := a_{2} + (−1)a_{1}} [ \begin{matrix}1 & 2 & 2 & b_{1} \\ 0 & −1 & −2 & b_{2} − b_{1} \\ 0 & 1 & 2 & b_{3} \\ −1 & 0 & −1 & b_{4} \end{matrix} ]
{ \atop a_{4} := a_{4} + a_{1}} [ \begin{matrix}1 & 2 & 2 & b_{1} \\ 0 & −1 & −2 & b_{2} − b_{1} \\ 0 & 1 & 2 & b_{3} \\ 0 & 2 & 1 & b_{4} + b_{1} \end{matrix} ]
{ \atop a_{2} := −a_{2}} [ \begin{matrix}1 & 2 & 2 & b_{1} \\ 0 & 1 & 2 & −b_{2} + b_{1} \\ 0 & 1 & 2 & b_{3} \\ 0 & 2 & 1 & b_{4} + b_{1} \end{matrix} ]
{ \atop a_{3} := a_{3} − a_{2}, a_{4} := a_{4} − 2a_{2}} [ \begin{matrix}1 & 2 & 2 & b_{1} \\ 0 & 1 & 2 & −b_{2} + b_{1} \\ 0 & 0 & 0 & b_{3} + b_{2} − b_{1} \\ 0 & 0 & −3 & b_{4} − b_{1} + 2 b_{2} \end{matrix} ]

{ \atop a_{3} ↔ a_{4}} [ \begin{matrix}1 & 2 & 2 & b_{1} \\ 0 & 1 & 2 & −b_{2} + b_{1} \\ 0 & 0 & −3 & b_{4} − b_{1} + 2 b_{2} \\ 0 & 0 & 0 & b_{3} + b_{2} − b_{1} \end{matrix} ]

From here, we can see that Ax = b has a solution if b_{3} +b_{2} −b_{1} = 0.

(b) What is the column space of A? Solution: The column space is the subspace spanned by the column vectors. From the computation in (a), we know that the column vectors of

A are independent. So Col(A) = span\{[ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] , [ \begin{matrix}2 \\ 1 \\ 1 \\ 0 \end{matrix} ] , [ \begin{matrix}2 \\ 0 \\ 2 \\ −1 \end{matrix} ]\}.

(c) Describe the subspace col(A)^{⊥} and find an basis for col(A)^{⊥}. Solution: col(A)^{⊥} = {x|x · y = 0 for all y ∈ col(A)}

= \{[\begin{matrix}x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{matrix}] | [\begin{matrix}x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{matrix}] · [\begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix}] = 0, [\begin{matrix}x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{matrix}] · [\begin{matrix}2 \\ 1 \\ 1 \\ 0 \end{matrix}] = 0, [\begin{matrix}x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{matrix}] [\begin{matrix}2 \\ 0 \\ 2 \\ −1 \end{matrix}] = 0\}

= \{[\begin{matrix}x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{matrix}] |x_{1} + x_{2} − x_{4} = 0, 2x_{1} + x_{2} + x_{3} = 0, 2x_{1} + 2x_{3} − x_{4} = 0\}

Consider [ \begin{matrix}1 & 1 & 0 & −1 \\ 2 & 1 & 1 & 0 \\ 2 & 0 & 2 & −1 \end{matrix} ] { \atop r_{2} := r_{2} − 2r_{1}} [ \begin{matrix}1 & 1 & 0 & −1 \\ 0 & −1 & 1 & 2 \\ 2 & 0 & 2 & −1 \end{matrix} ]

{ \atop r_{3} := r_{3} − 2r_{1}} [ \begin{matrix}1 & 1 & 0 & −1 \\ 0 & −1 & 1 & 2 \\ 0 & −2 & 2 & 1 \end{matrix} ] { \atop r_{2} := −r_{2}} [ \begin{matrix}1 & 1 & 0 & −1 \\ 0 & 1 & −1 & −2 \\ 0 & −2 & 2 & 1 \end{matrix} ]

{ \atop r_{3} := r_{3} + 2r_{2}} [ \begin{matrix}1 & 1 & 0 & −1 \\ 0 & 1 & −1 & −2 \\ 0 & 0 & 0 & −3 \end{matrix} ] { \atop r_{1} := r_{1} − r_{2}} [ \begin{matrix}1 & 0 & 1 & 1 \\ 0 & 1 & −1 & −2 \\ 0 & 0 & 0 & −3 \end{matrix} ]

{ \atop r_{3} := r_{3}/(−3)} [ \begin{matrix}1 & 0 & 1 & 1 \\ 0 & 1 & −1 & −2 \\ 0 & 0 & 0 & 1 \end{matrix} ] { \atop r_{1} := r_{1} − r_{3}, r_{2} := r_{2} + 2r_{3}} [ \begin{matrix}1 & 0 & 1 & 0 \\ 0 & 1 & −1 & 0 \\ 0 & 0 & 0 & 1 \end{matrix} ]

So x_{1}+x_{3} = 0, x_{2}−x_{3} = 0 and x_{4} = 0, x_{3} is free. This implies that

x_{1} = −x_{3}, x_{2} = x_{3} , x_{4} = 0 and x = [ \begin{matrix}x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{matrix} ] = [ \begin{matrix}−x_{3} \\ x_{3} \\ x_{3} \\ 0 \end{matrix} ] = x_{3} [ \begin{matrix}−1 \\ 1 \\ 1 \\ 0 \end{matrix} ].

Hence col(A)^{⊥} = span\{[ \begin{matrix}−1 \\ 1 \\ 1 \\ 0 \end{matrix} ]\} and \{[ \begin{matrix}−1 \\ 1 \\ 1 \\ 0 \end{matrix} ]\} is a basis for col(A)^{⊥}.

The dimension of col(A)^{⊥} is 1.

(d) Use Gram-Schmidt process to find an orthogonal basis for the column of the matrix A. Solution:

Let w_{1} = [ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ], w_{2} = [ \begin{matrix}2 \\ 1 \\ 1 \\ 0 \end{matrix} ] and w_{3} = [ \begin{matrix}2 \\ 0 \\ 2 \\ −1 \end{matrix} ].

Gram-Schmidt process is v_{1} = w_{1}, v_{2} = w_{2} − \frac{w_{2}·v_{1}}{v_{1}·v_{1}} v_{1} and v_{3} = w_{3} − \frac{w_{3}·v_{1}}{v_{1}·v_{1}} v_{1} − \frac{w_{3}·v_{2}}{v_{2}·v_{2}} v_{2}.

So v_{1} = [ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] . Compute w_{2} · v_{1} = [ \begin{matrix}2 \\ 1 \\ 1 \\ 0 \end{matrix} ] · [ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] = 3, v_{1} · v_{1} =

[ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] · [ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] = 3 and v_{2} = [ \begin{matrix}2 \\ 1 \\ 1 \\ 0 \end{matrix} ] − \frac{3}{3} [ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] = [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ] .

Compute w_{3} · v_{1} = [ \begin{matrix}2 \\ 0 \\ 2 \\ −1 \end{matrix} ] · [ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] = 3, w_{3} · v_{2} = [ \begin{matrix}2 \\ 0 \\ 2 \\ −1 \end{matrix} ] · [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ] = 3,

v_{2} · v_{2} = [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ] · [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ] = 3 and

v_{3} = w_{3} − \frac{w_{3}·v_{1}}{v_{1}·v_{1}} v_{1} − \frac{w_{3}·v_{2}}{v_{2}·v_{2}} v_{2} = [ \begin{matrix}2 \\ 0 \\ 2 \\ −1 \end{matrix} ] − \frac{3}{3} [ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] − \frac{3}{3} [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ]

= [ \begin{matrix}2−1−1 \\ 0−1−0 \\ 2−0−1 \\ −1+1−1 \end{matrix} ] = [ \begin{matrix}0 \\ −1 \\ 1 \\ −1 \end{matrix} ]. Hence \{[ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] , [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ] , [ \begin{matrix}0 \\ −1 \\ 1 \\ −1 \end{matrix} ]\} is an orthog-

onal basis for Col(A).

(e) Find an orthonormal basis for the column of the matrix A. Solution: Note that ||v_{1}|| = \sqrt{v_{1} · v_{1}} = \sqrt{3}, ||v_{2}|| = \sqrt{v_{2} · v_{2}} = \sqrt{3} and

||v_{3}|| = \sqrt{v_{3} · v_{3}} = \sqrt{3}. Hence \{\frac{v_{1}}{||v_{1}||}, \frac{v_{2}}{||v_{2}||}, \frac{v_{3}}{||v_{3}||}\} = \{[ \frac{1}{\sqrt{3}} ; \frac{1}{\sqrt{3}} ; 0 ; − \frac{1}{\sqrt{3}} ] , [ \frac{1}{\sqrt{3}} ; 0 ; \frac{1}{\sqrt{3}} ; \frac{1}{\sqrt{3}} ] , [ 0 ; − \frac{1}{\sqrt{3}} ; \frac{1}{\sqrt{3}} ; − \frac{1}{\sqrt{3}} ]\}

is an orthonormal basis for Col(A).

(f) Find the orthogonal projection of y = [ \begin{matrix}7 \\ 3 \\ 10 \\ −2 \end{matrix} ] onto the column

space of A and write y = ŷ+ z where ŷ ∈ col(A) and z ∈ col(A)^{⊥}. Also find the shortest distance from y to Col(A).

Solution: Since \{v_{1} = [ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] , v_{2} = [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ] , v_{3} = [ \begin{matrix}0 \\ −1 \\ 1 \\ −1 \end{matrix} ]\} is an

orthogonal basis for Col(A), y = ŷ + z where ŷ = \frac{y·v_{1}}{v_{1}·v_{1}} v_{1} + \frac{y·v_{2}}{v_{2}·v_{2}} v_{2} + \frac{y·v_{3}}{v_{3}·v_{3}} v_{3} ∈ Col(A) and z = y − ŷ ∈ Col(A)^{⊥}. Compute

y·v_{1} = [ \begin{matrix}7 \\ 3 \\ 10 \\ −2 \end{matrix} ]·[ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] = 7+3+0+2 = 12, v_{1} ·v_{1} = [ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ]·[ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] =

1 + 1 + 1 = 3, y · v_{2} = [ \begin{matrix}7 \\ 3 \\ 10 \\ −2 \end{matrix} ] · [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ] = 7 + 0 + 10 − 2 = 15,

v_{2} · v_{2} = [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ] · [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ] = 3,

y·v_{3} = [ \begin{matrix}7 \\ 3 \\ 10 \\ −2 \end{matrix} ]·[ \begin{matrix}0 \\ −1 \\ 1 \\ −1 \end{matrix} ] = 0−3+10+2 = 9, v_{3}·v_{3} = [ \begin{matrix}0 \\ −1 \\ 1 \\ −1 \end{matrix} ]·[ \begin{matrix}0 \\ −1 \\ 1 \\ −1 \end{matrix} ] = 3.

So ŷ = \frac{12}{3} [ \begin{matrix}1 \\ 1 \\ 0 \\ −1 \end{matrix} ] + \frac{(15)}{3} [ \begin{matrix}1 \\ 0 \\ 1 \\ 1 \end{matrix} ] + \frac{9}{3} [ \begin{matrix}0 \\ −1 \\ 1 \\ −1 \end{matrix} ] = [ \begin{matrix}4+5+0 \\ 4+0−3 \\ 0+5+3 \\ −4+5−3 \end{matrix} ] = [ \begin{matrix}9 \\ 1 \\ 8 \\ −2 \end{matrix} ] and

z = y − ŷ = [ \begin{matrix}7 \\ 3 \\ 10 \\ −2 \end{matrix} ] − [ \begin{matrix}9 \\ 1 \\ 8 \\ −2 \end{matrix} ] = [ \begin{matrix}2 \\ −2 \\ 2 \\ 0 \end{matrix} ]. Note that z ∈ Col(A)^{⊥} =

span\{[ \begin{matrix}1 \\ −1 \\ 1 \\ 0 \end{matrix} ]\}.

The shortest distance from y to Col(A) = ||y − ŷ|| = ||z|| = \sqrt{(2)^{2} + (−2)^{2} + (2)^{2} + (0)^{2}} = \sqrt{12}.

(g) Using previous result to explain why Ax = y has no solution. Solution: Since the orthogonal projection of y to Col(A) is not y, this implies that y is not in Col(A). So Ax = y has no solution.

(h) Use orthogonal projection to find the least square solution of Ax = y. Solution: The least square solution of Ax = y is the solution of

Ax = ŷ = [ \begin{matrix}9 \\ 1 \\ 8 \\ −2 \end{matrix} ] where ŷ is the orthogonal projection of y onto

the column space of A (from part (f), we know ŷ = [ \begin{matrix}9 \\ 1 \\ 8 \\ −2 \end{matrix} ].)

Consider the augmented matrix

[A ŷ] = [ \begin{matrix}1 & 2 & 2 & 9 \\ 1 & 1 & 0 & 1 \\ 0 & 1 & 2 & 8 \\ −1 & 0 & −1 & −2 \end{matrix} ] { \atop r_{2} := r_{2} − r_{1}, r_{3} := r_{3} + r_{1}} [ \begin{matrix}1 & 2 & 2 & 9 \\ 0 & −1 & −2 & −8 \\ 0 & 1 & 2 & 8 \\ 0 & 2 & 1 & 7 \end{matrix} ]

{ \atop r_{3} := r_{3} + r_{2}, r_{4} := r_{4} + r_{1}} [ \begin{matrix}1 & 2 & 2 & 9 \\ 0 & −1 & −2 & −8 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & −3 & −9 \end{matrix} ]

{ \atop r_{2} := −r_{2}, r_{4} := r_{4}/(−3), r_{3} ↔ r_{4}} [ \begin{matrix}1 & 2 & 2 & 9 \\ 0 & 1 & 2 & 8 \\ 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & 0 \end{matrix} ]

{ \atop r_{2} := r_{2} − 2r_{3}, r_{1} := r_{1} − 2r_{3}} [ \begin{matrix}1 & 2 & 0 & 3 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & 0 \end{matrix} ]

{ \atop r_{1} := r_{1} − 2r_{2}} [ \begin{matrix}1 & 0 & 0 & −1 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & 0 \end{matrix} ].

So x_{1} = −1, x_{2} = 2, x_{3} = 3 and the least square solution of

Ax = y is x = [ \begin{matrix}−1 \\ 2 \\ 3 \end{matrix} ] .

(i) Use normal equation to find the least square solution of Ax = y. Solution: The normal equation is A^{T} Ax = A^{T} y. Compute A^{T} A =

[ \begin{matrix}1 & 1 & 0 & −1 \\ 2 & 1 & 1 & 0 \\ 2 & 0 & 2 & −1 \end{matrix} ] [ \begin{matrix}1 & 2 & 2 \\ 1 & 1 & 0 \\ 0 & 1 & 2 \\ −1 & 0 & −1 \end{matrix} ] = [ \begin{matrix}3 & 3 & 3 \\ 3 & 6 & 6 \\ 3 & 6 & 9 \end{matrix} ]

and A^{T} y = [ \begin{matrix}1 & 1 & 0 & −1 \\ 2 & 1 & 1 & 0 \\ 2 & 0 & 2 & −1 \end{matrix} ] [ \begin{matrix}7 \\ 3 \\ 10 \\ −2 \end{matrix} ] = [ \begin{matrix}12 \\ 27 \\ 36 \end{matrix} ].

So the normal equation A^{T} Ax = A^{T} y is

[ \begin{matrix}3 & 3 & 3 \\ 3 & 6 & 6 \\ 3 & 6 & 9 \end{matrix} ] x = [ \begin{matrix}12 \\ 27 \\ 36 \end{matrix} ].

Consider the augmented matrix [ \begin{matrix}3 & 3 & 3 & 12 \\ 3 & 6 & 6 & 27 \\ 3 & 6 & 9 & 36 \end{matrix} ] ∼

r_{2} := r_{2} − r_{1}, r_{3} := r_{3} − r_{1} [ \begin{matrix}3 & 3 & 3 & 12 \\ 0 & 3 & 3 & 15 \\ 0 & 3 & 6 & 24 \end{matrix} ]

∼ r_{3} := r_{3} − r_{2} [ \begin{matrix}3 & 3 & 3 & 12 \\ 0 & 3 & 3 & 15 \\ 0 & 0 & 3 & 9 \end{matrix} ] ∼ r_{1} := r_{1}/3, r_{2} := r_{2}/3, r_{3} :=

r_{3}/3 [ \begin{matrix}1 & 1 & 1 & 4 \\ 0 & 1 & 1 & 5 \\ 0 & 0 & 1 & 3 \end{matrix} ]

∼ r_{2} := r_{2} − r_{3}, r_{1} := r_{1} − r_{3} [ \begin{matrix}1 & 1 & 0 & 1 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 3 \end{matrix} ]

∼ r_{1} := r_{1} − r_{2}, [ \begin{matrix}1 & 0 & 0 & −1 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 3 \end{matrix} ]

So x_{1} = −1, x_{2} = 2, x_{3} = 3 and the least square solution of

Ax = y is x = [ \begin{matrix}−1 \\ 2 \\ 3 \end{matrix} ] .

5. Find the equation y = a + mx of the least square line that best fits the given data points. (0, 1), (1, 1), (3, 2).

Solution: We try to solve the equations 1 = a, 1 = a + m, 2 = a + 3m, that is, a = 1, a+m = 1 and a+3m = 2. It corresponding to the linear system

[ \begin{matrix}1 & 0 \\ 1 & 1 \\ 1 & 3 \end{matrix} ] [ \begin{matrix}a \\ m \end{matrix} ] = [ \begin{matrix}1 \\ 1 \\ 2 \end{matrix} ]

Let A = [ \begin{matrix}1 & 0 \\ 1 & 1 \\ 1 & 3 \end{matrix} ]. We solve the normal equation

A^{T} A [ \begin{matrix}a \\ m \end{matrix} ] = A^{T} [ \begin{matrix}1 \\ 1 \\ 2 \end{matrix} ].

Compute A^{T} A = [ \begin{matrix}1 & 1 & 1 \\ 0 & 1 & 3 \end{matrix} ] [ \begin{matrix}1 & 0 \\ 1 & 1 \\ 1 & 3 \end{matrix} ] = [ \begin{matrix}3 & 4 \\ 4 & 10 \end{matrix} ] and

A^{T} [ \begin{matrix}1 \\ 1 \\ 2 \end{matrix} ] = [ \begin{matrix}1 & 1 & 1 \\ 0 & 1 & 3 \end{matrix} ] [ \begin{matrix}1 \\ 1 \\ 2 \end{matrix} ] = [ \begin{matrix}4 \\ 7 \end{matrix} ].

Consider the augmented matrix

[ \begin{matrix}3 & 4 & 4 \\ 4 & 10 & 7 \end{matrix} ] ∼ r_{2} := r_{2} − \frac{4}{3}r_{1} [ \begin{matrix}3 & 4 & 4 \\ 0 & \frac{14}{3} & \frac{5}{3} \end{matrix} ]

∼ r_{2} := \frac{3}{14}r_{2} [ \begin{matrix}3 & 4 & 4 \\ 0 & 1 & \frac{5}{14} \end{matrix} ] ∼ r_{1} := r_{1} − 4r_{2} [ \begin{matrix}3 & 0 & \frac{18}{7} \\ 0 & 1 & \frac{5}{14} \end{matrix} ]

∼ r_{1} := r_{1}/3 [ \begin{matrix}1 & 0 & \frac{6}{7} \\ 0 & 1 & \frac{5}{14} \end{matrix} ]

So the least square solution is a = \frac{6}{7} and m = \frac{5}{14}. The equation y = \frac{6}{7} + \frac{5}{14}x is the least square line that best fits the given data points. (0, 1), (1, 1), (3, 2).

6. (a) Show that the set of vectors

B = \{u_{1} = (−\frac{3}{5}, \frac{4}{5}, 0) , u_{2} = (\frac{4}{5}, \frac{3}{5}, 0) , u_{3} = (0, 0, 1)\}

is an orthonormal basis of ℝ^{3}.

Solution: Compute u_{1} · u_{2} = (−\frac{3}{5}, \frac{4}{5}, 0) · (\frac{4}{5}, \frac{3}{5}, 0) = \frac{−12}{5} + \frac{12}{5} = 0, u_{1} · u_{3} = (−\frac{3}{5}, \frac{4}{5}, 0) · (0, 0, 1) = 0, u_{2} · u_{3} = (\frac{4}{5}, \frac{3}{5}, 0) · (0, 0, 1) = 0, u_{1}·u_{1} = (−\frac{3}{5}, \frac{4}{5}, 0)·(−\frac{3}{5}, \frac{4}{5}, 0) = \frac{9}{25} + \frac{16}{25} = 1, u_{3}·u_{3} = (0, 0, 1)·(0, 0, 1) = 1, u_{2} · u_{2} = (\frac{4}{5}, \frac{3}{5}, 0) · (\frac{4}{5}, \frac{3}{5}, 0) = \frac{16}{25} + \frac{9}{25} = 1

(b) Find the coordinates of the vector (1, −1, 2) with respect to the basis in (a).

Solution: Let y = (1, −1, 2). So y = \frac{y·u_{1}}{u_{1}·u_{1}} u_{1} + \frac{y·u_{2}}{u_{2}·u_{2}} u_{2} + \frac{y·u_{3}}{u_{3}·u_{3}} u_{3} = (y·u_{1})u_{1}+(y·u_{2})u_{2}+(y·u_{3})u_{3}. Compute y·u_{1} = (1, −1, 2)·(−\frac{3}{5}, \frac{4}{5}, 0) = −\frac{3}{5} − \frac{4}{5} = −\frac{7}{5}, y · u_{2} = (1, −1, 2) · (\frac{4}{5}, \frac{3}{5}, 0) = \frac{4}{5} − \frac{3}{5} = \frac{1}{5}, y · u_{3} = (1, −1, 2) · (0, 0, 1) = 2.

So the coordinate of y with respect to the basis in (a) is (−\frac{7}{5}, \frac{1}{5}, 2).

7. (a) Let A = [\begin{matrix}3 & 6 & 7 \\ 0 & 2 & 1 \\ 2 & 3 & 4 \end{matrix}] . Find the inverse matrix of A if possible.

Solution: Consider the augmented matrix [A I] = [ \begin{matrix}3 & 6 & 7 & 1 & 0 & 0 \\ 0 & 2 & 1 & 0 & 1 & 0 \\ 2 & 3 & 4 & 0 & 0 & 1 \end{matrix} ]

{ \atop r_{1} := r_{1} − r_{3}} [ \begin{matrix}1 & 3 & 3 & 1 & 0 & −1 \\ 0 & 2 & 1 & 0 & 1 & 0 \\ 2 & 3 & 4 & 0 & 0 & 1 \end{matrix} ]

{ \atop r_{3} := r_{3} − 2r_{1}} [ \begin{matrix}1 & 3 & 3 & 1 & 0 & −1 \\ 0 & 2 & 1 & 0 & 1 & 0 \\ 0 & −3 & −2 & −2 & 0 & 3 \end{matrix} ]

{ \atop r_{2} := r_{2} + r_{3}} [ \begin{matrix}1 & 3 & 3 & 1 & 0 & −1 \\ 0 & −1 & −1 & −2 & 1 & 3 \\ 0 & −3 & −2 & −2 & 0 & 3 \end{matrix} ] { \atop r_{2} := −r_{2}} [ \begin{matrix}1 & 3 & 3 & 1 & 0 & −1 \\ 0 & 1 & 1 & 2 & −1 & −3 \\ 0 & −3 & −2 & −2 & 0 & 3 \end{matrix} ]

{ \atop r_{3} := r_{3} + 3r_{2}} [ \begin{matrix}1 & 3 & 3 & 1 & 0 & −1 \\ 0 & 1 & 1 & 2 & −1 & −3 \\ 0 & 0 & 1 & 4 & −3 & −6 \end{matrix} ]

{ \atop r_{2} := r_{2} − r_{3}, r_{1} := r_{1} − 3r_{3}} [ \begin{matrix}1 & 3 & 0 & −11 & 9 & 17 \\ 0 & 1 & 0 & −2 & 2 & 3 \\ 0 & 0 & 1 & 4 & −3 & −6 \end{matrix} ]

{ \atop r_{1} := r_{1} − 3r_{2}} [ \begin{matrix}1 & 0 & 0 & −5 & 3 & 8 \\ 0 & 1 & 0 & −2 & 2 & 3 \\ 0 & 0 & 1 & 4 & −3 & −6 \end{matrix} ].

So A^{−1} = [ \begin{matrix}−5 & 3 & 8 \\ −2 & 2 & 3 \\ 4 & −3 & −6 \end{matrix} ].

(b) Find the coordinates of the vector (1, −1, 2) with respect to the basis B obtained from the column vectors of A.

Solution: The coordinate is x = A^{−1} [ \begin{matrix}1 \\ −1 \\ 2 \end{matrix} ] = [ \begin{matrix}−5 & 3 & 8 \\ −2 & 2 & 3 \\ 4 & −3 & −6 \end{matrix} ] [ \begin{matrix}1 \\ −1 \\ 2 \end{matrix} ] =

[ \begin{matrix}8 \\ 2 \\ −5 \end{matrix} ].

8. Let H = \{ [\begin{matrix}a + 2b − c \\ a − b − 4c \\ a + b − 2c \end{matrix}] : a, b, cany real numbers\}.

a. Explain why H is a a subspace of R^{3}.

Solution: [\begin{matrix}a + 2b − c \\ a − b − 4c \\ a + b − 2c \end{matrix}] = a [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] + b [\begin{matrix}2 \\ −1 \\ 1 \end{matrix}] + c [\begin{matrix}−1 \\ −4 \\ −2 \end{matrix}]

So H = Span\{[\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] , [\begin{matrix}2 \\ −1 \\ 1 \end{matrix}] , [\begin{matrix}−1 \\ −4 \\ −2 \end{matrix}]\} and H is a subspace.

b. Find a set of vectors that spans H.

Solution:\{[\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] , [\begin{matrix}2 \\ −1 \\ 1 \end{matrix}] , [\begin{matrix}−1 \\ −4 \\ −2 \end{matrix}]\} spans the space H.

c. Find a basis for H.

Solution: Consider the matrix A = [\begin{matrix}1 & 2 & −1 \\ 1 & −1 & −4 \\ 1 & 1 & −2 \end{matrix}]

{ \atop r_{2} := r_{2} − r_{1}, r_{3} := r_{3} − r_{1}} [\begin{matrix}1 & 2 & −1 \\ 0 & −3 & −3 \\ 0 & −1 & −1 \end{matrix}]

{ \atop r_{2} := r_{2}/(−3)} [\begin{matrix}1 & 2 & −1 \\ 0 & 1 & 1 \\ 0 & −1 & −1 \end{matrix}] { \atop r_{3} := r_{3} + r_{2}} [\begin{matrix}1 & 2 & −1 \\ 0 & 1 & 1 \\ 0 & 0 & 0 \end{matrix}].

So the first two vectors are pivot vectors and \{[\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] , [\begin{matrix}2 \\ −1 \\ 1 \end{matrix}]\} is a basis.

The dimension of the subspace is 2.

d. What is the dimension of the subspace? Solution:The dimension of the subspace is 2. e. Find an orthogonal basis for H.

Solution: Let u_{1} = [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] and u_{2} = [\begin{matrix}2 \\ −1 \\ 1 \end{matrix}].

Then v_{1} = u_{1} = [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] and v_{2} = u_{2} − \frac{u_{2}·v_{1}}{v_{1}·v_{1}} v_{1}. Compute u_{2} · v_{1} =

[\begin{matrix}2 \\ −1 \\ 1 \end{matrix}] · [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] = 2 − 1 + 1 = 2 and v_{1} · v_{1} = [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] · [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] = 1 + 1 + 1 = 3.

v_{2} = [\begin{matrix}2 \\ −1 \\ 1 \end{matrix}] − \frac{2}{3} [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] = [−\frac{5}{3}_{\frac{1}{3}}^{\frac{4}{3}} ]. Thus \{v_{1} = [\begin{matrix}1 \\ 1 \\ 1 \end{matrix}] , v_{2} = [−\frac{5}{3}_{\frac{1}{3}}^{\frac{4}{3}} ]\} ia an

orthogonal basis for H. We can verify that v_{1} · v_{2} = 0.

9. Determine if the following systems are consistent and if so give all solutions in parametric vector form. (a)

x_{1} −2x_{2} = 3
2x_{1} −7x_{2} = 0
−5x_{1} +8x_{2} = 5

Solution: The augmented matrix is [ \begin{matrix}1 & −2 & 3 \\ 2 & −7 & 0 \\ −5 & 8 & 5 \end{matrix} ] ∼ (r_{2} := r_{2} −2r_{1})

[ \begin{matrix}1 & −2 & 3 \\ 0 & −3 & −6 \\ −5 & 8 & 5 \end{matrix} ] ∼ (r_{3} := r_{3} + 5r_{1}) [ \begin{matrix}1 & −2 & 3 \\ 0 & −3 & −6 \\ 0 & −2 & 20 \end{matrix} ]

∼ (r_{2} := r_{2}/ − 3, r_{3} := r_{3}/ − 2) [ \begin{matrix}1 & −2 & 3 \\ 0 & 1 & 2 \\ 0 & 1 & −10 \end{matrix} ] ∼ (r_{3} := r_{3} −

r_{2}) [ \begin{matrix}1 & −2 & 3 \\ 0 & 1 & 2 \\ 0 & 0 & −12 \end{matrix} ]. The last row implies that 0 = −12 which is impossible. So this system is inconsistent. (b)

\begin{aligned}x_{1} & +2x_{2} & −3x_{3} & +x_{4} & = 1 \\ −x_{1} & −2x_{2} & +4x_{3} & −x_{4} & = 6 \\ −2x_{1} & −4x_{2} & +7x_{3} & −x_{4} & = 1 \end{aligned}

The augmented matrix is [ \begin{matrix}1 & 2 & −3 & 1 & 1 \\ −1 & −2 & 4 & −1 & 6 \\ −2 & −4 & 7 & −1 & 1 \end{matrix} ] ∼ (r_{2} := r_{2} + r_{1})

[ \begin{matrix}1 & 2 & −3 & 1 & 1 \\ 0 & 0 & 1 & 0 & 7 \\ −2 & −4 & 7 & −1 & 1 \end{matrix} ] ∼ (r_{3} := r_{3} + 2r_{1}) [ \begin{matrix}1 & 2 & −3 & 1 & 1 \\ 0 & 0 & 1 & 0 & 7 \\ 0 & 0 & 1 & 1 & 3 \end{matrix} ]

∼ (r_{3} := r_{3}−r_{2}) [ \begin{matrix}1 & 2 & −3 & 1 & 1 \\ 0 & 0 & 1 & 0 & 7 \\ 0 & 0 & 0 & 1 & −4 \end{matrix} ] ∼ (r_{1} := r_{1}−r_{3}) [ \begin{matrix}1 & −2 & 3 \\ 0 & 1 & 2 \\ 0 & 0 & −12 \end{matrix} ]

∼ (r_{1} := r_{1}−r_{3}) [ \begin{matrix}1 & 2 & −3 & 0 & 5 \\ 0 & 0 & 1 & 0 & 7 \\ 0 & 0 & 0 & 1 & −4 \end{matrix} ]∼ (r_{1} := r_{1}+3r_{2}) [ \begin{matrix}1 & 2 & 0 & 0 & 26 \\ 0 & 0 & 1 & 0 & 7 \\ 0 & 0 & 0 & 1 & −4 \end{matrix} ].

So x_{2} is free. The solution is x_{1} = 26 − 2x_{2}, x_{3} = 7, x_{4} = −47. Its

parametric vector form is [\begin{matrix}x_{1} \\ x_{2} \\ x_{4} \\ x_{4} \end{matrix}] = [\begin{matrix}26 − 2x_{2} \\ x_{2} \\ 7 \\ −4 \end{matrix}] = [\begin{matrix}26 \\ 0 \\ 7 \\ −4 \end{matrix}] + x_{2} [\begin{matrix}−2 \\ 1 \\ 0 \\ 0 \end{matrix}].

10. Let A = [ \begin{matrix}1 & −3 & 4 & −2 & 5 \\ 2 & −6 & 9 & −1 & 8 \\ 2 & −6 & 9 & −1 & 9 \\ −1 & 3 & −4 & 2 & −5 \end{matrix} ] which is row reduced to [ \begin{matrix}1 & −3 & −2 & −20 & −3 \\ 0 & 0 & 1 & 3 & 3 \\ 0 & 0 & 1 & 3 & 4 \\ 0 & 0 & 0 & 0 & 0 \end{matrix} ]

(a) Find a basis for the column space of A

(b) Find a basis for the nullspace of A

(c) Find the rank of the matrix A

(d) Find the dimension of the nullspace of A.

(e) Is [\begin{matrix}1 \\ 4 \\ 3 \\ 1 \end{matrix}] in the range of A?

(e) Does Ax = [\begin{matrix}0 \\ 3 \\ 2 \\ 0 \end{matrix}] have any solution? Find a solution if it’s solvable.

Solution: Consider the augmented matrix [ \begin{matrix}1 & −3 & 4 & −2 & 5 & 1 & 0 \\ 2 & −6 & 9 & −1 & 8 & 4 & 3 \\ 2 & −6 & 9 & −1 & 9 & 3 & 2 \\ −1 & 3 & −4 & 2 & −5 & 1 & 0 \end{matrix} ]

{ \atop −2r_{1} + r_{2}, −2r_{1} + r_{3}, r_{1} + r_{4}}[ \begin{matrix}1 & −3 & 4 & −2 & 5 & 1 & 0 \\ 0 & 0 & 1 & 3 & −2 & 2 & 3 \\ 0 & 0 & 1 & 3 & −1 & 1 & 2 \\ 0 & 0 & 0 & 0 & 0 & 2 & 0 \end{matrix} ]

{ \atop −r_{2} + r_{3}} [ \begin{matrix}1 & −3 & 4 & −2 & 5 & 1 & 0 \\ 0 & 0 & 1 & 3 & −2 & 2 & 3 \\ 0 & 0 & 0 & 0 & 1 & −1 & −1 \\ 0 & 0 & 0 & 0 & 0 & 2 & 0 \end{matrix} ]

{ \atop 2r_{3} + r_{2}, −5r_{3} + r_{1}} [ \begin{matrix}1 & −3 & 4 & −2 & 0 & 6 & 5 \\ 0 & 0 & 1 & 3 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 1 & −1 & −1 \\ 0 & 0 & 0 & 0 & 0 & 2 & 0 \end{matrix} ]

{ \atop −4r_{2} + r_{1}} [ \begin{matrix}1 & −3 & 0 & −14 & 0 & 6 & 1 \\ 0 & 0 & 1 & 3 & 0 & 0 & 1 \\ 0 & 0 & 0 & 0 & 1 & −1 & −1 \\ 0 & 0 & 0 & 0 & 0 & 2 & 0 \end{matrix} ].

So the first, third and fifth vector forms a basis for Col(A), i.e \{[ \begin{matrix}1 \\ 2 \\ 2 \\ −1 \end{matrix} ] , [ \begin{matrix}4 \\ 9 \\ 9 \\ −4 \end{matrix} ] , [ \begin{matrix}5 \\ 8 \\ 9 \\ −5 \end{matrix} ]\}

is a basis for Col(A). The rank of A is 3 and the dimension of the null space is 5 − 3 = 2. x ∈ Null(A) if x_{1} − 3x_{2} − 14x_{4} = 0, x_{3} + 3x_{4} = 0 and x_{5} = 0. So

x = [\begin{matrix}3x_{2} + 14x_{4} \\ x_{2} \\ −x_{4} \\ x_{4} \\ 0 \end{matrix}] = x_{2} [\begin{matrix}3 \\ 1 \\ 0 \\ 0 \\ 0 \end{matrix}] + x_{4} [\begin{matrix}14 \\ 0 \\ −1 \\ 1 \\ 0 \end{matrix}]. Thus\{[\begin{matrix}3 \\ 1 \\ 0 \\ 0 \\ 0 \end{matrix}] , [\begin{matrix}14 \\ 0 \\ −1 \\ 1 \\ 0 \end{matrix}] is a basis

for NULL(A).

From the result of row reduction, we can see that Ax = [\begin{matrix}1 \\ 4 \\ 3 \\ 1 \end{matrix}] is inconsistent (not solvable) and [\begin{matrix}1 \\ 4 \\ 3 \\ 1 \end{matrix}] is not in the range of A.

From the result of row reduction, we can see that Ax = [\begin{matrix}0 \\ 3 \\ 2 \\ 0 \end{matrix}] is solvable.

11. Determine if the columns of the matrix form a linearly independent set. Justify your answer.

[\begin{matrix}1 & 1 \\ 1 & 2 \end{matrix}], [\begin{matrix}1 & −2 \\ −2 & 4 \\ 3 & 6 \end{matrix}] , [\begin{matrix}−4 & −3 & 0 \\ 0 & −1 & 4 \\ 1 & 0 & 3 \\ 5 & 4 & 6 \end{matrix}] , [\begin{matrix}−4 & −3 & 1 & 5 & 1 \\ 2 & −1 & 4 & −1 & 2 \\ 1 & 2 & 3 & 6 & −3 \\ 5 & 4 & 6 & −3 & 2 \end{matrix}].

Solution: det [\begin{matrix}1 & 1 \\ 1 & 2 \end{matrix}] = 2 − 1 = 1 ≠ 0. So the columns of the matrix

form a linearly independent set.

[\begin{matrix}1 & −2 \\ −2 & 4 \\ 3 & 6 \end{matrix}]. The second column vector is a multiple of the first column

vector. So the columns of the matrix form a linearly dependent set.

[\begin{matrix}−4 & −3 & 0 \\ 0 & −1 & 4 \\ 1 & 0 & 3 \\ 5 & 4 & 6 \end{matrix}] { \atop interchange first and third row} [\begin{matrix}1 & 0 & 3 \\ 0 & −1 & 4 \\ −4 & −3 & 0 \\ 5 & 4 & 6 \end{matrix}]

{ \atop r_{3} + 4r_{1}, r_{4} + (−5)r_{1}} [\begin{matrix}1 & 0 & 3 \\ 0 & −1 & 4 \\ 0 & −3 & 12 \\ 0 & 4 & −9 \end{matrix}]  { \atop (−1)r_{2}} [\begin{matrix}1 & 0 & 3 \\ 0 & 1 & −4 \\ 0 & −3 & 12 \\ 0 & 4 & −9 \end{matrix}]

{ \atop r_{3} + 3r_{2}, r_{4} + (−4)r_{2}} [\begin{matrix}1 & 0 & 3 \\ 0 & 1 & −4 \\ 0 & 0 & 0 \\ 0 & 0 & 7 \end{matrix}] { \atop interchange 3rd and 4th row, \frac{1}{7}r_{4}} [\begin{matrix}1 & 0 & 3 \\ 0 & 1 & −4 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{matrix}]

This matrix has three pivot vectors. So the columns of the matrix form a linearly independent set.

The column vectors of

[\begin{matrix}−4 & −3 & 1 & 5 & 1 \\ 2 & −1 & 4 & −1 & 2 \\ 1 & 2 & 3 & 6 & −3 \\ 5 & 4 & 6 & −3 & 2 \end{matrix}]

form a dependent set since we have five column vectors in R^{4}.

12. Let A be a 12 × 5 matrix. You may assume that Nul(A^{T} A) = Nul(A). (This relation holds form any matrix A.) a. What is the size of A^{T} A? b. Use the Rank Theorem to obtain an equation involving rankA. Find another equation involving rank(A^{T} A). What is the connection between these two ranks? c. Suppose the columns of A are linearly independent. Explain why A^{T} A is invertible.

Solution: a. Note that Nul(A) is the dimension of the null space of A. Since A^{T} is a 5 × 12 matrix and A is a 12 × 5, we know that A^{T} A is a 5 × 5 matrix. b. rank(A)+Nul(A) = 5 and rank(A^{T} A)+Nul(A^{T} A) = 5. Using the fact that Nul(A^{T} A) = Nul(A), we know that rank(A) = rank(A^{T} A). c. The columns of A are linearly independent implies that rank(A) = 5. So rank(A^{T} A) = 5. Recall that A^{T} A is a 5 × 5 matrix. This implies that A^{T} A is a invertible matrix.