\subsection{Linear Algebra} \begin{itemize} \item \textbf{Linear} The study of items/planes and objects which are flat \item \textbf{Algebra} Objects are not as simple as numbers \end{itemize} \subsection{Linear Systems \& Their Solutions} Points on a straight line are all the points $(x, y)$ on the $xy$ plane satisfying the linear eqn: $ax + by = c$, where $a, b > 0$ \subsubsection{Linear Equation} Linear eqn in $n$ variables (unknowns) is an eqn in the form $$ a_1x_1 + a_2x_2 + ... + a_nx_n = b$$ where $a_1, a_2, ..., a_n, b$ are constants. \begin{note} In a linear system, we don't assume that $a_1, a_2, ..., a_n$ are not all 0 \begin{itemize} \item If $a_1 = ... = a_n = 0$ but $b \neq 0$, it is \textbf{inconsistent} E.g. $0x_1 + 0x_2 = 1$ \item If $a_1 = ... = a_n = b = 0$, it is a \textbf{zero equation} E.g. $0x_1 + 0x_2 = 0$ \item Linear equation which is not a zero equation is a \textbf{nonzero equation} E.g. $2x_1 - 3x_2 = 4$ \item The following are not linear equations \begin{itemize} \item $xy = 2$ \item $\sin\theta + \cos\phi = 0.2$ \item $x_1^2 + x_2^2 + ... + x_n^2 = 1$ \item $x = e^y$ \end{itemize} \end{itemize} \end{note} In the $xyz$ space, linear equation $ax + by + cz = d$ where $a, b, c > 0$ represents a plane \subsubsection{Solutions to a Linear Equation} Let $a_1x_1 + a_2x_2 + ... + a_nx_n = b$ be a linear eqn in n variables \\ For real numbers $s_1+ s_2+ ... + s_n$, if $a_1s_1 + a_2s_2 + ... + a_ns_n = b$, then $x_1 = s_1, x_2 = s_2, x_n = s_n$ is a solution to the linear equation \\ The set of all solutions is the \textbf{solution set}\\ Expression that gives the entire solution set is the \textbf{general solution} \textbf{Zero Equation} is satified by any values of $x_1, x_2,... x_n$ General solution is given by $(x_1, x_2, ..., x_n) = (t_1, t_2, ..., t_n)$ \subsubsection{Examples: Linear equation $4x-2y = 1$} \begin{itemize} \item x can take any arbitary value, say t \item $x = t \Rightarrow y = 2t - \frac{1}{2}$ \item General Solution: $ \begin{cases} x = t & \text{t is a parameter}\\ y = 2t - \frac{1}{2} \end{cases} $ \item y can take any arbitary value, say s \item $y = s \Rightarrow x = \frac{1}{2}s + \frac{1}{4}$ \item General Solution: $ \begin{cases} y = s & \text{s is a parameter}\\ x = \frac{1}{2}s + \frac{1}{4} \end{cases} $ \end{itemize} \subsubsection{Example: Linear equation $x_1 - 4x_2 + 7x_3 = 5$} \begin{itemize} \item $x_2$ and $x_3$ can be chosen arbitarily, $s$ and $t$ \item $x_1 = 5 + 4s -7t$ \item General Solution: $ \begin{cases} x_1 = 5 + 4s -7t \\ x_2 = s & s, t \text{ are arbitrary parameters}\\ x_3 = t \\ \end{cases} $ \end{itemize} \subsection{Linear System} Linear System of m linear equations in n variables is \begin{equation} \begin{cases} a_{11}x_1 + a_{12}x_2 + ... + a_{1n}x_n = b_1 \\ a_{21}x_1 + a_{22}x_2 + ... + a_{2n}x_n = b_2 \\ \vdots \\ a_{m1}x_1 + a_{m2}x_2 + ... + a_{mn}x_n = b_m \\ \end{cases} \end{equation} where $a_{ij}, b$ are real constants and $a_{ij}$ is the coeff of $x_j$ in the $i$th equation \begin{note} Linear Systems \begin{itemize} \item If $a_{ij}$ and $b_i$ are zero, linear system is called a \textbf{zero system} \item If $a_{ij}$ and $b_i$ is nonzero, linear system is called a \textbf{nonzero system} \item If $x_1 = s_1, x_2 = s_2, ..., x_n = s_n$ is a solution to \textbf{every equation} in the system, then its a solution to the system \item If every equation has a solution, there might not be a solution to the system \item \textbf{Consistent} if it has at least 1 solution \item \textbf{Inconsistent} if it has no solutions \end{itemize} \end{note} \subsubsection{Example} \begin{equation} \begin{cases} a_1x + b1_y = c_1 \\ a_2x + b2_y = c_2 \\ \end{cases} \end{equation} where $a_1, b_1, a_2, b_2$ not all zero In $xy$ plane, each equation represents a straight line, $L_1, L_2$ \begin{itemize} \item If $L_1, L_2$ are parallel, there is no solution \item If $L_1, L_2$ are not parallel, there is 1 solution \item If $L_1, L_2$ coinside(same line), there are infinitely many solution \end{itemize} \begin{equation} \begin{cases} a_1x + b1_y + c_1z = d_1 \\ a_2x + b2_y + c_2z = d_2 \\ \end{cases} \end{equation} where $a_1, b_1, c_1, a_2, b_2, c_2$ not all zero In $xyz$ space, each equation represents a plane, $P_1, P_2$ \begin{itemize} \item If $P_1, P_2$ are parallel, there is no solution \item If $P_1, P_2$ are not parallel, there is $\infty$ solutions (on the straight line intersection) \item If $P_1, P_2$ coinside(same plane), there are infinitely many solutions \item Same Plane $\Leftrightarrow a_1 : a_2 = b_1 : b_2 = c_1 : c_2 = d_1: d_2$ \item Parallel Plane $\Leftrightarrow a_1 : a_2 = b_1 : b_2 = c_1 : c_2$ \item Intersect Plane $\Leftrightarrow a_1 : a_2, b_1 : b_2, c_1 : c_2$ are not the same \end{itemize} \subsection{Augmented Matrix} $ \begin{amatrix}{3} a_{11} & a_{12} & a_{1n} & b_1 \\ a_{21} & a_{12} & a_{2n} & b_2 \\ a_{m1} & a_{m2} & a_{mn} & b_m \\ \end{amatrix} $ \subsection{Elementary Row Operations} To solve a linear system we perform operations: \begin{itemize} \item Multiply equation by nonzero constant \item Interchange 2 equations \item add a constant multiple of an equation to another \end{itemize} Likewise, for a augmented matrix, the operations are on the \textbf{rows} of the augmented matrix \begin{itemize} \item Multiply row by nonzero constant \item Interchange 2 rows \item add a constant multiple of a row to another row \end{itemize} \subsection{Recap} Given the linear equation $a_1x_1 + a_2x_2 + ... + a_nx_n = b$ \begin{enumerate} \item $a_1 = a_2 = ... = a_n = b = 0$ zero equation Solution: $x_1 = t_1, x_2 = t_2, ... = x_n = t_n$ \item $a_1 = a_2 = ... = a_n = 0 \neq b$ inconsistent No Solution \item Not all $a_1 ... a_n$ are zero. Set $n-1$ of $x_i$ as params, solve for last variable \end{enumerate} \subsection{Elementary Row Operations Example} \begin{center} \begin{minipage}{.3\linewidth} \systeme{x+y+3z = 0, 2x-2y+2z=4, 3x+9y=3} \end{minipage}% \begin{minipage}{.3\linewidth} \begin{equation*} \begin{amatrix}{3} 1 & 1 & 3 & 0 \\ 2 & 2 & 2 & 4 \\ 3 & 9 & 0 & 3 \end{amatrix} \end{equation*} \end{minipage} \end{center} \subsection{Row Equivalent Matrices} 2 Augmented Matrices are row equivalent if one can be obtained from the other by a series of elementary row operations Given a augmented matrix $A$, how to find a row equivalent augmented matrix B of which is of a \textbf{simple} form? \subsection{Row Echelon Form} \begin{defn}[Row Echelon Form (Simple)] Augmented Matrix is in row-echelon form if \begin{itemize} \item Zero rows are grouped together at the bottom \item For any 2 successive nonzero rows, The first nonzero number in the lower row appears to the right of the first nonzero number on the higher row $\begin{amatrix}{4} 0 & 0 & 1 & 2 & 3 \\ 0 & 0 & 0 & 1 & 2 \\ \end{amatrix}$ \item Leading entry if a nonzero row is a \textbf{pivot point} \item Column of augmented matrix is called \begin{itemize} \item \textbf{Pivot Column} if it contains a pivot point \item \textbf{Non Pivot Column} if it contains no pivot point \end{itemize} \item Pivot Column contains exactly 1 pivot point \# of pivots = \# of leading entries = \# of nonzero rows \end{itemize} \end{defn} Examples of row echlon form: \begin{equation*} \begin{amatrix}{2} 3 & 2 & 1 \\ \end{amatrix} \begin{amatrix}{2} 1 & -1 & 0 \\ 0 & 1 & 0 \\ \end{amatrix} \begin{amatrix}{2} 2 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{amatrix} \begin{amatrix}{3} 1 & 2 & 3 & 4 \\ 0 & 1 & 1 & 2 \\ 0 & 0 & 2 & 3 \\ \end{amatrix} \begin{amatrix}{4} 0 & 1 & 2 & 8 & 1 \\ 0 & 0 & 0 & 0 & 3 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ \end{amatrix} \end{equation*} Examples of NON row echlon form: \begin{equation*} \begin{amatrix}{2} 0 & \textbf{1} & 0 \\ \textbf{1} & 0 & 0 \\ \end{amatrix} \begin{amatrix}{2} 0 & 0 & \textbf{1} \\ \textbf{1} & -1 & 0 \\ 0 & 0 & 1 \\ \end{amatrix} \begin{amatrix}{3} \textbf{1} & 0 & 2 & 1 \\ 0 & \textbf{1} & 0 & 2 \\ 0 & \textbf{1} & 1 & 3 \\ \end{amatrix} \begin{amatrix}{4} \textbf{0} & \textbf{0} & \textbf{0} & \textbf{0} & \textbf{0} \\ 1 & 0 & 2 & 0 & 1 \\ 0 & 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & 0 & 0 \\ \end{amatrix} \end{equation*} \subsection{Reduced Row-Echelon Form} \begin{defn}[Reduced Row-Echelon Form] \label{def:rref} Suppose an augmented matrix is in row-echelon form. It is in \textbf{reduced row-echelon form} if \begin{itemize} \item Leading entry of every nonzero row is 1 \subitem Every pivot point is one \item In each pivot column, except the pivot point, all other entries are 0. \end{itemize} \end{defn} Examples of reduced row-echelon form: \begin{equation*} \begin{amatrix}{2} 1 & 2 & 3 \\ \end{amatrix} \begin{amatrix}{2} 0 & 0 & 0 \\ 0 & 0 & 0 \\ \end{amatrix} \begin{amatrix}{2} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{amatrix} \begin{amatrix}{3} 1 & 0 & 0 & 1 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 3 \\ \end{amatrix} \begin{amatrix}{4} 0 & 1 & 2 & 0 & 1 \\ 0 & 0 & 0 & 1 & 3 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ \end{amatrix} \end{equation*} Examples of row-echelon form but not reduced: (pivot point is not 1 / all other elements \textbf{in pivot column} must be zero) \begin{equation*} \begin{amatrix}{2} \textbf{3} & 2 & 1 \\ \end{amatrix} \begin{amatrix}{2} 1 & \textbf{-1} & 0 \\ 0 & 1 & 0 \\ \end{amatrix} \begin{amatrix}{2} \textbf{2} & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{amatrix} \begin{amatrix}{3} \textbf{-1} & 2 & 3 & 4 \\ 0 & 1 & 1 & 2 \\ 0 & 0 & 2 & 3 \\ \end{amatrix} \begin{amatrix}{4} 0 & 1 & 2 & \textbf{8} & 1 \\ 0 & 0 & 0 & \textbf{4} & 3 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ \end{amatrix} \end{equation*} To note: 2nd matrix has -1 in the pivot column, but 5th matrix has 2 in a non-pivot column so its fine \subsection{Solving Linear System} If Augmented Matrix is in reduced row-echelon form, then solving it is easy \begin{equation*} \begin{amatrix}{3} 1 & 0 & 0 & 1 \\ 0 & 1 & 0 & 2 \\ 0 & 0 & 1 & 3 \\ \end{amatrix} \text{then } x_1 = 1, x_2 = 2, x_3 = 3 \end{equation*} \begin{note} \begin{itemize} \item If any equations in the system is inconsistent, the whole system is inconsistent \end{itemize} \end{note} \subsubsection{Examples} Augmented Matrix: $\begin{amatrix}{4} 1 & -1 & 0 & 3 & -2 \\ 0 & 0 & 1 & 2 & 5 \\ 0 & 0 & 0 & 0 & 0 \\ \end{amatrix}$ \begin{itemize} \item The zero row can be ignored. \systeme{x_1 - x_2 + 3x_4 = -2, x_3 + 2x_4 = 5} \item Degree of freedom(\# cols): 4, number of restrictions (\# pivot cols): 2, arbitrary vars(\# non pivot cols): 4-2 = 2. Set this to the non-pivot cols \end{itemize} \begin{enumerate} \item Let $x_4 = t$ and sub into 2nd eqn \subitem $x_3 + 2t = 5 \Rightarrow x_3 = 5-2t$ \item sub $x_4 = t$ into 1st eqn \subitem $x_1 - x_2 + 3t = -2$ \subitem Let $x_2 = s$. Then $x_1 = -2 + s - 3t$ \item Infinitely many sols with ($s$ and $t$ as arbitrary params) \subitem $x_1 = -2 + s - 3t, x_2 = s, x_3 = 5-2t, x_4 = t$ \end{enumerate} Augmented Matrix: $\begin{amatrix}{5} 0 & 2 & 2 & 1 & -2 & 2 \\ 0 & 0 & 1 & 1 & 1 & 3 \\ 0 & 0 & 0 & 0 & 2 & 4 \\ \end{amatrix}$ \begin{itemize} \item \systeme{0x_1 + 2x_2 + 2x_3 + 1x_4 -2x_5 = 2,x_3 + x_4 +x_5 = 3,2x_5 = 4} \item Degree of freedom: 5, number of restrictions: 3, arbitrary vars: 5-3 = 2 \end{itemize} \begin{enumerate} \item by 3rd eqn, $2x_5 = 4 \Rightarrow x_5 = 2$ \item sub $x_5 = 2$ into 2nd eqn \subitem $x_3 + x_4 + 2 = 3 \Rightarrow x_3 + x_4 = 1$ \subitem let $x_4 = t$. Then $x_3 = 1-t$ \item sub $x_5 = 2, x_3 = 1-t, x_4 = t$ into 1st eqn \subitem $2x_2 + 2(1-t) + t - 2(2) = 2 \Rightarrow 2x_2 -t = 4 \Rightarrow x_2 = \frac{t}{2} + 2$ \item system has inf many solns: $x_1 = s, x_2 = \frac{t}{2} + 2, x_3 = 1-t, x_4 = t, x_5 = 2$ where $s$ and $t$ are arbitrary \end{enumerate} \subsubsection{Algorithm} Given the augmented matrix is in row-echelon form. \begin{enumerate} \item Set variables corresponding to non-pivot columns to be arbitrary parameters \item Solve variables corresponding to pivot columns by back substitution (from last eqn to first) \end{enumerate} \subsection{Gaussian Eliminiation} \begin{defn}[Gaussian Elimination]\ \\ \begin{enumerate} \item Find the left most column which is not entirely zero \item If top entry of such column is 0, replace with nonzero number by swapping rows \item For each row below top row, add multiple of top row so that leading entry becomes 0 \item Cover top row and repeat to remaining matrix \end{enumerate} \end{defn} \begin{note}[Algorithm with Example]\ \\ $\begin{amatrix}{6} 0 & 0 & 0 & 2 & 4 & 2 & 8 \\ 0 & 1 & 2 & 4 & 5 & 3 &-9 \\ 0 &-2 &-4 &-5 &-4 & 3 & 6 \\ \end{amatrix}$ \begin{enumerate} \item Find the left most column which is not all zero (2nd column) \item Check top entry of the selection. If its zero, replace it by a nonzero number by interchanging the top row with another row below \subitem $\begin{amatrix}{6} 0 & 1 & 2 & 4 & 5 & 3 &-9 \\ 0 & 0 & 0 & 2 & 4 & 2 & 8 \\ 0 &-2 &-4 &-5 &-4 & 3 & 6 \\ \end{amatrix}$ \item For each row below the top row, adda suitable multiple of top row so that leading entry becomes 0. \subitem $2R_1 + R_3$ will ensure that the -2 turns to 0 \subitem $\begin{amatrix}{6} 0 & 1 & 2 & 4 & 5 & 3 &-9 \\ 0 & 0 & 0 & 2 & 4 & 2 & 8 \\ 0 & 0 & 0 & 3 & 6 & 9 &-12\\ \end{amatrix}$ \item Cover top row and repeat procedure to the remaining matrix \subitem $\begin{amatrix}{6} 0 & 1 & 2 & 4 & 5 & 3 &-9 \\ \hline 0 & 0 & 0 & 2 & 4 & 2 & 8 \\ 0 & 0 & 0 & 3 & 6 & 9 &-12\\ \end{amatrix}$ \subitem Look at $C_4$. $R_3 \times -1.5R_2$ will set $R_3C_4$ to zero. \subitem $\begin{amatrix}{6} 0 & 1 & 2 & 4 & 5 & 3 &-9 \\ \hline 0 & 0 & 0 & 2 & 4 & 2 & 8 \\ 0 & 0 & 0 & 0 & 0 & 6 & -24\\ \end{amatrix}$ \subitem This is now in row echelon form. \end{enumerate} Only use $R_i \Leftrightarrow R_j or R_i + CR_j$ in this method. \end{note} \subsection{Gauss-Jordan Elimination} \begin{defn}[Gauss Joran Elimination]\ \\ \begin{enumerate} \item[1-4.] Use Gaussian Eliminiation to get row-echelon form \setcounter{enumi}{4} \item For each nonzero row, multiply a suitable constant so pivot point becomes 1 \item Begin with last nonzero row and work backwords \subitem Add suitable multiple of each row to the rows above to introduce 0 above pivot point \end{enumerate} \begin{itemize} \item Every matrix has a unique reduced row-echelon form. \item Every nonzero matrix has infinitely many row-echelon ofrm \end{itemize} \end{defn} \begin{note}[Gauss Jordan Elimination Example] Suppose an augmented matrix is in row-echelon form. $\begin{amatrix}{5} 1 & 2 & 4 & 5 & 3 & -9 \\ 0 & 0 & 2 & 4 & 2 & 8 \\ 0 & 0 & 0 & 0 & 6 & -24 \\ \end{amatrix}$ \begin{enumerate} \item All pivot points must be 1 \subitem multiply $R_2$ by $\frac{1}{2}$ and $R_3$ by $\frac{1}{6}$ \subitem $\begin{amatrix}{5} 1 & 2 & 4 & 5 & 3 & -9 \\ 0 & 0 & 1 & 2 & 1 & 4 \\ 0 & 0 & 0 & 0 & 1 & -4 \\ \end{amatrix}$ \item In each pivot col, all entries other than pivot point must be 0. Work backwards \subitem $R_1 + -3R_1$, $R_2 + -R_1$ \subitem $\begin{amatrix}{5} 1 & 2 & 4 & 5 & 0 & 3 \\ 0 & 0 & 1 & 2 & 0 & 8 \\ 0 & 0 & 0 & 0 & 1 & -4 \\ \end{amatrix}$ \subitem $R_1 + -4R_2$ \subitem $\begin{amatrix}{5} 1 & 2 & 0 & -3 & 0 & -29 \\ 0 & 0 & 1 & 2 & 0 & 8 \\ 0 & 0 & 0 & 0 & 1 & -4 \\ \end{amatrix}$ \end{enumerate} \end{note} \subsection{Review} \begin{align*} I: & cR_i, c \neq 0 \\ II: & R_i \Leftrightarrow R_j \\\ III: & R_i \Rightarrow R_i + cR_j \end{align*} Solving REF: \begin{enumerate} \item Set var -> non-pivot cols as params \item Solve var -> pivot cols backwards \# of nonzero rows = \# pivot pts = \# of pivot cols \end{enumerate} Gaussian Elimination \begin{enumerate} \item Given a matrix $A$, find left most non-zero \textbf{column}. If the leading number is NOT zero, use $II$ to swap rows. \item Ensure the rest of the column is 0 (by subtracting the current row from tht other rows) \item Cover the top row and continue for next rows \end{enumerate} \subsection{Consistency} \begin{defn}[Consistency]\ \\ Suppose that $A$ is the Augmented Matrix of a linear system, and $R$ is a row-echelon form of $A$. \begin{itemize} \item When the system has no solution(inconsistent)? \subitem There is a row in $R$ with the form $(0 0 ... 0 | \otimes)$ where $\otimes \neq 0$ \subitem Or, the last column is a pivot column \item When the system has exactly one solution? \subitem Last column is non-pivot \subitem All other columns are pivot columns \item When the system has infinitely many solutions? \subitem Last column is non-pivot \subitem Some other columns are non-pivot columns. \end{itemize} \end{defn} \begin{note} Notations\ \\ For elementary row operations \begin{itemize} \item Multiply $i$th row by (nonzero) const $k$: $kR_i$ \item Interchange $i$th and $j$th rows: $R_i \leftrightarrow R_j$ \item Add $K$ times $i$th row to $j$th row: $R_j + kR_i$ \end{itemize} \textbf{Note} \begin{itemize} \item $R_1 + R_2$ means "add 2nd row to the 1st row". \item $R_2 + R_1$ means "add 1nd row to the 2st row". \end{itemize} \textbf{Example} $$ \begin{pmatrix} a \\ b \end{pmatrix} \xrightarrow{R_1 + R_2} \begin{pmatrix} a + b \\ b \end{pmatrix} \xrightarrow{R_2 + (-1)R_1} \begin{pmatrix} a + b \\ -a \end{pmatrix} \xrightarrow{R_1 + R_2} \begin{pmatrix} b \\ -a \end{pmatrix} \xrightarrow{(-1)R_2}\begin{pmatrix} b \\ a \end{pmatrix} $$ \end{note} \subsection{Homogeneous Linear System} \begin{defn}[Homogeneous Linear Equation \& System]\ where \begin{itemize} \item Homogeneous Linear Equation: $a_1x_1 + a_2x_2 + ... + a_nx_n = 0 \iff x_1 = 0, x_2 = 0,... , x_n = 0$ \item Homogeneous Linear Equation: $\begin{cases} a_{11}x_1 + a_{12}x_2 + ... + a_{1n}x_n = 0 \\ a_{21}x_1 + a_{22}x_2 + ... + a_{2n}x_n = 0 \\ \vdots \\ a_{m1}x_1 + a_{m2}x_2 + ... + a_{mn}x_n = 0 \\ \end{cases}$ \item This is the trivial solution of a homogeneous linear system. \end{itemize} You can use this to solve problems like Find the equation $ax^2 + by^2 + cz^2 = d$, in the $xyz$ plane which contains the points $(1, 1, -1), (1, 3, 3), (-2, 0, 2)$. \begin{itemize} \item Solve by first converting to Augmented Matrix, where the last column is all 0. During working steps, this column can be omitted. \item With the \hyperref[def:rref]{RREF}, you can set $d$ as $t$ and get values for $a, b, c$ in terms of $t$. \item sub in $t$ into the original equation and factorize $t$ out from both sides, for values where $t \neq 0$ \end{itemize} \end{defn}