Linear Algebra

Index

This index gives a list of keywords that are associated with various cells in this Notebook. If you Command-Double-click one of the keywords below,  Mathematica  will find the cell associated with that keyword, select it, and display it in the center of  your window.

Array  
Array`tensor  
Decompositions  
Det  
DiagonalMatrix  
Dimensions`tensor  
Dot`Product`tensor  
Eigensystem  
Eigenvalues  
Eigenvectors  
IdentityMatrix  
Inner`tensor  
Inverse  
LinearSolve  
Matrices`pieces  
Matrix`column  
MatrixExp  
MatrixForm  
MatrixPower  
Minors  
NullSpace  
Operations`basic  
Operations`matrix  
Outer`tensor  
Part  
PseudoInverse  
QRDecomposition  
RowReduce  
SchurDecomposition  
SingularValues  
Sum  
Systems`linear  
Table  
Table`tensor  
TensorRank  
Tensors  
Transpose  
Transpose`tensor

Basic Operations

Syntax (see pages 121 and 649)

{a1, a2, a3 } is a vector with 3 components.

{{a11, a12 },{a21, a22 }} is a two-by-two matrix .

c v , c m ,  etc.  multiplies by a scalar.

v .w   gives the dot product of the vectors v  and w.

v .m  and m .v   give the product of the vector v  and the matrix m.

m .n   gives the product of the two matrices m  and n .

Adding Two Vectors

{x, y} + {2, 3}

Multiplying by a Scalar

5 {x,y}

Dot or Inner Product

{x,y} . {s, t}

Multiplying a  Matrix and a Vector

m = {{a,b},{c,d}}

v = {x,y}

m . v

v . m

Multiplying Two Matrices

m = {{a,b},{c,d}}

m . m

Triple Scalar Product

v . m . v

Vectors as Function Arguments

Log[ {a, b, c}]

D[{ x, x^2, x^3}, x]

A Note on Matrices and Vectors in Mathematica

Vectors and matrices in Mathematica  are simply represented by lists and lists of lists, respectively. Most mathematical functions in Mathematica  are set up to apply themselves separately to each element in a list. A consequence is that most mathematical functions are applied element by element to matrices and vectors.

It is important to realize that Mathematica  treats an object as a vector in a particular operation only if the object is explicitly a list at the time the operation is done. If the object is not explicitly a list, Mathematica  always treats it as a scalar. Thus, for example,  5 + {x, y} gives a result of           {5 + x, 5 + y}.

Lists are very important and general structures in Mathematica  and there are many powerful list manipulation operations. For more information, see pages 56, 118, 201 and 215 as well as the sections on lists in this Notebook.

Constructing Vectors and Matrices

Syntax (see pages 122 and 649)

Table[f ,{i ,n }] builds a length-n vector by evaluating f  with   i =1,  i =2, ..., i =n .

Table[f ,{i , m },{j ,n }] builds an m by n matrix  by evaluating f with i =1, i =2, ..., i =m , j =1, ..., j =n .

Array[a ,n ] builds a length-n  vector of the form              {a[1], a[2], ...}.

Array[a ,{m ,n }] builds an m by n matrix with entries a[i,j ].

DiagonalMatrix[ list ] generates a diagonal matrix with elements of list  on the diagonal.

IdentityMatrix[n ] generates an n  by n  identity matrix.

Constructing a Vector

Table[ Sin[ n Pi/ 5], {n, 5}]

Constructing a Symbolic Vector

Array[ a, 5]

Defining Values for the Vector

Note that the values can be defined in any order and that components without numerical values are in symbolic form.

b[2] = 3

b[5] = 7

b[3] = 1

Table[b[i], {i, 5}]

Constructing a Matrix

mat = Table[i+j, {i, 3},{j, 3}]

MatrixForm[mat]

Constructing a Symbolic Matrix

Table[a[i,j], {i, 2}, {j, 2}]

Array[a, {2,2}]

Dimensions of a Matrix

Array[a, {2,3}]

Dimensions[%]

Diagonal Matrix

DiagonalMatrix[{a, b, 5}]

Identity Matrix

IdentityMatrix[3]

Zero Matrix

Table[ 0, {3}, {2}]

Upper-Triangular Matrix

Table[If[i <= j, i^2 + j^2, 0], {i, 3}, {j, 3}]

MatrixForm[%]

Tridiagonal Matrix

Table[Switch[ i-j, -1, a, 0, b, 1, c, _, 0],
                  {i, 5}, {j, 5}]

MatrixForm[%]

Getting Pieces of Matrices

Syntax (see pages 125 and 651)

m [[i , j ]] gives the i , j  th entry of the matrix m .

m [i ] gives the i th row.

m [[{i1 , ..., ir }, {j1 , ..., jr }]] gives the r  by s  submatrix of m  having row indices ik  and column indices jk .

Transpose[m ][ j ]] or Map[#[[ j ]]&, m ] gives the j th column.

Minors[ m, k ] gives a matrix of the k  by k  minors of m .

The Matrix

t = Array[a, {3, 3}]

Single Element of the Matrix

t[[2, 3]]

Second Row of the Matrix

t[[2]]

Second Column of the Matrix

Map[#[[2]]&, t]

Transpose[t][[2]]

A Submatrix

t[[ {1,2}, {2,3}]]

t[[ Range[1,2], Range[2,3] ]]

2 by 2 Minors of the Matrix

Minors[ t, 2]

Matrix Operations

Syntax  (see page 657)

Inverse[m ] gives the inverse of the matrix m .

Transpose[m ] gives the transpose of m .

Det[m ] gives the determinant of m .

Sum[m [[i , i ]], {i , Length[m ]}] gives the trace.

MatrixPower[m , n ] gives the n th matrix power.

MatrixExp[m ] gives the matrix exponential.

Symbolic Inverse

m = {{a,b},{c,d}}

Inverse[m]

Exact Numerical Inverse

hil = Table[1/(i + j), {i, 4}, {j, 4}]

invhil = Inverse[hil]

hil . invhil

Approximate Numerical Inverse

Note that when there are approximate numbers in the matrix Inverse gives a numerical approximation of the inverse.

mat = {{1.32, 3.4}, {-2.3, 5.4}}

invmat = Inverse[ mat ]

Transpose

Transpose[ {{a, b, c},{ap, bp, cp}}]

Determinant

Det[{{a,b},{c,d}}]

Two by Two Minors of a Matrix

m = Array[ a, {3, 3}]

Minors[ m, 2]

Characteristic Polynomial

hil = Table[1/(i + j), {i, 3}, {j, 3}]

Det[ hil - x IdentityMatrix[3] ]

Trace

m = {{a,b},{c,d}}

tracem = Sum[ m[[i, i]], {i, 2}]

Twelfth Power of a Matrix

m = {{0.4, 0.6}, {0.525, 0.475}}

MatrixPower[ m, 12]

Matrix Exponential

MatrixExp[m]

Solving Linear Systems

Syntax (see page 659)

LinearSolve[m ,b ] gives a vector x  that solves the matrix equation m .x ==b .

NullSpace[m ] gives a list of basis vectors whose linear combinations satisfy the matrix equations m .x == 0.

RowReduce[m ] gives a simplified version of m  obtained by taking linear combinations of rows.

Solution Using  LinearSolve

m = {{1, 5}, {2, 1}}

LinearSolve[ m, {a, b}]

Together[%]

Solution Using Solve

Solve[ {{1, 5}, {2, 1}} . {x, y} == {a, b},
                       {x, y}]

Together[{x,y} /. %[[1]] ]

A Note on Solving Linear Systems

In many cases you will find it convenient to write down a set of linear equations explicitly, and then solve them using Solve.  In some cases, however, you may prefer to convert the system into a matrix equation. This approach is often useful when the system of equations arises as part of a general algorithm, and you do not know in advance how many variables are invloved. Particularly when you have large, sparse, matrices, LinearSolve is the most efficient method to use. LinearSolve returns the solution as a list  while Solve returns transformation rules.

Null Space

matrix = Table[ i - j, {i, 3}, {j, 3} ]

Det[ matrix]

NullSpace[ matrix]

Row Reduction

RowReduce[ {{3, 4}, {6, -2}, {2, 9}}]

Eigenvalues, Eigenvectors and Decompositions

Syntax (see pages 664 and 667)

Eigenvalues[m ] gives a list of the eigenvalues of m .

Eigenvectors[m ] gives a list of the eigenvectors of m .

Eigensystem[m ] gives a list of the form                              {eigenvalues , eigenvectors }.

SingularValues[m ] gives the singular value decomposition of a numerical matrix m . The result is a list {u , w , v }, where w  is the list of singular values, and m  can be written as                                     Transpose[u ].DiagonalMatrix[w ].v .

PseudoInverse[m ] computes the pseudoinverse of a numerical rectangular matrix.

QRDecomposition[m ] finds the QR decomposition for a numerical matrix m . The result is a list {q ,r }, where q  is an orthogonal matrix and r  is an upper-triangular matrix.

SchurDecomposition[m ] finds the Schur decomposition of a numerical matrix. The result is a list {q ,t }, where q  is an orthogonal matrix and t  is a block upper-triangular matrix.

Symbolic Eigenvalue Computation

Eigenvalues[{{a, b}, {-b, 2a}} ]

Exact Eigenvector Computation

Eigensystem[{{2, 1}, {1, 1}}]

Numerical Eigensystem Computation

Note that an assignment is a convenient way to pick out pieces of the result.  This assigment sets vals to the list of eigenvalues, and vects to the list of eigenvectors.

{vals, vects} = 
         Eigensystem[ {{2.3, 4.5}, {6.7, -1.2}}]

A Note on Eigensytem Computations

Finding the eigenvalues of a n  x n  matrix in principle involves solving an n th degree polynomial equation. Since for n  greater than or equal to 5 explicit algebraic solutions cannot in general be found, it is impossible to give explicit algebraic results for the eigenvalues and eigenvectors of all but the simplest or sparsest matrices. If you give a matrix of approximate real numbers, Mathematica  will find the approximate numerical eigenvalues and eigenvectors.

Eigenvalues Using the Characteristic Polynomial

Note that this is in general less efficient than using Eigenvalues directly.

m = {{2.3, 4.5}, {6.7, -1.2}}

charpoly = Det[ m - x IdentityMatrix[2] ]

NSolve[ charpoly == 0, x]

Singular Value Decomposition

{ u, md, v} =
         SingularValues[N[{{1, 3}, {-4, 3}}]]

Pseudoinverse

PseudoInverse[ N[{{1, 2, 3}, {2, 3, 5}}] ]

QR Decomposition

{q, r} =
      QRDecomposition [ N[{{1, 5}, {-7, 3}}]]

Schur Decomposition

{q, t} =
     SchurDecomposition [ N[{{1, 5}, {-7, 3}}]]

Tensors

Syntax (see page 667)

Table[f ,{i1 ,n1 },{i2 ,n2 }, ...,{ik ,nk }]   creates an n1  by n2  by ... by nk  tensor whose elements are the values of f .

Array[a .{n1 ,n2 , ...,nk }]   creates an n1  by n2  by ...  by nk  tensor with elements given by applying a  to each set of indices.

Dimensions[t ] gives a list of the dimensions of the tensor.

TensorRank[t ] finds the rank of a tensor.

MatrixForm[t ] prints with elements of t  arranged in a two-dimensional array.

Transpose[t ] transposes the first two indices in a tensor.

Outer[f ,t1,t2 , ...] forms the generalized outer product of the tensors ti  with "multiplication operator" f .

t1  . t2  forms the dot product of t1  and t2  (last index of t1  contracted with first index of t2  ).

Inner[f ,t1 ,t2 ,g ] forms the generalized inner product, with "multiplication operator" f   and "addition operator" g .

A 2 x 3 x 2 Tensor using Table

t = Table[ i1 + i2 i3, {i1, 2}, {i2, 3}, {i3, 2} ]

A 2 x 3 x 2 Tensor using Array

Array[ (#1 + #2 #3)&, {2, 3, 2}]

Dimension and Rank

t = 
  Table[ i1 + i2 i3, {i1, 2}, {i2, 3}, {i3, 2} ]

Dimensions[ t ]

TensorRank[ t ]

Transpose

tens = Array[ a, {2, 3, 1, 2}]

ttens = Transpose[ tens ]

Dimensions[ ttens ]

Outer Product

Outer[ Times,
          {{m11, m12}, {m21, m22}}, {a, b, c} ]

Dimensions[ % ]

Generalized Outer Product

Outer[ f, {{m11, m12}, {m21, m22}}, {a, b, c} ]

Inner Product

a = Array[ 2&, {3, 2, 2}]

b = Array[ 2&, {2, 3, 1}]

Inner[ Times, a, b, Plus ]

a . b

Generalized Inner Product

Note that a  and b are defined in the previous example.

Inner[ f , a, b, g ]