Eisenberg: Favorite linear algebra projects
Worldwide Mathematica Conference • Chicago 1998

Project:  Fitting a polynomial by least-squares

Mathematica background

Functions for least-squares

A simple example to illustrate the ideas.  First, the data:

xy  =  {{-1, 8}, {0, 8},  {1, 4},  {2, 16}};

Use built-in function Fit to fit a quadratic polynomial to the data:

Fit[xy,  {1,  x,  (x^2)},  x]

Define a function to automate forming the list of functions in the basis:

polyfit[data_,  deg_,  var_]  :=  
      Fit[data,  var^Range[0,  deg],  var]

The example again in terms of the defined function:

poly[x_]  =  polyfit[xy,  2,  x]

Compare the least-squares "predicted" y-values with the given, "observed" y-values:

poly[{-1,  0,  1,  2}]

Last @ Transpose @ xy

Could do it without symbolics!

One can do everything working just with the coefficients of the fitted polynomial and not the actual expression of the polynomial in terms of a variable.

\(altpolyfit[data_, \ deg_]\  := \ 
      Module[{x, \ poly}, 
	poly\  = \ Fit[data, \ x^Range[0, \ deg], \ x]; 
        
	CoefficientList[\ poly, \ x]
]\)

c  =  altpolyfit[xy,  2]

In such a situation, one must, unfortunately, explicitly split off the constant term in order to avoid Mathematica's indeterminate (0^0):

\(polyval[c_, \ x_]\  := \ 	
      c[[1]]\  + \ 
        Sum[c[[j + 1]]\ x\^j, 
          \ {j, \ 1, \ Length[c]\  - \ 1}]\)

polyval[c,  2]

Getting data for the exercises

The data with which each student works is obtained by loading a package that generates random data of a certain form, with a random seed based upon the student's ID number.  In practice, the package is first encoded by using the Mathematica function Encode.

The student obtains the data as follows:

Enter you student ID number below and evaluate the cell.

myID = "";    
(* type your student ID number inside the quotes *)

Type your first name below and evaluate the cell.

myName = "";    
(* type your name inside the quotes *)

Load the package that creates your data: evaluate the next cell.

Get[Directory[] <> $PathnameSeparator <> "FitData.m"]

Change the first three letters in the symbol myName in the following cell to match the first three letters of myName that you typed earlier!

myData = MURdata

Exercises

Exercise 1.  For degrees n  =  1,  2,   ... ,  15, find the polynomial (Subscript[p, n]) of degree at most n that best fits your data in the least-squares sense.

Exercise 2.  Calculate, for each n  =  1,  2,   ... ,  15, the residual error (Subscript[r, n]) in the polynomial (Subscript[p, n]) for your data.

Do the results confirm that the fit becomes more accurate as the degree n increases?

Exercise 3.  The truth!  Your data is due to experimental error in measuring the y-values at your given x-values for a certain polynomial of degree  ≤  15.

Which of the polynomials (Subscript[p, n]) is most likely to be that true polynomial?  (Explain via residuals and graphs.)

Exercise 4.  Let (Subscript[p, n]) be that polynomial you selected. 

Perturb each coefficient of (Subscript[p, n]) by a random amount (between, say, -0.1and 0.1) to get a new polynomial q.  

Compare the residual errors r and (Subscript[r, n]) in q and (Subscript[p, n]), respectively—which is smaller?  Which ought to be smaller, and why?

To main notebook