Only the first nine parts of the Basic Tutorial are included in this draft.

Basic Tutorial (9 of 10)

9 Working with Files

In this ninth part of the basic tutorial you will learn about:

• Reading data from a file
• Writing results to a file
• Working with simple file formats

File formats

This tutorial describes functions for reading information from external files in to the Mathematica kernel and for writing results from the Mathematica kernel to external files. You can use these functions to read data from any file and to write data in any format. A thorough discussion of this subject can be found in Sections 1.11 and 2.11 of The Mathematica Book.

There are many other common operations with external files that are mentioned here only briefly. Most of these operations involve the notebook front end or special file formats for graphics or typesetting.

The two basic file formats that are specific to Mathematica are notebook format and package format. Notebook format is the format used by the Mathematica front end for storing notebook documents. Package format is the format used for reading information in to the Mathematica kernel.

Files in many common graphics formats can be written using Display and Export. These files can also be read in to Mathematica using Import. For more information, see the documentation for these functions.

Import and Export can also be used to read and write common data formats, such as HDF (hierarchical data format) and various tabular formats.

Mathematica notebooks can be saved in format using the TeXForm function, or by selecting TeX from the Save as Special submenu of the File menu. Notebooks can be saved in HTML format using the HTMLSave function, of by selecting HTML from the Save as Special submenu of the File menu.

Saving and reading expressions

The basic functions for saving expressions to a file are Put, PutAppend, Write, WriteString, and Export. The basic functions for reading expressions from a file are Get, Read, ReadList, and Import.

In generic examples involving a single expression you can use Put to write the expression to a file and Get to read that expression back in to the Mathematica kernel.

These same operations can also be entered using that notation Expand[(1+π)^5]>>data in place of Put[Expand[(1+π)^5],"data"] and <<data in place of Get["data"].

Functions such as Write, WriteString, and PutAppend are useful in more complicated examples, such as examples involving more than one expression, or examples in which expressions are written to a file in pieces.

Here is an input that uses Write to write five expressions to a file.

     data

In this example, OpenWrite is used to open a stream for output, and Close is used to close that stream. Streams are discussed in Section 2.11.3 of The Mathematica Book. A stream is just a channel to a file. In the initial examples, a stream was opened and closed automatically by Put.

The five expressions in this file can be read in to a list using ReadList. In the following example, OpenRead is used to open a stream for input, and the result from ReadList is assigned as the value of result.

If you know how to open files using a text editor or word processor on your computer, an instructive exercise is to use that editor or word processor to look at the file that was created by Mathematica. The file in this example will be a plain text file containing five Mathematica inputs. You can find this file by looking for a file named data in the current working directory. You can determine the current working directory by evaluating Directory[].

Although Get could also have been used to read this file, Get would have returned only the last expression. Get reads expressions from a file and evaluates each expression, returning the result of the last evaluation. Get is intended primarily for reading definitions from a file, as discussed in the next section.

Saving and reading definitions

The Save function is used for saving definitions (assignments) to a file. Since assignments are expressions, they can in principle also be saved using Put or Write, or any other function that can be used for writing expressions to a file. The Save function, however, provides a convenient tool for saving expressions in the form of assignments.

Here is an example showing the use of Save to save the definitions for a function f to a file f.m. Although any file name can be used, names of files that contain Mathematica definitions conventionally end in .m.

The contents of this file can be displayed by evaluating !!f.m or by opening the file with an ordinary text editor or word processor.

f[0] = 1
 
f[(n_Integer)?Positive] := n*f[n - 1]

This file can be read in to Mathematica to introduce these definitions for f. To demonstrate that reading this file has that effect, first clear the definitions for f, then read the file using Get and observe that the definitions have been restored.

The file f.m is an example of a Mathematica package. A package is file that can be read in to the Mathematica kernel to introduce functions or to add definitions for functions.

Packages can also be created by entering definitions directly in to a file, without using Save. You can enter definitions into a file using a text editor of your choice, or using the notebook front end as an editor. One way to create a package using the notebook front end is to type your definitions into a notebook and use Package Format in the Save As Special submenu of the Edit menu to save the notebook as a package. For more information, see the documentation for Package Format. See also exercise 9.10.

Writing formatted files

Many common file formats used by other programs are variants of a basic table of numbers. There are several ways in Mathematica to write a table of numbers to a file.

Tables are normally written either by writing the table to a file one row at a time, or by using TableForm or MatrixForm to format the entire table in Mathematica before writing the table to a file.

Here is a table formatted using TableForm.

The default format type for this output is StandardForm, which includes Mathematica typeset formatting instructions that are not normally of interest when writing files that are to be read by other programs. Files to be read by other programs are usually written using OutputForm format type. Here is an example showing TableForm formatting in OutputForm. The option TableSpacing option is used here to set the spacing between rows and the spacing between columns in the table.

     1.5   2.3   1.7
     2.1   5.7   11.2
     3.2   6.9   4.1

Once the desired table has been obtained in OutputForm on your computer screen, the same formatting instructions can be used to write the table to a file.

The following inputs write this table to a file, close the stream to the file, and display the result in Mathematica. You can also check this file by viewing it with any editor.

1.5   2.3   1.7
2.1   5.7   11.2
3.2   6.9   4.1

Although OutputForm is useful for setting up the overall format of the table, it is often desirable to format the elements of the table in some other format.

Here is another table of numbers and a TableForm display showing basic OutputForm output. The exponents in this output are displayed as superscripts. This format is not suitable for reading by most programs.

     1.    22026.5        0.367879
     
                     8
     2.    4.85165 10     0.0183156
     
                     13
     3.    1.06865 10     0.00012341
     
                     17             -7
     4.    2.35385 10     1.12535 10
     
                     21             -11
     5.    5.18471 10     1.38879 10
     
                     26             -16
     6.    1.14201 10     2.31952 10
     
                     30             -22
     7.    2.51544 10     5.24289 10
     
                     34             -28
     8.    5.54062 10     1.60381 10
     
                    39              -36
     9.    1.2204 10      6.63968 10
     
                     43             -44
     10.   2.68812 10     3.72008 10

Here is another TableForm display in which FortranForm is mapped on to each element in the data. This display is typical of common formats used for storing tables of numbers.

     1.    22026.465794806703      0.36787944117144233
     2.    4.851651954097898e8     0.018315638888734182
     3.    1.0686474581524445e13   0.00012340980408667962
     4.    2.353852668370195e17    1.1253517471925921e-7
     5.    5.184705528587058e21    1.388794386496404e-11
     6.    1.1420073898156808e26   2.3195228302435736e-16
     7.    2.5154386709191576e30   5.242885663363477e-22
     8.    5.540622384393486e34    1.6038108905486433e-28
     9.    1.2204032943178348e39   6.639677199580763e-36
     10.   2.6881171418161212e43   3.7200759760208555e-44

As before, a table with this format can be written to a file using exactly the same formatting instructions as are used to format the table on the computer screen. For example:

1.    22026.465794806703      0.36787944117144233
2.    4.851651954097898e8     0.018315638888734182
3.    1.0686474581524445e13   0.00012340980408667962
4.    2.353852668370195e17    1.1253517471925921e-7
5.    5.184705528587058e21    1.388794386496404e-11
6.    1.1420073898156808e26   2.3195228302435736e-16
7.    2.5154386709191576e30   5.242885663363477e-22
8.    5.540622384393486e34    1.6038108905486433e-28
9.    1.2204032943178348e39   6.639677199580763e-36
10.   2.6881171418161212e43   3.7200759760208555e-44

Here is a more elaborate example in which PaddedForm with the NumberFormat and ExponentFunction options are used to format the elements of the table.

       1.0000e0     2.2026e4      3.6788e-1
       2.0000e0     4.8517e8      1.8316e-2
       3.0000e0     1.0686e13     1.2341e-4
       4.0000e0     2.3539e17     1.1254e-7
       5.0000e0     5.1847e21     1.3888e-11
       6.0000e0     1.1420e26     2.3195e-16
       7.0000e0     2.5154e30     5.2429e-22
       8.0000e0     5.5406e34     1.6038e-28
       9.0000e0     1.2204e39     6.6397e-36
       1.0000e1     2.6881e43     3.7201e-44

Reading formatted files

Tables of numbers with spaces or numbers separating the elements in each row of the table can be read in to Mathematica using ReadList. By default, ReadList will all of the numbers in a table into a single list.

To demonstrate the use of ReadList, first generate a file containing a table of numbers.

This shows the file to be read.

1. 22026.465794806703    0.36787944117144233
2. 4.851651954097898e8   0.018315638888734182
3. 1.0686474581524445e13 0.00012340980408667962
4. 2.353852668370195e17  1.1253517471925921e-7
5. 5.184705528587058e21  1.388794386496404e-11

By default, ReadList reads all of the numbers in the file in to a single list. The first argument in ReadList gives the file or stream from which to read, and the second argument specifies the type of expression to read. In this example the type to read is Number.

The RecordLists option can be used to specify that the numbers on each row of the file should be read in to separate lists. This gives the original data.

Exercises

Exercise 9.1 Saving and loading expressions

Recall the example showing the use of Put to write an expression to a file and Get to read that expression from the file.

Show how to do this operation using Write to write the expression to a file and Read to read that expression from the file. Be sure to close all of the streams that are opened by your program.

Exercise 9.2 Writing a table of expressions to a file

These inputs show one way to use TableForm to display an array of expressions, with each element of the table displayed in InputForm.

     1.5   2.8*^6    -1
     4.1   5.3*^6    -3
     7.3   1.25*^7   -7
     8.1   2.71*^7   -6
     9.5   3.11*^7   -3

Use Write to write this same table to a file. The contents of the file should be the same as the display above. You can check your result by looking at the file with a text editor, or by displaying the file in Mathematica using !!file.

Exercise 9.3 Reading a table of expressions from a file

Use ReadList to read the file from the previous exercise into a matrix expression in the Mathematica kernel. The result should be equivalent to the original expression (the value of data).

Exercise 9.4 Writing comma-separated numbers

Here is a function WriteMatrix that is designed to write an array of numbers to a file. Each element is written using FortranForm. The elements in each row are separated by a comma. This function writes out the array one row at a time, rather than using TableForm or MatrixForm to format the entire table.

Demonstrate the use of this function by showing how to use it to write the data array from exercise 9.2 (the value of data) to a file.

Exercise 9.5 Reading comma-separated numbers

Here is a program that reads a comma-separated table of numbers from a file named file in to a matrix in the Mathematica kernel. If you used file as the name of the file in the previous exercise, then the output of this program will be as shown here.

Starting with this program, define a function ReadCSV for reading data from a file containing a matrix of comma-separated numbers. Demonstrate the use of ReadCSV by using it to read the file saved in the previous exercise.

Exercise 9.6 Writing data using Export

Here again is the data from the previous exercise.

Show how to use Export to write this data to a file in Table format. This format is specified using Table as the third argument in Export.

Exercise 9.7 Reading data using Import

Use Import to read the file generated by Export in the previous exercise.

Exercise 9.8 Saving results from a differential equation

The result from NDSolve in this example is returned using an InterpolatingFunction expression.

If this result is to be exported to another program, it is frequently more convenient to have the result in the form of a table of values. Here is one way to construct such a table of values.

Write this table to a file so that it can be read by in to Mathematica using ReadList as shown here:

Exercise 9.9 Reading and writing non-numerical information

Although numbers have been the focus of this tutorial, functions such as Read, ReadList, and Write can be used with any type of information.

To demonstrate reading and writing of non-numerical information, first evaluate the following inputs to create a short file.

The contents of this file should appear as shown here.

f[1] = 23.5
f[2] = 47.5
f[3] = 11.9

The three lines in this file can be read using ReadList and String format.

The strings read from the file can be converted into expressions using ToExpression. The expressions in this example are assignments that assign values to f.

Now consider a similar file, but without the "=" signs.

f[1] 14.9
f[2] 33.5
f[3] 9.2

Read this file in to Mathematica and construct assignments similar to the assignments in the previous example. There are at least two qualitatively different ways of solving this problem. One solution is to insert the necessary "=" signs using StringInsert, and then proceed as in the previous example. Another solution is to read the variables as strings (this can be done using Word format) and the values as numbers and construct the assignments directly. Use whatever approach you prefer.

Open a new notebook and copy the following four cells in to that notebook.

Make the cells in the new notebook into initialization cells and save the notebook in package format. You can make the new cells into initialization cells by selecting the cell brackets and choosing Initialization Cell from the Cell Properties submenu of the Cell menu. You can save the notebook in package format by selecting Package Format from the Save As Special submenu of the File menu.

If you use fib.m as the name of the file, the contents of the file should appear as shown here:

fib::usage = "fib[n] gives the nth Fibonacci number."
fib[n_Integer /; n > 1] := fib[n - 1] + fib[n - 2]
fib[0] = 0
fib[1] = 1

Now evaluate Clear[fib] and load your package to introduce the rules for fib. You can check that the rules have been loaded correctly by evaluating ?fib.

Note that the same package could have been created by typing these definitions into a file using an ordinary editor.