How Do I Do Basic Image Processing in Mathematica

This how-to demonstrates how Mathematica can be used to do image analysis tasks. In addition to many common operations, the ListInterpolation function can be used to do new and exciting manipulations and analyses.

Note: To save bandwidth, the original imported picture is not shown in every subsection. Click here to see it.

Importing, Displaying, and Exporting Graphics Files Color Conversions Accessing Image Data Simple Math Histograms Histogram Operations Selecting Pieces of Images Area Selection Thresholding Convolution and Correlation List Interpolation and Data Recovery The Doghill Mountains Data Recovery Mathematical Operations

Importing, Displaying, and Exporting Graphics Files

The Import command provides Mathematica users with an efficient and intuitive way to read files in most standard graphics formats, including bitmap, JPEG, GIF, TIFF, and numerous platform-specific formats. In many cases, Mathematica even automatically detects the correct graphics format; otherwise, users can specify it themselves. Note that one has to use double backslashes in the file path. To interactively select the file, use the GetFilePath command in the Input menu. This command opens a standard file dialog and returns a correctly formatted path to the file at the current text cursor position.

The graphic can now be displayed using Show.

It can also be exported in a different format. For a complete list of import and export formats, consult the documentation for Import and Export.

Color Conversions

ColorOutput converts Mathematica graphics between different color models. The three standard models are RGBColor, CMYKColor, and GrayLevel. Note that ColorOutput changes the image representation, not just the rendering.

Accessing Image Data

Mathematica handles many different kinds of objects: mathematical formulas, lists, and graphics, to name but a few. Although objects often look very different, Mathematica represents all in one uniform way; they are all expressions. The imported graphics files are, for example, stored as a Graphics object.

The object f in an expression f[x, y, ... ] is known as the head of the expression. You can extract it using Head. Particularly when you write programs in Mathematica, you will often want to test the head of an expression to find out what kind of thing the expression is.

Both the imported JPEG and TIFF files are, of course, Graphics objects.

The easiest way to access the pixel data is to change the head of the expression to a Mathematica list and extract the array of pixel values. The following command replaces the Graphics head with a List head. Explaining the mechanism is beyond the scope of this document. If you are interested, consult the documentation for Replace.

The list contains an array of pixel values and some supplemental information on image size and color model. Short gives an overview.

The pixel values are always the first element of the data structure. They can now be treated like any other list in Mathematica. An RGB image, for example, is represented as an array of {r, g, b} triples.

The next line extracts the array of pixel values. To learn more about matrix manipulation, consult Section 1.8.4 of The Mathematica Book or How Do I Create and Manipulate Matrices?

As you can see, the color values are now stored in a 302x291x3 array.

For example, here is a ListDensityPlot of the red channel of the image. All is a new function in Mathematica 4 that simply instructs Mathematica to take all elements in the respective dimension of the array.

Simple Math

All Mathematica functions that can be applied to lists can also work on the image data. For example, the following two lines are two different ways of inverting a grayscale image.

The same procedure works for color images, too. Using Apply is a shortcut to convert all three color values in one step. Also, RGBColor expects its values to be between 0 and 1, so pixelvalues gets divided by 256.

Histograms

Although Mathematica's histogram functions in Graphics`Graphics` work only for small data sets, histograms can be created easily by using BinCounts and ListPlot.

BinCounts is part of the Statistics`DataManipulation` package that ships with Mathematica.

Read in the package.

Count the occurrences of pixel values in the red channel of the image.

Histogram Operations

Because the image has been converted into a Mathematica list, any Mathematica function can be applied to parts of the list. This makes histogram operations trivial. For example, add a reddish tint by taking the square root of the red channel. Note that the color values for RasterArray have to be scaled between 0 and 1. The #'s in the code are pure functions. To learn more about them, see The Mathematica Book.

In the following picture, the red channel is decimated by 70 percent.

The next picture shows the effect of taking the square root of the color values in the red channel. Because the values are scaled between 0 and 1, the image has a reddish tint.

Selecting Pieces of Images

Slices

Once the pixel information is converted into a Mathematica list, it can be operated on like any other Mathematica list of lists. In the following picture, the three color channels are extracted by slicing the 266 x 276 x 3 data set along the z axis.

Click here to learn more about selecting pieces of lists.

Or, take a slice through row 200 of the red channel.

To visualize this slice, set all the pixels along this line to white. Copying the array into a new variable first preserves the original image.

The easiest way to plot multiple slices at once is to use the MultipleListPlot standard package.

Because ListDensityPlot and Raster work only with two-dimensional arrays, you must use RasterArray to put markers on the full-color image.

Area Selection

Selecting areas of a graphic is also straightforward. To find the values for Take, click the graphic and move the mouse while holding down the CONTROL key. The pointer location will be displayed in the lower-left corner of the window. To select multiple points, click all that interest you; then copy, unselect the graphic, and paste. Mathematica will return a list of all the locations you selected. For more information on how to select coordinates in Mathematica graphics, see How Do I Extract Coordinates from a Plot?

This line of code selects a 51-by-31-pixel rectangle of the picture.

The next line sets the red channel to .

Putting the results back into the original image is also pretty easy and straightforward. First copy the image into a new variable to preserve the original image.

Then replace the corresponding values with the transformed ones.

Thresholding

To select pixels based on some property, simply use If with the appropriate cutoff value. For example, the following line selects all pixels with an R value greater than 0.5x256.

For more information on selecting parts of lists, see How Do I Create and Manipulate Matrices?

Convolution and Correlation

The new Mathematica 4 functions ListConvolve and ListCorrelate make a number of advanced image-processing tasks easy to do. The following graphic shows the effect of applying a Sobel edge filter to the image by convolving the image with two 3x3 kernels.

List Interpolation and Data Recovery

An interesting feature of Mathematica is its ability to rapidly calculate list interpolations of any degree in arbitrary dimensions. This feature opens up new ways to manipulate images and recover information lost by the discretizing of data.

The Doghill Mountains

Before we do anything useful with list interpolation, let's try something fun. Perform a first-order list interpolation over the data set, and plot the color values in 3D. Using the new real-time 3D graphics, then rotate the graphic and see the image pop in and out of view.

Data Recovery

Let's go back to the small piece of the image selected in "Selecting Pieces of Images."

As you can see, the image above consists of only 51x31 data points, so the resolution is not terribly high. List interpolation provides a convenient way to recover more features.

The following command creates a list interpolation over the red channel of the image and scales the resulting function between 0 and 1 in the x and y coordinates.

Mathematical Operations

One can do all kinds of fancy image manipulations and transformations using ListInterpolation. For example, differentiation finds changes in color values. Two-dimensional differentiation also creates a nice embossing.