Mathematica 9 is now available

How Do I Denoise a Data Set?

Mathematica 4 offers a wide range of possibilities to denoise experimental data sets. This how-to demonstrates two common approaches, Fourier and wavelet filtering.

Fourier Filtering

This reads in the content of the data file. 

data = Get["Wavelets/Data/shocknoi.dat"];

This plots the data. 

noisy=ListPlot[data, PlotJoined -> True,
AxesLabel -> {"x", "u(x)"}];

[Graphics:Images/index_gr_1.gif]

Define an appropriate filter function. 

[Graphics:Images/index_gr_2.gif]

Generate the corresponding data set of filter values. 

[Graphics:Images/index_gr_3.gif]
[Graphics:Images/index_gr_4.gif]

[Graphics:Images/index_gr_5.gif]

Convolve the data and the filter. 

[Graphics:Images/index_gr_6.gif]

[Graphics:Images/index_gr_7.gif]


Spline Filtering Using Wavelets

Load the application package Wavelet Explorer.

Needs["Wavelets`Wavelets`"]

This reads in the content of the file. 

data = Get["Wavelets/Data/shocknoi.dat"];

This plots the data. 

ListPlot[data, PlotJoined -> True,
AxesLabel -> {"x", "u(x)"}]

[Graphics:Images/index_gr_8.gif]

[Graphics:Images/index_gr_9.gif]

Transform the data using the spline wavelet of order 4. 

(s4 = SplineFilter[4, 8];
 wtdata = WaveletTransform[data, s4];)

Set the coefficients below the threshold of 1.5 to zero, and shrink the rest. Note that the residual trend is excluded from shrinking. 

Compress[Rest[wtdata], 1.5, Shrinking -> True];

Next, we do the inverse transform and obtain the "denoised" data. 

InverseWaveletTransform[
Join[{wtdata[[1]]}, %], s4];

This shows the data after denoising. 

waveletfilter=ListPlot[%, PlotJoined -> True,
AxesLabel -> {"x", "u(x)"}];

[Graphics:Images/index_gr_10.gif]