How do I denoise a dataset?

Mathematica 4 offers a wide range of possibilities to denoise experimental data sets. This how-to demonstrates two common approaches, fourier and wavelet filtering.

Fourier filtering

This reads in the content of the data file.

data = Get["Wavelets/Data/shocknoi.dat"];

This plots the data.

noisy=ListPlot[data, PlotJoined -> True,
AxesLabel -> {"x", "u(x)"}];

Define an appropiate filter function.

FilterFunc[x_, a_] := (1. /(1 + ((((x/10)))^a)))

Generate the corresponding dataset of filter values.

filter  = 
        N[Table[FilterFunc[x, 1], {x, 1. , 4096}] + 
            Reverse[Table[FilterFunc[x, 1], {x, 1. , 4096}]]];

ListPlot[filter];

Convolve the data and the filter.

filtered = 
        ListPlot[InverseFourier[Fourier[data] Fourier[filter]], 
          PlotJoined -> True];

Spline filtering using wavelets

Load the Wavelet Explorer application package.

Needs["Wavelets`Wavelets`"]

This reads in the content of the file.

data = Get["Wavelets/Data/shocknoi.dat"];

This plots the data.

ListPlot[data, PlotJoined -> True,
AxesLabel -> {"x", "u(x)"}]

Transform the data using the spline wavelet of order 4.

(s4 = SplineFilter[4, 8];
 wtdata = WaveletTransform[data, s4];)

Set the coefficients below the threshold of 1.5 to zero and shrink the rest. Note that the residual trend is excluded from shrinking.

Compress[Rest[wtdata], 1.5, Shrinking -> True];

Next we do the inverse transform and obtain the "denoised" data.

InverseWaveletTransform[
Join[{wtdata[[1]]}, %], s4];

This shows the data after denoising.

waveletfilter=ListPlot[%, PlotJoined -> True,
AxesLabel -> {"x", "u(x)"}];