Statistics in Mathematica

A Brief Overview

Darren Glosemeyer

Wolfram Research, Inc.

Overview

The following areas will be discussed:

Descriptive and Exploratory Statistics

Descriptive measures, counting, smoothing, and clustering

Symbolic Distributions

Properties of distributions, generating random numbers

Regression Models

Linear and nonlinear regression, ANOVA

Related graphics, data import, data operations and symbolic and numeric computation will be seen in the context of examples.

Descriptive and Exploratory Statistics

Descriptive Measures

Here are 20 uniformly distributed random numbers between 0 and 10.

data=RandomReal[10,20]

This gives descriptive measures of data based on the first 4 central moments.

{Mean[data],Variance[data],Skewness[data],Kurtosis[data]}

The data can be visualized as a sequence using ListPlot.

ListPlot[data]

A number of other common data visualizations are included in packages

Needs["StatisticalPlots`"]
Needs["Histograms`"]
Needs["BarCharts`"]
Needs["PieCharts`"]

Here a histogram and a stem-and-leaf plot for the same data set.

{{Histogram[data],StemLeafPlot[data]}}//GraphicsGrid

These are bar and pie charts based on count data.

{{BarChart[{5,2,10,4,6}],PieChart[{5,2,10,4,6}]}}//GraphicsGrid

Alternative measures of location and dispersion are included.

{Median[data],RootMeanSquare[data],GeometricMean[data]}

{StandardDeviation[data],MeanDeviation[data],MedianDeviation[data]}

Quantile based statistics are also built-in.

Quantile[data,.6]

InterquartileRange[data]

A box-and-whisker plot visualizes the spread of the data.

BoxWhiskerPlot[data]

Descriptive functions also work on matrix data, returning the descriptive measures for columns.

multidata=RandomReal[10,{10,3}]

Mean[multidata]

MedianDeviation[multidata]

Quantile[multidata,.6]

The covariance and correlation can be computed for two lists of numbers.

data2=RandomReal[10,20]

Covariance[data,data2]

Correlation[data,data2]

A side-by-side stem-and-leaf plot can be used to compare two univariate data sets.

StemLeafPlot[data,data2]

Covariance and correlation matrices can also be computed for data matrices.

Covariance[multidata]//MatrixForm

Correlation[multidata]//MatrixForm

Counting Functions

Counting based on some categorization often provides a useful description of data. Collecting data into bins is one common categorization.

This counts continuous data in bins of width 2 from 0 to 10.

BinCounts[data,{0,10,2}]

Here explicit cutoffs of 0, 4, 5, 9, and 10 are used.

BinCounts[data,{{0,4,5,9,10}}]

Multidimensional data can be counted in bins of appropriate dimension.

BinCounts[multidata,{0,10,5},{0,10,5},{0,10,5}]

The values in the bins can be obtained using BinLists.

BinLists[data,{0,10,2}]

For discrete or nominal data, it is often useful to count the distinct values.

Tally counts the frequencies of distinct elements.

Tally[{c,a,b,a,d,c,c,b,a}]

Commonest gives a list of the most frequently occurring elements.

Commonest[{c,a,b,a,d,c,c,b,a}]

The Count function can be used to obtain more general counts based on a pattern.

For instance, the number of occurrences of a particular element in a list or the number of numbers in a list might be counted.

Count[{c,a,b,a,d,c,c,b,a},b]

Count[{3,a,c,4,1,b,3,Sin[x],4,3},_?NumberQ]

Data Smoothing

Built-in smoothers include MovingAverage, MovingMedian, and ExponentialMovingAverage.

This gives a two-term moving average.

MovingAverage[data,2]

Weighted moving averages can be obtained by giving a list of weights.

MovingAverage[data,{1/3,2/3}]

This gives a three-term moving median.

MovingMedian[data,3]

The following gives exponentially smoothed data using smoothing constant 1/4.

ExponentialMovingAverage[data,1/4]

Other smoothers can be constructed using functions such as ListCorrelation and ListConvolve, and arithmetic and linear algebra functions.

Clustering

Clustering of data is often used to separate data into groups, or to try to identify subgroups within a data set.

The main clustering function is FindClusters, which can be used on numeric, binary/Boolean or string data.  A number of distance and dissimilarity functions are also included.

Names["*Distance"]

Names["*Dissimilarity"]

Used with FindClusters, these define measures for separating data.

A Numeric 3D Data Example

As an example, we will start with 5 clusters that are samples from trivariate normal distributions.

Needs["MultivariateStatistics`"]

origclusters=BlockRandom[SeedRandom[12345];
muvals=RandomReal[{-10,10},{5,3}];
sigmavals=RandomReal[WishartDistribution[{{1,.3,-.3},{.3,1.5,.4},{-.3,.4,1.2}},4],5];
Join[MapThread[RandomReal[MultinormalDistribution[#1,#2],RandomInteger[{75,100}]]&,{muvals,sigmavals}]]];

Next we will mix up the elements of the original data so the data are not already grouped, and use FindClusters identify clusters in the data.

newclusters=BlockRandom[SeedRandom[1];RandomSample[Flatten[origclusters,1]]];

clusters=FindClusters[newclusters];

FindClusters identifies 4 distinct clusters.

Length[clusters]

Visualization of Clusters

For numeric data in up to three dimensions, the clusters are easy to visualize with list plotting functions.

Here we compare the original data sets and the clusters found by FindClusters.

{{ListPointPlot3D[origclusters],ListPointPlot3D[clusters]}}//GraphicsGrid

The desired number of clusters can be specified if that number is known.

{{ListPointPlot3D[origclusters],ListPointPlot3D[FindClusters[newclusters,5]]}}//GraphicsGrid

A cluster hierarchy can also be visualized using dendrogram plots. The HierarchicalClustering package is needed for this type of visualization.

Needs["HierarchicalClustering`"]

As an example, take the abstract of this talk split into string elements.

textdata=StringSplit["This talk will give a brief overview of statistics functions in Mathematica. The overview will touch on features for descriptive statistics, symbolic distributions, exploratory data analysis, model fitting, and related charts and plots."]

The following generates a dendrogram labelling the leaves with the string elements they represent.

DendrogramPlot[textdata,LeafLabels->(#&),Orientation->Left]

Symbolic Distributions

A total of 36 univariate continuous and discrete distributions are built into the kernel.  An additional 8 distributions derived from multivariate distributions are included in the MultivariateStatistics package.

GumbelDistribution, InverseGaussianDistribution, MaxwellDistribution, TriangularDistribution, BetaBinomialDistribution, BetaNegativeBinomialDistribution and ZipfDistribution were all added in version 6.0.

Distribution Properties

Many properties of distributions such as PDF, CDF, mean, variance and characteristic function.

This gives moment-based measures for a noncentral (χ^2) distribution.

{Mean[NoncentralChiSquareDistribution[ν,λ]],Variance[NoncentralChiSquareDistribution[ν,λ]],Skewness[NoncentralChiSquareDistribution[ν,λ]],Kurtosis[NoncentralChiSquareDistribution[ν,λ]]}

General expectations can be computed as well.  Here is the (n^th) raw moment of a geometric distribution.

ExpectedValue[x^n,GeometricDistribution[p],x]

The characteristic function of a Laplace distribution.

CharacteristicFunction[LaplaceDistribution[μ,β],t]

Visualization

Functions for plotting functions can be useful for visualizing symbolic properties of distributions, such as distribution and density functions.

This plots the cumulative distribution function for a gamma distribution.

Plot[CDF[GammaDistribution[3,5],x],{x,0,50}]

This shows the probability mass function for a binomial distribution.

ListPlot[Table[{i,PDF[BinomialDistribution[20,.3],i]},{i,0,20}],Filling->Axis,FillingStyle->Black]

This plots the density for a bivariate t-distribution.

Plot3D[PDF[MultivariateTDistribution[{{1,1/2},{1/2,1}},20],{x,y}],{x,-3,3},{y,-3,3}]

Random Number Generation

Random numbers can be generated from each of the defined distributions.  They are obtained using RandomInteger for discrete distributions and RandomReal for continuous distributions.

Random elements can be generated one at a time or in a list or tensor of arbitrary dimension.

RandomInteger[NegativeBinomialDistribution[20,1/3]]

RandomInteger[NegativeBinomialDistribution[20,1/3],5]

RandomInteger[NegativeBinomialDistribution[20,1/3],{2,3,3}]

Elements from continuous distributions can also be generated in arbitrary precision.

RandomReal[BetaDistribution[3,4]]

RandomReal[BetaDistribution[3,4],5]

RandomReal[BetaDistribution[3,4],{3,5}]//MatrixForm

RandomReal[BetaDistribution[3,4],WorkingPrecision->20]

The same holds for multivariate distributions.

mu={0,0,0,0};
sigma={{1.2,2.3,-2.3,-0.2},
{2.3,9.7,3.2,-2.},
{-2.3,3.2,16.8,-2.3},
{-0.2,-2.,-2.3,0.7}};

This give a random observation from a four-dimensional normal distribution.

RandomReal[MultinormalDistribution[mu,sigma]]

The correlation structure of the data can be visualized using a pairwise scatter plot.

PairwiseScatterPlot[RandomReal[MultinormalDistribution[mu,sigma],100]]

Regression Models

Mathematica includes packages for linear and nonlinear regression and ANOVA.

Linear Regression

Here we import some data from an Excel document.

regressdata=Import[ToFileName[NotebookDirectory[],"linearfitdata.xls"],{"Sheets",1}]

The data can be fitted to a linear model using Regress.

Needs["LinearRegression`"]

Regress[regressdata,{x,y,z},{x,y,z}]

Regress generates a default set of diagnostics and results.  Desired results can be specified via the RegressionReport option, and possible RegressionReport values can be obtained from the RegressionReportValues function.

RegressionReportValues[Regress]

Elements from the RegressionReport can be extracted via replacement rule and then worked with.

{stresids,cookd}={StandardizedResiduals,CookD}/.Regress[regressdata,{x,y,z},{x,y,z},RegressionReport->{StandardizedResiduals,CookD}]

In a linear regression, we might wish to look at plots of the standardized residuals and Cook's distances to access approximate normally of errors and influence of individual points.

ListPlot[stresids]

ListPlot[cookd,Filling->Bottom,FillingStyle->Black,PlotRange->All,Ticks->{Range[1,20],Automatic}]

Nonlinear Regression

The NonlinearRegress function in the NonlinearRegression package works for nonlinear models in much the same way Regress does for linear models.

Needs["NonlinearRegression`"]

NonlinearRegress[regressdata,Sqrt[a*x+b*y*z],{a,b},{x,y,z}]

The possible report values for NonlinearRegress differ somewhat from those for Regress because some results are only meaningful for linear or nonlinear regression.

RegressionReportValues[NonlinearRegress]

Here we obtain the functional form of the fitted model.

BestFit/.NonlinearRegress[regressdata,Sqrt[a*x+b*y*z],{a,b},{x,y,z},RegressionReport->BestFit]

Constraints can also be included. Some regression report values, however, may not be valid.

Here b is constrained to be greater than 15.

NonlinearRegress[regressdata,{Sqrt[a*x+b*y*z],b>15},{a,b},{x,y,z}]

ANOVA

The analysis of variance (ANOVA) model is a categorical linear regression model.

Here we import some data from a text file.

anovadata=Import[ToFileName[NotebookDirectory[],"anovadata.txt"],"Table"]

The ANOVA function is defined in the ANOVA package.

Needs["ANOVA`"]

By default an ANOVA table and table of cell means are returned.

ANOVA[anovadata,{x,y},{x,y}]

The cell means can be omitted and significance tests performed if desired.

This gives an analysis of variance performing Bonferroni and Tukey tests for significant differences between levels of categorical variables.

ANOVA[anovadata,{x,y},{x,y},CellMeans->False,PostTests->{Bonferroni,Tukey}]

Interaction terms can be included in the model as products.

ANOVA[anovadata,{x,y,x*y},{x,y},CellMeans->False]

Conclusion

We have seen a brief introduction to statistics in Mathematica. This is only a broad sampling of the statistical functionality built into Mathematica.

Many features including numeric, symbolic, and graphical functionality, and extensive file I/O and database support are also extremely important in statistical applications. The integration of these areas in Mathematica makes it possible to not only solve typical problems, but to define and explore new applications and analyses.