Time Series

Gosia Konwerska

Wolfram Research

TimeSeries and EventSeries are new objects introduced and comprehensively supported in Mathematica 10 to add convenience to working with chronologically ordered data. In the talk we will show examples of how to obtain, manipulate, and work with these objects, as well as explore the Wolfram Language functionality for data analytics on time series.

Basic Examples - Creation, Sources, and Visualizations

EventSeries

EventSeries - discrete data

data={1,4,3,6,4,3};

es=EventSeries[data]

Value at a given time:

es[3]

The values between data points are not being interpolated.

es[3.4]

ListPlot[es]

TimeSeries

TimeSeries - continuous data

data={1,4,3,6,4,3};

ts=TimeSeries[data]

Values between data point are being interpolated:

{ts[3],ts[3.4],ts[4]}

ListLinePlot[ts]

Setting option ResamplingMethod specifies how values in bewteen data points should be interpolated.

ts1=TimeSeries[data,ResamplingMethod->None];
{ts1[3],ts1[3.4],ts1[4]}

ts2=TimeSeries[data,ResamplingMethod->{"Constant",0}];
{ts2[3],ts2[3.4],ts2[4]}

The deafult is interpolation with order 1:

ts3=TimeSeries[data,ResamplingMethod->{"Interpolation",InterpolationOrder->1}];
{ts3[3],ts3[3.4],ts3[4]}

Specifying Times with Dates

data={1,4,3,6,4,3};

ts=TimeSeries[{data},{"3 Jan 2013",Automatic,"Day"}]

DateListPlot[ts]

Time-Value Pairs

TimeSeries and EventSeries can be specified using a path:

path={{1,3.2},{2,4.5},{5,3.2},{6,3.4}};

ts=TimeSeries[path]

ListLinePlot[ts]

Also most functions that accept argument value pairs will work with TimeSeries.

TimeSeries from Wolfram Alpha

{Michael,John}=TimeSeries/@WolframAlpha["Michael vs John",{{"History:GivenNameData",1},"TimeSeriesData"}]

DateListPlot[{Michael,John},PlotLegends->{"Michael","John"}]

TimeSeries from DataPaclets

pressure=AirPressureData[Entity["City",{"Champaign","Illinois","UnitedStates"}],{DateObject[{2014,1,1}],DateObject[{2014,1,2}]}]

DateListPlot[pressure]

TimeSeries from FinancialData

prices=TimeSeries@FinancialData["SBUX","Close",{{2014,5,1},{2014,9,30}}]

DateListPlot[prices]

The catch:

RegularlySampledQ[prices]

The *good* solution is to create regularly sampled data:

aux=FinancialData["SBUX","Close",{{2014,5,1},{2014,9,30}}];
prices2=TimeSeries[{aux[[All,2]]},{{2014,5,1},{2014,9,30},"BusinessDay"},HolidayCalendar->{"UnitedStates","NYSE"}]

RegularlySampledQ[prices2]

The *bad* quick3 fix:

prices3=TimeSeries[aux,TemporalRegularity->True]

RegularlySampledQ[prices3]

The *ugly* truth - the minimum time increment is not exact:

prices2["MinimumTimeIncrement"]

prices3["MinimumTimeIncrement"]

UnitConvert[Quantity[%,"Second"],"Day"]

Note on TemporalData

In Mathematica 9 the object TemporalData was introduce to store the result of simulation of a random process.

TimeSeries and EventSeries are SINGLE PATH objects. Hence for a given specified time stamp they return a value:

ts=TimeSeries[];

ts[4]

TemporalData allows multiple paths and for a defined time stamp is understood as a sllice distribution. Introduced in Mathematica 9 RandomFunction returns a TemporalData object:

td=RandomFunction[ARProcess[{.4},1],{20},100]

td[4]

Plot[CDF[td[4],x],{x,-2,2}]

Multiple TimeSeries objects can be combined into a single TemporalData:

{Michael,John,Jennifer}=TimeSeries/@WolframAlpha["Michael vs John vs Jennifer",{{"History:GivenNameData",1},"TimeSeriesData"}]

td=TemporalData[{Michael,John,Jennifer}]

DateListPlot[td,PlotLegends->{"Michael","John","Jennifer"}]

Also, multivariate data may be interpreted differently by default:

data=RandomVariate[NormalDistribution[],{7,3}];

Dimensions[data]

TimeSeries has only one option - has to create single path object:

ts=TimeSeries[data,Automatic]

ts["ValueDimensions"]

ts["Path"]

TemporalData understands this as multipath object:

td=TemporalData[data,Automatic]

td["ValueDimensions"]

TemporalData here has 7 univariate paths of 3 elements each:

td["Paths"]

Specify ValueDimensions to get single vector valued path:

TemporalData[data,Automatic,ValueDimensions->3]

%["Paths"]

Basic Manipulations

Shifting

Shifting times with TimeSeriesShift

data=RandomFunction[ARIMAProcess[{.3},1,{.4},1],{9}]

st=TimeSeriesShift[data,1]

ListLinePlot[{data,st}]

Shifting values with TimeSeriesMap

sv=TimeSeriesMap[#+1&,data]

ListPlot[{data,sv},Filling->{1->{2}}]

Extracting

TimeSeriesWindow

ts=TimeSeries[];

ts1=TimeSeriesWindow[ts,{"1 Jan 1993","31 Dec 2003"}];
ts2=TimeSeriesWindow[ts,{"1 Jan 2004",Automatic}];

DateListPlot[{ts1,ts2},GridLines->{{"31 Dec 2003"},Automatic}]

Inserting

Insert time-value pair into existing path:

TimeSeriesInsert[{{0,x1},{2,x3},{3,x4}},{1,x2}]

Insert a point into a time series:

ts=TimeSeries[{{0,x1},{2,x3},{3,x4}}]

ts["Path"]

TimeSeriesInsert[ts,{1,x2}]

%["Path"]

Join two time series:

ts1=TimeSeries[{{0,x1},{2,x3},{3,x4}}]

ts2=TimeSeries[{{1,x2},{4,x5}}]

ts12=TimeSeriesInsert[ts1,ts2]

ts12["Path"]

Resampling

TimeSeriesResample

This function could be used to downsample:

data=TimeSeries[];

Select every other point - subsampling in steps of 2:

data1=TimeSeriesResample[data,2]

data1["Path"]

It could also be used to create denser data, but it requires specifying ResamplingMethod, that is how to assign values to the times between given data; by default interpolation of order 1 is being used:

data2=TimeSeriesResample[data,1/2]

data2["Path"]

Compare to using 0 order interpolation:

data3=TimeSeriesResample[data,1/2,ResamplingMethod->{"Interpolation",InterpolationOrder->0}]

data3["Path"]

ListPlot[{data,data2,data3},Joined->{False,True,True},PlotTheme->{"CoolColors","PlotMarkers"},Filling->{1->0},Frame->True,FillingStyle->Thick,PlotLegends->{"data","upsampled - order 1","upsampled - order 0"}]

Resample irregularly sampled time series to create its regularly sampled version:

prices=TimeSeries@FinancialData["SBUX","Close",{{2014,5,1},{2014,9,30}}]

Check the step:

prices["MinimumTimeIncrement"]

UnitConvert[Quantity[%,"Second"],"Day"]

Check if regularly sampled:

RegularlySampledQ[prices]

Why? Beacause it is sampled on days when stock exchange is open. We can resample by “BusinessDay”:

prices1=TimeSeriesResample[prices,"BusinessDay"]

The paths are identical:

prices["Path"]==prices1["Path"]

Resampled time series is regularly sampled:

RegularlySampledQ[prices1]

Rescaling

TimeSeriesRescale

This is like “renaming” the time stamps:

data=TimeSeries[];

data["Times"]

Make times even:

data1=TimeSeriesRescale[data,{2,12}]

data1["Times"]

Useful to switch between different types of time stamps:

ts=TimeSeries[];

ts["PathLength"]

ts1=TimeSeriesRescale[ts,{1,21}]

The new times are propotional to business days in the original time series:

ts1["Times"]

ts1["Times"]//Length

To get time series on integers now use TimeSeriesResample:

ts11=TimeSeriesResample[ts1,1]

ts11["Times"]

Going back to dates:

ts2=TimeSeriesRescale[ts1,{"01 May 2014","30 May 2014"}]

Because of the conserved propotionality we get time stamps by “BusinessDay.” But trying to rescale integer time stamps back to dates creates regularly sampled time series with larger MinmumTimeIncrement:

ts22=TimeSeriesRescale[ts11,{"01 May 2014","30 May 2014"}]

ts22["MinimumTimeIncrement"]

ts2["MinimumTimeIncrement"]//UnitConvert[Quantity[#,"Second"],"Day"]&

Descriptive Statistics

Descriptive statistics functions work on the values only, ignoring the time stamps:

ts=TimeSeries[];

DateListPlot[ts]

{m,v}={Mean[ts],Variance[ts]}

DistributionFitTest[ts,dist=NormalDistribution[m,Sqrt[v]],"TestConclusion"]

QuantilePlot[ts,dist]

Map[#[ts]&,{Histogram,BoxWhiskerChart}]

Other Manipulations

MovingMap and Filtering

Use MovingAverage or general MovingMap to smooth a time series:

data=RandomFunction[ARMAProcess[{5/6},{1/6},1],{1000}];

mavg=MovingAverage[data,100];

ListLinePlot[{data,mavg},PlotStyle->{Opacity[.4],Thick}]

Use MovingMap with an arbitrary function:

mq=MovingMap[Quartiles,data,100];

ListLinePlot[mq]

Filters also work with regularly sampled time series, however they operate on values only and disregard time stamps:

minf=MinFilter[data,10];
maxf=MaxFilter[data,10];

ListPlot[{data,minf,maxf},Joined->{False,True,True}]

Names["*Filter"]

Aggregation

Use TimeSeriesAggregate to compute aggregated statistics:

ExampleData[{"Statistics","USAccidentalDeaths"},"LongDescription"]

data=ExampleData[{"Statistics","USAccidentalDeaths"},"EventSeries"]

Mean values over quarters:

qavg=TimeSeriesAggregate[data,{3,"Month"}]

DateListPlot[{data,qavg},Joined->{False,True}]

Calculate quarterly totals:

qtotals=TimeSeriesAggregate[data,{"Quarter",Right},Total]

DateListPlot[{data,qtotals},Joined->False,Filling->{2->0}]

General Maps

TimeSeriesMap maps over the values of a time series:

ts=TimeSeries[RandomVariate[NormalDistribution[],10],Automatic]

Shift the values:

min=Min[ts]

map1=TimeSeriesMap[#+Abs@min&,ts]

ListPlot[{ts,map1},Filling->{1->{2}}]

TimeSeriesThread maps over time series - add two time series:

map2=TimeSeriesThread[#[[1]]+#[[2]]&,{ts,map1}]

ListPlot[{ts,map1,map2},Joined->{False,False,True}]

TimeSeriesMapThread maps over times AND values of a time series - add linear trend:

slope=2;
map3=TimeSeriesMapThread[slope*#1+#2&,map2]

ListLinePlot[{map2,map3}]

In Depth - Accumulate

Example1

data=RandomFunction[BernoulliProcess[.6],{20}]

ListPlot[data]

Cumulative number of events:

res=Accumulate[data]

ListPlot[res]

Check:

res["Values"]==Accumulate[data["Values"]]

Example 2

records={{"1 Jul 2014",8.3},{ "2 Jul 2014",3.3},{"3 Jul 2014",5},{"7 Jul 2014",8},{"8 Jul 2014",3},{"9 Jul 2014",4.5},{"10 Jul 2014",5},{"11 Jul 2014",7.9},{"14 Jul 2014",3.1},{"15 Jul 2014",5.2},{"16 Jul 2014",4.1},{"17 Jul 2014",6.3},{"18 Jul 2014",7.1},{"21 Jul 2014",7.4},{"22 Jul 2014",4.1},{"23 Jul 2014",2.3},{"24 Jul 2014",3.2},{"25 Jul 2014",7.1},{"28 Jul 2014",8.1},{"29 Jul 2014",7.6},{"30 Jul 2014",7.1},{"31 Jul 2014",5.3}};

ts=TimeSeries[records]

DateListPlot[ts]

res=Accumulate[ts]

res["Values"]

Accumulate[records[[All,2]]]

Check:

res["Values"]==Accumulate[records[[All,2]]]

The difference is that in accumulating ts[“Values”] all values are considered reguraly spaced. However, the original data is not regularly sampled:

RegularlySampledQ[ts]

In such a case Accumulate resamples according to the MinimumTimeIncrement with InterpolationOrder 1 and then accumulates all the upsampled values:

ts1=TimeSeriesResample[ts]

Accumulate and then downsample according to the original time stamps:

res1=TimeSeriesResample[Accumulate[ts1],{ts["Dates"]}]

res1["Path"]==res["Path"]

To get correct result from Accumulate we need to make sure that the input time series is regularly sampled. One way is to define time series in a way that secures regular time stamps:

ts2=TimeSeries[{records[[All,2]]},{"1 Jul 2014","31 Jul 2014","BusinessDay"}]

RegularlySampledQ[ts2]

res2=Accumulate[ts2]

res2["Values"]==Accumulate[records[[All,2]]]

Another way is to force temporal regularity to be True:

ts3=TimeSeries[ts,TemporalRegularity->True]

res3=Accumulate[ts3]

Check:

res3["Values"]==Accumulate[records[[All,2]]]

Missing Data

MissingDataMethod specifies how the missing values should be handled:

records={{"1 Jul 2014",8.3},{ "2 Jul 2014",3.3},{"3 Jul 2014",5},{"7 Jul 2014",8},{"8 Jul 2014",Missing[]},{"9 Jul 2014",7},{"10 Jul 2014",5},{"11 Jul 2014",7.9},{"14 Jul 2014",3.1},{"15 Jul 2014",5.2},{"16 Jul 2014",4.1},{"17 Jul 2014",6.3},{"18 Jul 2014",7.1},{"21 Jul 2014",7.4},{"22 Jul 2014",4.1},{"23 Jul 2014",2.3},{"24 Jul 2014",3.2},{"25 Jul 2014",7.1},{"28 Jul 2014",8.1},{"29 Jul 2014",7.6},{"30 Jul 2014",7.1},{"31 Jul 2014",5.3}};

ts=TimeSeries[records[[All,2]],{"1 Jul 2014","31 Jul 2014","BusinessDay"}]

ts["Values"]

Fix a value:

m=Mean[DeleteMissing[records[[All,2]]]];
ts1=TimeSeries[records[[All,2]],{"1 Jul 2014","31 Jul 2014","BusinessDay"},MissingDataMethod->{"Constant",m}]

Interpolate:

ts2=TimeSeries[records[[All,2]],{"1 Jul 2014","31 Jul 2014","BusinessDay"},MissingDataMethod->{"Interpolation",InterpolationOrder->1}]

DateListPlot[#,GridLines->{{"8 Jul 2014"},None}]&/@{ts,ts1,ts2}

res=Accumulate[ts];
res1=Accumulate[ts1];
res2=Accumulate[ts2];

DateListPlot[#,GridLines->{{"8 Jul 2014"},None}]&/@{res,res1,res2}

Last[#["Values"]]&/@{res,res1,res2}

Thank you!

Next Step: Workshop Time Series Modeling 

Fri 24 Oct at 1:30pm 

in Spartan/GoldenGopher

New Time Series Functions