Programmatic Report Generation with Mathematica

Adam Berry

Wolfram Research, Inc

Note: This presentation includes code and references to a package currently under development.

Introduction

ReportGeneration is a new tool that will take a template notebook and data and generate a report notebook.

Motivation

An easier way to generate notebooks.

Data analysis and reporting using Mathematica.

Basics

Once we have loaded the application, we can then access the example templates.

<<ReportGeneration`

NotebookOpen[FindReportTemplate["BasicTemplate"]]

The template consists of a header that declares necessary metadata for the runtime to process the template and data.

There are two types of points where insertions can be made into a report;

EvaluateMarker - a Mathematica expression that will be evaluated and its output inserted into that location

VariableMarker - a placeholder for data that will be passed in at runtime.

When a report is executued, all substitutions are made, and then the notebook is evaluated using NotebookEvaluate

The final thing to note is that we can control the behavior of the input cells, they can be left as normal, hidden or deleted. In this example we hide the input.

ReportGenerate["basic.nb","BasicTemplate",{"ReportTitle"->"Test Title"}]

Grouped Data and Content

Report Grouping

More powerful is the ability to group elements of the report, that can be repeated dependent on data values that are passed in.

NotebookOpen[FindReportTemplate["BasicGroup"]]

CellTags are used in the notebook to define the hierarchies in the cell content, and must match the data grouping hierarchy declared in the header.

In the header we also declare a data transform, this allows us to pass in tabular data, which is then transformed into the appropriate data hierarchy

To generate the report;

ReportGenerate["basic.nb","BasicGroup",{"Variable"->"Test1",DataTransform["Depth1",{{"val1","val2"},{"val3","val4"}}]}]

Scoping of Group Variables

Each level of the hierarchy inherits the variables from above. ie all variables at this level, and all levels above are in scope, but all lower down not in scope.

Data Hierarchy

You can create the data hierarchy separately of running the report, using ReportDataGenerate. You can then pass this into ReportGenerate bypassing the transform at runtime. This allows you to experiment with the data hierarchy, as well as some of the further features which we’ll talk about later.

ReportDataGenerate["BasicGroup",{"Variable"->"Test1",DataTransform["Depth1",{{"val1","val2"},{"val3","val4"}}]}]

Merging Data

The data transform will merge records as appropriate recursively through the hierarchy. A good example of this is using the BookSearch example

NotebookOpen[FindReportTemplate["BookSearch"]]

This shows a deeper hierarchy, and shows merging at two levels of the data. It also shows that you can pass data that isn’t a string, you can use all expressions as well.

data={{"James Smith","PA","Gardening","2008-02-20",44.95,"10"},{"James Smith","PA","Bug Squashing","2007-11-17",69.95,"10"},{"James Smith","PB","2Gardening","2008-02-20",44.95,"10"},{"James Smith","PB","2Bug Squashing","2007-11-17",99.95,"10"},{"Bob Jones","PA","Foo and Bar","2006-08-10",29.95,"10"},{"Bob Jones","PA","Network Outages","2003-10-23",59.95,"10"},{"Bob Jones","PB","2Foo and Bar","2006-08-10",229.95,"10"},{"Bob Jones","PB","2Network Outages","2003-10-23",259.95,"10"}};

ReportGenerate["report.nb","BookSearch",{"ReportTitle"->"Amazon Search","AuthorList"->{"James Smith","Bob Jones"},"PublisherList"->{"PA","PB"},DataTransform["Author",data]}]

Runtime Notebook Manipulation

The last example also showed runtime manipulation of the notebook based upon the values of the data.

There is an option that you can use in the data hierarchy called GroupStyleFunction. This receives two inputs; the current cell expression and the list of rules of data that is currently in scope.

The GroupStyleFunction is applied to all cells that are in the matching place in the hierarchy, and will be included whether you use the DataTransform or not.

In the example above, we see that the FontColor was made red, but we can use SetCellOptions (an included utility function) to conveniently change any of the style options easily. But, as you get the entire cell expression this is a very flexible and powerful runtime manipulation.

Template Authoring

There is also a palette which supplies an easy to use template authoring environment.

Much of the runtime operations is based on CellTags, but the intent is that using the palette you will never need to edit the tags yourself. Tools for creating an empty template, editing the group hierarchies, adding insertion points and even verifying the report template are supplied.

Advantages

Mathematica really is ideal for a reporting tool. As well as the flexibility and typesetting available, you also have the full power for data analysis and visualization.

Additionally a range of importers and data links make it easy to get your data into a report regardless of the format it is currently locked into.

Plans

Should be part of the Mathematica 8 release cycle, if not quite for 8.0.0 it will be in 8.0.1.

Version 2 plans are underway, some ideas we currently have (but we’d love to hear more!);

Pass through/subreports

Emitter/Publisher additions - the ability to automatically create other formats than notebooks

Data extraction/mining integration

Technology crossover such as WebServices, webMathematica etc