Cooking Up a Mathematica Cookbook

Automating the Creation of Production-Ready Book Authored in Mathematica

Sal Mangano (smangano@into-technology.com)

Into Technology Inc./O`Reilly Media

What is a Cookbook?

({{, , , , , }})

~50 Titles in Cookbook Series

Recipe - A Description of Something a Person Wants to Achieve

Problem - A one line statement of the problem

Solution - The shortest path solution to the problem. Minimal text, no variations, no complexities

Discussion - Explains the solution in more detail, shows variations, caveats, etc.

See Also -  References to further information in other recipes, book, online, etc.

Other Characteristics

Cookbooks are not tutorials. Some experience assumed

Nevertheless, some introductory material precedes each chapter

Example from The Mathematica Cookbook...

Understanding When to Use Longest and Shortest in Patterns

Problem

A replacement rule is not working the way you think it should.

Solution

By default sequence patterns  like a__ and a___ act as if they are surrounded by Shortest. This means they match as little as possible to still be consistent with the pattern. The following repeated replacement seems like it should shuffle items in the list around until all equal values are adjacent. It almost works but a 3 and a 1 stubbornly remain in place. This happens because on the final pass a___ matches nothing, b_ matches 1, c__ matches 1, b_ matches the third 1 and d___ matches the remainder. This results in a null transformation so ReplaceRepeated stops.

{1, 3, 1, 4, 1, 3, 4, 2, 7, 1, 8} //. {{a___, b_, c__, b_, d___} -> {b , b, a, c, d}}

Contrast this to the same transformation using Longest.

{1, 3, 1, 4, 1, 3, 4, 2, 7, 1, 8} //. {{Longest[a___], b_, c__, b_, d___} -> {b , b, a, c, d}}

By forcing a___ to match as much as it can such that the rest of the pattern is satisfied forces all sequences of identical elements separated by one or more other elements (b_, c__, b_) to be found.

Discussion

Readers familiar with regular expression recognize the solution example as illustrating the difference between greedy and non-greedy matching. This difference is the source of infinite frustration to pattern writers because depending on your test case, non-greedy patterns can appear to work. The only advice I can give is to always consider what will happen if patterns like a__ match only one item and a___ matches nothing. Often this is what you want but almost as often it is not!

A reasonable question two ask is why there is a Shortest if Longest is the default? The answer is that for string patterns (see Chapter 5) the default is reversed. You may also want to consider using Shortest to document that it is what you really meant but this probably should only be done for portions of the pattern that are up front.

Also keep in mind that if multiple Shortest or Longest directives are used then the ones that appear earlier are given higher priority to match the shortest or longest number of elements respectively.

{1,2,3,4,5} /. {Shortest[a__], Shortest[b__]} -> {{a},{b}}

{1,2,3,4,5} /. {Longest[a__], Longest[b__]} -> {{a},{b}}

See Also

O'Reilly's Mastering Regular Expressions by Jeffery E. F. Friedel has an extensive discussion of Greedy versus Lazy matching that is relevant to understanding Longest and Shortest.

Present Chapter Outline of The Mathematica Cookbook (Aprox 850 pgs)

1) Numerics
2) Functional Programming
3) Data Structures
4) Patterns and Rule - Based Programming
5) Strings and Text Processing
6) 2 D Graphics
7) 3 D Graphics
8) Image Processing
9) Audio Synthesis and Processing
10) Calculus and Differential Equations
11) Statistics and Data Analysis
12) Physics and Engineering
13) Financial Engineering
14) Interactivity
15) Web Mathematica
16) Parallel Mathematica
17) Publishing with Mathematica
18) Interfacing to Other Languages
19) Meta Mathematica
20) Debugging and Testing

Converting the Mathematica Cookbook to DocBook XML

Problem

You want to Author a Book in the Mathematica Front End but your publisher needs DocBook Output

({{, , }, {Cells in Mathematica Syntax,  , DocBook in XML Syntax}})

Solution

Solution 1) Export Notebook to XHTML + MathML and use XSLT to Transform to DocBook

Solution 2) Export Notebook to Expression ML and use XSLT to Transform to DocBook

Solution 3) Transform Notebook to DocBook Using Mathematica Code

Discussion

The Rest of My Talk

General Advice on Publishing with Mathematica

Stylesheets are essential    ⟹

Grouping is essential  ⟹

Tagging is recommended ⟹

What is DocBook?

XML Standard for Technical Documents

Presentation Neutral Markup

Developed in 1991 by O`Reilly and HAL Computer Systems. Now managed by OASIS,

Example - subset of Longest/Shortest Recipe Above

<chapter id="ch04">
	<title>Patterns and Rule-Based Programming</title>
	<sect1>
		<title>Understanding When to Use Longest and Shortest in Patterns<title>
		<sect2>
			<title>Problem</title>
			<para>A replacement rule is not working the way you think its should.</para>
		</sect2>
		<sect2>
			<title>Solution</title>
			<para>By default sequence patterns  like a__ and a___ act as if they are surronded by Shortest. ...</para>
		</sect2>
	</sect1>
<chapter>

Export Notebook to XHTML + MathML and use XSLT to Transform to DocBook

Save As XHTML + MathML

Transform Using XSLT

Looked like the obvious solution for me since XSL is my specialty

WRONG!

Challenges

Hierarchical Grouping is lost when you export.

There are XSLT 2.0 Programming Techniques that can recover the structure.

The XSLT becomes more complex and error prone

Even an XSLT Expert does not want to be constantly debugging XSLT when he is supposed to be doing Mathematica

Deal Breakers

Export to XHTML+MathML will sometimes choke resulting in malformed XML

Loss of Information in Output ⟹

Export Notebook to Expression ML and use XSLT to Transform to DocBook

Use Export[] to export the Notebook to ExpressionML

Write XSLT to transform ExpressionML to DocBook

Challenges

ExpressionML is a very low level XML vocabulary. Only 4 elements

Expression

Function

Symbol

String

Deal Breakers

Does not use MathML

You definitely don't want to write your own MathML translation in XSLT!

Still has similar problem of information loss

Does not automatically export images to GIFS like XHTML export does

Transform Notebook to DocBook Using Mathematica Code

Write Mathematica Code to transform Notebook Cells to XMLObject/XMLElement

Serialize the XMLObject to XML

Use MathML Export When You want XML

Use Image Export When you want Image Files

Ultimate control of how information is converted to text.

Challenges

Notebook Cell Language is not 100% Documented

Bit of Reverse Engineering Needed

Code May Break Between Mathematica Versions

Getting what you want can still be challenging!

Deal Breakers

Have not found any.

Unlikely that I will. You can always code around these.

Methodology

Table Driven Book Organization  ⟹

Preprocessing to automate, validate and clean up notebooks ⟹

Use Tags and Hyperlinks for Figure and Recipe Cross References  ⟹

Recursive Descent Pattern Direct Translation  ⟹

Akin to how XSLT Works

Results in a program that is easy to extend to new situations

Convert to MathML using Mathematica Export[ ]

Use MathML as much as possible

Externalize MathML in files separate from DocBook

Use XLink to link MathML to DocBook XML

Convert Everything that Can't be Represent in XML to GIF Images

Images are linked to docbook using docbook conventions

Images associated with a tag are automatically given a caption

References to figures in the text are automatically resolved

The User Interface

Generate a Simple GUI to Drive the Process ⟹

A Goal That Will Have to wait for Mathematica 10 ☺

Useful 3rd-Party Tools

XML Mind Editor for view DocBook linked to MathML in presentation form

XML Spy for viewing and validating DocBook XML

Future Enhancements

Support for a larger subset of DocBook then needed by Cookbook

DocBook is a fairly large vocabulary

My code only supports about a dozen or so elements of the DocBook vocabulary

A more data driven mapping of Notebook elements to DocBook elements

Make it easier to customize

Miscellaneous

I am looking for Mathematica users interested in contributing Recipes to the book

Some areas I am interested in include (but not limited to):

Finance

Physical Science and Engineering

Mathematics Education

Bioinformatics

Audio Synthesis and Audio Processing

Users of Linked In: There is a Mathematica Group that I am the moderator

http://www.linkedin.com/groups?gid=62614

Or just Search the Groups Directory

If you'd like to track the progress of the Mathematica Cookbook on Facebook

http://www.facebook.com/pages/Mathematica-Cookbook/30325088182

Or Search for "Mathematica Cookbook"

Questions?