Notebooks for
Integrated Applications

John M. Novak, April 1994

The Mathematica  notebook interface is a useful tool for informing users and creating "live" documents. These characteristics allow notebooks to be used to good effect in any Mathematica-based application. A number of issues of good notebook design need to be considered in this context, including appropriate contents and portability. Some of the guidelines developed for the Electrical Engineering Pack (referred to hereafter as the "EE Pack") address these issues, and are discussed in this document.

What Should Notebooks Contain?

The notebook front end provides Mathematica with a very flexible document interface. However, in the context of a Mathematica-based application, only some of the possible ways of using the front end should be employed. A notebook in an application is best used as an expository tool, or as application documentation. Generally it is not a good idea to place the application code in notebooks (or, at least, not in notebooks intended to be opened by the user).

Mathematica applications can take many forms, so very few of the rules given here are hard and fast. However, general principles of good design suggest that you use techniques such as information hiding to simplify the interface of your application. This will usually give your users a better experience with your application, and make it more useful overall.

This principle means placing the implementation code in packages, and showing the use of those packages in the notebooks. While notebooks can be useful for documenting code as it is developed, this information is generally not useful to users; the application should not present its code in notebook form. Note that this is distinct from the case where the purpose of the notebook is to demonstrate code-development techniques; code developed in this fashion usually is less complete in some fashion than full application code, and can be presented in an interactive fashion. (For example, one of the notebooks in the EE Pack develops code for selecting a standard resistor value; a fleshed-out version with better error checking, usages messages, contexts, etc., is contained separately in a package provided with the EE Pack.)

Notebooks can act as live books, containing text explaining the application or employing it for problem solving. The text can be mixed with the actual examples of the use of application functions, presented so that the user can modify parameters or other aspects of the function calls. This is especially useful when the application is courseware; in many cases, the application will consist only of notebooks, with no auxiliary packages.

A user reads a notebook for a particular purpose: to learn a concept, to solve a particular problem, or to learn how to use an application. As such, information that is extraneous to the primary objective should not be placed in the notebook. For some advanced users, implementation details become important; these users can always delve into packages, where detailed information on implementation should be placed. Again, simplify the interface for the user, and the user will thank you.

How Should Documents be Structured?

The hierarchical structure of the notebook allows the use of another aspect of information hiding. Essentially, you can delay presentation of various concepts until the user is ready by placing information in closed cell groups. The tree structure of cells allows you to summarize the contents of parts of the notebook in cell headings. The user can choose the heading of interest, where the topic will be expanded upon.

You can model the presentation of information in a notebook the same way you would a book, with chapters, sections, subsections, and so forth. To reduce the amount of information presented to the user at once (and, incidentally, to reduce memory consumption so users with smaller machines can handle the documents), it is often a good idea to place separate "chapters" in separate notebooks.

The information-hiding technique is particularly useful in courseware that contains problem sets. The problem can be presented in a cell visible to the student, with the answer in a closed cell that the student opens after solving the problem. (This technique has been used in many of the popular instances of calculus courseware.)

The live input in a notebook must be carefully handled. For instance, do not use direct references to items in the command history (e.g., %52, Out[5]). In general, the user's session will not match your session. Relative references are acceptable in a close-knit group of computations (e.g., %, %%). Generally it is not good to refer to anything more than three prior computations back, and ideally all the code referred to in this fashion should be on-screen at once. For more distant references, assign output to descriptive symbols, and reference those symbols in the later calculations. If it is likely that the user may not be able to easily locate the prior reference, document it in the text or reenter the definition.

The typical application notebook contains a brief summary of the purpose and/or the contents of the notebook after the title. This portion of the document is usually grouped so that it is visible when the title is visible. Some initialization code may be placed after the introduction. If it is simple (e.g., loading a package) it may be visible to the user. If it is more complex, and less relevant to the purpose of the notebook (e.g., setting font styles for graphics, setting paths), it may be in a closed group or cell. Initialization material of this sort should go in initialization cells. Finally, the text of the document is given, divided into appropriate sections.

These principles are essentially "rules of thumb" for clear notebooks; they will not apply to every notebook. Further, as more capabilities are added to future versions of the notebook front end, the straightforward tree structure may become less applicable. Keep in mind that the goal is to make information accessible to the user.

How Do You Create Portable Notebooks?

The notebook format is essentially portable across the various platforms that have notebook front ends. However, some aspects of the portability are limited. Generally, the text, code, and graphics created in Mathematica on one platform will be readable and useable on another. However, the style of the notebook may not be identical, and imported graphics may not transport. Some of the points to keep in mind if a notebook is intended to be ported while maintaining a substantially similar appearance are described in this section.

There are two major types of Mathematica notebook front ends: those based on the Macintosh version of the front  end (which currently includes the Power Macintosh, NeXT and X Windows front ends), and those based on the Microsoft Windows version of the front end (including all versions for Windows NT). Both of these systems are continually growing and adding new features, so the information given here is subject to change. This information was current for Version 2.2. For convenience, front ends based on the Macintosh will be referred to as "Mac-style" front ends, while those based on MS Windows will be referred to as "Win-style" front ends.

Default Notebooks

One of the most important tools in porting notebooks is the use of the Defaults notebook. If a notebook that is moved to another platform arrives in a disordered state, you can usually clear up many of the problems by applying the "All Default Styles" menu item (usually under the Style menu). It may be necessary to also apply the "Uniform Style" menu item. In Version 2.2, these were only available on Mac-style front ends. Because of their convenience, they are being added to the Version 2.2.2 Win-style front end.

You can change the Defaults notebook for the front end. You may need to restart the front end for the new defaults to take effect. On Mac-style front ends, the default is specified by the "File locations" preferences menu, while in Win-style front ends it is specified by the "Style..." item under the "Options" menu.

Fonts

Fonts are the biggest problem in cross-platform notebook compatibility. The first problem of this type comes from fonts that do not exist on the target machine. For maximum compatibility, use only fonts that are virtually guaranteed to exist on all machines; Times and Courier are good choices. Often, we also want a sans-serif font; Helvetica is the font of choice. Note that Helvetica is not guaranteed to exist on MS Windows platforms; you can generally substitute Arial, which is very similar to Helvetica, but is part of the standard Windows distribution. Symbol is also an acceptable font. Conversion between the standard fonts can be accomplished by use of a proper Defaults notebook, or the notebook file itself can be directly modified by an automated procedure to convert instances of one font name to another.

The front ends have various mechanisms to substitute a known font for an unknown font. The Mac-style front ends will first try substituting the default font for that cell type. If that font is unknown, then a generic default font for the platform in question will be used. On the NeXT, this is "Times", on the Macintosh, "Chicago", and under X, "Courier". The Win-style front ends currently use only the default type "Courier" when encountering an unknown font.

This substitution is particularly noticeable when porting a notebook created on a NeXT. The current NeXT front end occasionally causes certain kinds of cells that use the 12-point "Times" font to be saved in the notebook file without any font specification. (This goes unnoticed on the NeXT since it is identical to the default substitution for an unknown font.) When a notebook with this condition is passed to another system, cells with this specification will take on that system's default font. The problem is often identifiable by opening a notebook where adjacent cells that apparently should have the same style appear with different fonts. This happens when one cell contains multiple fonts; the multi-font style causes a correct style specification to be given, while in the single-font cell, the front end falls back on the default, which is missing. This is correctable in the same fashion as any other font substitution problem.

A major portability problem arises from use of extended character sets. Most systems support extended character sets for a given font, allowing, for instance, characters with umlauts, or a subset of Greek characters. (Technically, these are characters above ASCII 127.) In general, these character sets are not identical across platforms. The characters will be incorrect when the notebook is opened on another platform. This is especially a problem with Macintosh and NeXT notebooks, since users of these platforms are used to drawing upon extended character sets. Unfortunately, there is currently no solution for this problem. To work aound it you should spell out Greek characters in a Mathematica-like notation (e.g., Pi). You should use, for instance, oe instead of umlaut-o. We realize this is not convenient, particularly for international authors; we are investigating various alternatives for correcting this difficulty. The only other workaround is to translate characters by hand to the proper codes for the system in question.

Akin to the problem of extended character sets is the use of alternate character sets. The Japanese character sets (e.g., JIS) are one example. Current techniques for the use of alternate character sets are not consistent across platforms yet, so a notebook depending on a particular character set is not yet portable. We hope to eventually correct this problem by increased support of Unicode characters.

Multiple-font cells are not currently portable to the Win-style front ends. While a notebook with multiple fonts in a given cell can be read by the Windows front end, the front end will use whatever font has been assigned to the entire cell for all characters. Use of styles to indicate other fonts for particular words in a cell is ignored. (Thus, italicization of Mathematica does not occur.) This will eventually be corrected in a future version of Mathematica for Windows. Multiple-font cells generally move well between Mac-style front ends.

Note that in the current version of Mac-style front ends, tabs are not correctly handled in multi-font cells. In the MS Windows front end, tabs cannot be reset, they are automatically equivalent to 8 spaces. In Mac-style front ends, tab size can be set, but this applies to the front end as a whole, not the notebook. This can cause alignment problems when notebooks are moved between platforms. In general, you should use only multiple spaces, not tabs, to align text at the current time. (This works best, of course, when monospaced fonts such as "Courier" are involved.)

For maximum font portability, then, use only a few standard fonts, the basic character set, and no multi-font cells. If a multi-font cell must be used, understand that it will not be identical on-screen in current Win-style front ends. If the notebook was created on a NeXT, you may still need to transform the notebook to take care of style problems.

There is a file-transfer problem that may manifest itself as a font problem. A file that has been sent by NeXT mail as a Unix text file is often line-broken at 80 characters. Unfortunately, this creates problems with the notebook specification, which requires all header lines (often longer than 80 characters) to be on a single line. This can cause irregular line-breaking throughout the file, and italicization and other font changes to appear in shifted locations. Transfer the file by other means if this occurs.

Graphics

Most graphics created as Mathematica output should be completely portable. The only known exceptions are top-level Mathematica commands that generate raw PostScript (e.g., the Arrow primitive from the Graphics`Arrow` package). All current versions of the front end should support this primitive. Some older versions of the front end will generate PostScript errors when the Arrow primitive has been used, however. Nothing can easily be done about this, though the command using the Arrow primitive can often be modified to use standard Mathematica graphics primitives. This is only a concern if compatiblity with older versions (Version 2.1 and earlier) of the front end needs to be maintained. Otherwise, all Mathematica-generated graphics should render correctly.

Some authors may consider modifications to color to be a problem. Often when graphics are moved between platforms (or even between different computers with the same architecture) colors with an identical specification in Mathematica appear different on the screen. Generally, this is unavoidable. Good color-correction routines for a particular system may minimize this distortion, but some differences will usually occur.

Graphics generated outside of Mathematica and pasted into a notebook are the greatest problem. First note that most externally generated graphics are kept only in the binary portion of a notebook, which is not portable. This means that such graphics are lost when the notebook is moved. If you are using a Mac-style front end, you have the capability to convert imported graphics to PostScript. This has in the past had some problems. When it works, it is generally portable, but certain types of curves did not convert correctly (e.g., those in Macintosh PICT images). Generally this can be overcome by first converting the image to a bitmap, then converting it to PostScript. The main problem with this is that you lose resolution-independence. Since most imported graphics are bitmaps, however, this difficulty is alleviated. Another problem occurs in moving converted bitmap graphics to the Win-style front ends (Version 2.2.1 and earlier). The Windows front end only understands graphics with eight bit-planes, while the Macintosh front end will automatically use the fewest number of bit-planes possible (converting, for instance, a black-and-white image to one bit-plane). This will be corrected in a future version of the Win-style front ends.

An alternate technique that may work in some cases (currently the only option if the notebook is being created in MS Windows) is to convert the externally created graphic to InputForm, and then execute the InputForm to create a Mathematica-generated graphic. This is then put in place of the externally generated graphic.

In general, it is best to avoid the use of externally generated graphics in a notebook, unless you are on a platform that will allow you to successfully convert to PostScript. If you are, such conversion should be performed immediately on import of the graphic.

Certain kinds of machine-specific graphics will probably not convert well at all. QuickTime animations from a Macintosh are a good example.

You may occasionally encounter problems with fonts in graphics. Most problems are identical to those explained in the preceding section. However, fonts in graphics usually require more precise placement. Because of the irregularity of consumption of screen space for fonts in Mathematica graphics, you may occasionally need to shift to a smaller point size when porting a graph for optimal appearance. Some versions of the NeXT front end do not allow font names with spaces to be used in graphics (e.g., MS Windows "Times New Roman"). Substitute a font without a space in the name. The Macintosh front end will occasionally consume excess memory when certain fonts (e.g., "Courier-Oblique") are used in graphics. This memory is consumed when the PostScript is rendered, and is not freed when the notebook is closed. Once the PostScript has been rendered, and the notebook is reopened, the excess memory consumption does not reoccur, so you can save the notebook while porting, and occasionally close and reopen the document to reduce memory consumption. These problems will be corrected in a forthcoming version of the front end.

There is a difference in graphics positioning between Mac-style front ends and the Win-style front ends. In the Windows front end, graphics are positioned with respect to the left page margin, while in the Macintosh front end, they are positioned with respect to the left cell margin. (Note that the Windows front end does not currently support cell margins.) This causes graphics to be incorrectly positioned on the screen in a ported notebook. You may need to reposition graphics by hand in some cases for ideal appearance. It is possible to create automated routines to convert notebook files that will work for most cases. This involves tranforming notebook cell header lines.

Other Issues

The MS Windows version does not currently support cell-based margins, and the document margins are sometimes unreliable. Furthermore, they are stored in the binary portion of the notebook. All of this means that a notebook created in a Win-style front end will adopt the default margins of the target platform, which are usually suitable, but are rarely, if ever, identical. Conversely, margins specified in a Mac-style front end will not be used in a notebook ported to MS Windows. There is currently no solution to this difference of front end capabilities.

None of the front ends store page headers in the notebook text portion, so this is not portable. Also, the Win-style front ends do not currently allow you to use tabs in the running head to specify left, center, and right alignment. If notebooks are intended to be printed by the user, be aware that some characteristics will not be identical across platforms (or even across Mathematica installations).

All front ends are supposed to correctly handle notebooks using carriage-return(cr)/linefeed(lf) [MS-DOS style], cr [Macintosh style], and lf [Unix style] end-of-line indicators. However, problems have been encountered porting notebooks to the Macintosh and NeXT that can only be explained by incorrect handling of cr/lf, so currently you should run any notebook transported from a Win-style front end to other platforms through a file filter to convert to the correct end-of-line marker. Transferring a file by FTP using ASCII mode is generally a sufficient correction; Apple File Exchange can also be used. The current Win-style front end is generally good at handling end-of-line markers from other platforms.

ASCII Codes		Control Character
	Macintosh		13			^M
	Unix			10			^J
	MS-DOS		13 10			^M^J

Cell grouping in a notebook created in a Win-style front end sometimes does not port correctly to a Mac-style front end. This can usually be corrected by making the notebook use automatic grouping. If you do not like the way automatic grouping works, and group your cells in a nonstandard fashion, you may encounter problems that will need to be corrected by grouping and ungrouping cells by hand.

Certain features such as closed cells (not closed groups!) are not supported in the Win-style front ends. This can be a problem for cell-hiding. Also, the brackets for closed individual cells sometimes do not appear correctly in the NeXT front end. You should not use this feature in portable notebooks.

Porting to Print

For some applications, it is useful to create an actual hard-copy document from the notebooks. Several issues peculiar to printing arise in this context.

For the EE Pack, we created two versions of the notebooks; one for on-screen use, and one for printed use. These changes are really only useful for production printing, not casual printing. Different margin settings are often appropriate for good graphic design on paper versus on screen use.

We used a special default style sheet for printing that uses fonts and alignments suggested by professional graphic designers in the creation of the EE Pack. Note that the printer font "Courier" renders the backquote character (`) incorrectly on many printers.We use an in-house font called "WRICourier" to correct this.

Page breaking is often inappropriate, so you may need to examine the locations of page breaks, and reset them. Currently, this can only be done on a Mac-style front end.

Line graphics that use color should be carefully checked; the grayscales that the colors are translated to while printing need to be readily visible on the printed output.

It is generally not a good idea to print out the cells of an animation sequentially. At the same time, the first frame of the animation is generally not informative. You can place the frames in a GraphicsArray for improved appearance. (Unix versions of Mathematica come with a package "FlipBookAnimation`" that can automate this procedure.)

Conclusions

Production of notebooks for inclusion in a Mathematica-based application involves careful choices in the design of the notebooks. Certain principles such as information hiding and portability may improve the reception of your notebooks by users.