Package Design

Todd Gayley
Vertical Applications

Good design is the most important criterion for creating packages for distribution. It is easy to become immersed in the specifics of implementing the functions; however, it is the design that manifests itself to the user, not the implementation details. Even the cleverest, fastest code can be made useless if it is clumsily packaged. The time and thought that you put into design considerations will be well rewarded in increased usability, easier maintenance, and a shortened development cycle.

It is sometimes stated that the complete design specification for a programming project should be created before work on the actual code begins. This is an extreme view, especially given the ease of incremental, "build-as-you-go" development in the Mathematica environment. Even if you like to jump right into coding, make sure that you spend time considering the top-down aspects of your design.

You are creating an interface to a suite of functionality, so keep in mind the primary goal of interface design: usability. The usability of your package will be enhanced by adhering to two simple principles: make it "look and feel" like Mathematica, and make it mesh well with the rest of the system.

The built-in kernel functions and the standard packages have established many precedents for how Mathematica functions are expected to be named and to behave. Resist the temptation to come up with another style. No matter how sensible and logical your scheme seems to you, chances are it will look unfamiliar and feel uncomfortable to users. Don't envision that you are creating a complete stand-alone application; keep in mind that others will be using your package(s) in ways you cannot foresee. Design the functions so that they can be used as part of a "pipeline". Make sure that the input to your functions is easy to obtain from other Mathematica functions, and likewise that the output is in a form that is immediately useful for further processing. Another aspect of meshing with the system is ensuring that the state of the user's session will not be unduly modified by loading your package.

The guidelines presented here will help ensure that your packages comply with these two principles. These are not unbendable rules, but they are well-tested guidelines that match users' expectations. Two important references on package design are  Roman Maeder's Programming in Mathematica, Second Edition (Addison-Wesley 1991) and Stephen Wolfram's "Some Elements of Mathematica Design" (The Mathematica Journal, Volume 2, Number 2).

Naming

Use the conventions established by the built-in symbols for all your exported symbol names. Names should be full English words or sequences of words, with the first letter of each word capitalized. When in doubt, use the longer or more explicit term; avoid abbreviations and names that are not descriptive. If a function with similar functionality exists in Mathematica, use a similar name. Think carefully before using abbreviations or acronyms in function names, even when they are commonly used in your specialty. Keep the number of function names to a minimum, and instead use options and differing argument sequences to provide different behaviors. To ensure portability across all platforms, keep your package names unique within the first eight letters.

Modularity

The structure of a package provides a means to control which symbols will be exported (i.e., visible to users after your package has loaded). Consult Maeder's book as the canonical reference on the structure of packages and the context issues that are involved. Here is the basic structure:

BeginPackage["MyPackage`"]

     Introduce all the exported symbols (including functions, variables,
     option names and their values, etc.) here by supplying usage
     messages. Any symbols that appear before Begin["`Private`"] will be
     visible to users, so make sure that only symbols you want exported
     appear. The method of supplying a usage message is not required--it is
     merely a convenient means of referring to the symbols, thus creating
     them in the package context, and at the same time ensuring that each one has
     a usage message attached.

FirstFunction::usage = "FirstFunction does nothing."

SecondFunction::usage = "SecondFunction does nothing."

Ordering::usage = "Ordering is an option to FirstFunction.
                        It specifies..."

     If your package will Protect symbols it defines at the end, make sure it
     unprotects them at the beginning. This will avoid a long series of "this
     symbol is protected" messages if users inadvertently read your package in
     twice during a session:

Unprotect[FirstFunction, SecondFunction, Ordering]

     By defining a subcontext of the package context, you hide all the symbols
     you introduce in the implementation of your functions. By convention,
     use "`Private`" as the name of this subcontext:

Begin["`Private`"]

    All the code for implementing your package goes here.

End[]

    If you choose, you can Protect the symbols you introduced:

Protect[FirstFunction, SecondFunction, Ordering]

EndPackage[];

It is also good to include a set of special comments at the beginning of the file. In addition to supplying important information to the reader, they can be extracted by special utilities (e.g., the function Annotation in Utilities`Package`). A good example of these comments is the package Utilities`Package`. Most of the comment types are self-explanatory. The :PackageVersion: comment gives the revision number of the package itself (a number that is meaningful to the author), and the :Mathematica Version: comment gives the minimum version number of Mathematica required to run it.

(* :Title: Package Utilities *)

(* :Context: Utilities`Package` *)

(* :Author: John M. Novak *)

(* :Summary: 
	This package provides various utilities for
	searching and working with Mathematica packages.
*)

(* :Package Version: 1.0 *)

(* :Mathematica Version: 2.1 *)

(* :Copyright: Copyright 1992-1993, Wolfram Research, Inc.  *)

(* :History:
	V. 1.0 June 92, by John M. Novak.
*)

(* :Keywords:
	packages, path, keywords
*)

(* :Limitations:  *)

(* :Discussion:  *)

Programming Style

A few quick comments on programming style are appropriate here. (Longer discussions of this issue can be found elsewhere.) Use transformation rules and functional programming where possible; procedural programming is less efficient, longer, and clumsier in Mathematica. You have achieved this goal to the extent that functions like Map, Apply, Replace, Table, Fold, and Function appear in your code, and ones like For, Do, While, and Append do not.

Make sparing use of the Print function. You will notice that it is virtually never used by the built-in functions or the standard packages. If you have information to present to the user that is a warning or an error, use the Message function. Chances are that anything else truly meaningful should be returned from your function not printed out so it can be used in further processing. The liberal use of Print statements is a habit forged by use of more traditional languages like C and FORTRAN, which are not interactive and are locked into a small set of rigid data types. The same comments apply to the Input function. If you find yourself coding many Input statements to get input from the user and Print statements to display results, this is a good indication that you have not made the shift to a Mathematica mindset, and others will find using your package unfamiliar and inconvenient. In cases where you want to provide diagnostic output (for example, to alert users to the progress of a calculation), make printed output an option. I suggest the name Verbose->(True or False), with False being the default.

Argument Handling

If there are built-in functions that take similar arguments to your functions, use the same argument order. For example, in virtually all functions that take a filename or stream as an argument, it is the first argument.

Proper use of patterns on the left-hand side of function definitions can greatly simplify the code required on the right-hand side. For example, you could write a single rule with a generic pattern that requires you to do a lot of argument testing on the right-hand side:

f[args___] := Module[{var},
                     If[Head[{args}[[1]]] =!= List,
                         do something
                     ];
                     If[Length[{args}] == 1,
                          do something,
                     (* else *)
                          do something else
                     ];
                     etc...
              ]

Instead, define multiple rules that match specific cases:

f[x_List, y_] := something...

f[x_] := something...

etc...

You might also want to write a "fall-through" pattern that catches any cases not matched by the other definitions. This definition can trigger an appropriate error message and then return unevaluated. This technique is demonstrated in the "Error Handling" section that follows.

Learning to write nontrivial patterns is time well spent, as it will save you a lot of work validating arguments in the body of your functions; consult Section 2.3 of The Mathematica Book for more information.

There are two ways of supplying nonrequired arguments to a function. The first, and most common, is to use Options. The second is to use the Optional pattern type. There are no hard and fast rules on when to use an Optional pattern, but it should be used sparingly, and only one Optional pattern should appear in any function definition (you will understand why if you try to write a definition with multiple Optional objects). Examples of built-in functions that use optional arguments are Log (where the optional argument is the base) and Map (where it is a level specification). The order in which the various types should appear in a definition is always required arguments first, followed by Optional ones, followed by Options. For example:

f[x_, level_Integer:1, opts___?OptionQ] := body...

Inside the body of the above function you can use the parameter level and be sure that it has a value--it will be 1 if the user left it out (as in f[a]). The opts___?OptionQ pattern is a standard idiom in Mathematica. It will match a (possibly empty) sequence of rules or a list of rules. The function OptionQ is not documented in The Mathematica Book, but it is an established part of the kernel since Version 2.0. It is a very clean and self-documenting way of matching options.

There is also a standard way of extracting the values of options. This is discussed more completely in Maeder's book, but here is a brief example. Let's say the function f takes one option, Method, which has possible values Quick and Dirty. You might write a function like:

Options[f] = {Method->Quick}

f[a_, opts___?OptionQ] :=
          Module[{meth},
                 meth = Method /. {opts} /. Options[f];
                 etc...
          ]

Using this technique, meth will be whatever value the user specified for the Method option, or the default value Quick if the option was not specified.

Error Handling

User errors should be detected whenever possible, and meaningful messages should be generated. This is often overlooked in Mathematica programming, probably because it is not seen as a high priority and because it requires careful, and sometimes clever, programming for which there are few well-known idioms.

When bad input is given to a function, or an error is detected during execution, you generally have three choices: try to recover from the error condition and continue with the calculation, bail out of the calculation and return the symbol $Failed, or bail out and return the input "unevaluated". If the error is not fatal, so that continuation is possible, it is generally a good idea to issue a Message indicating what has happened (for example, an invalid option value was given, and the default is being used instead). If the evaluation cannot be performed, it is simplest to just return $Failed. Any time you return $Failed, a message should also be issued.

Most built-in functions return "unevaluated" when given inappropriate input. "Unevaluated" is in quotes because, of course, the functions have been evaluated--they are just unchanged by evaluation. Imitating this behavior in your own functions is a bit more complicated than returning $Failed. For many types of inappropriate input, your functions will behave this way without any effort on your part. For example, if you only define a rule for f when given a list as an argument and a user executes f[1] it will return unevaluated. Ideally, though, your function should issue an error message before returning. Here's how you could do that:

f::list1 = "The first argument to f must be a list.";

f[a_List] := something

f[_] := holycow /; Message[f::list1]

f[1]

The last definition for f is a "catch-all" case that traps any calls to f where the first argument is not a list. The statement holycow is never executed because the Message function, which is the test in the Condition, doesn't return True. In other words, you write a phony rule for f that will never match any input (because the Condition always fails), but which triggers a side effect of displaying a Message.

In the above example, we knew from pattern-matching that the input was invalid. If a more complex test of the input is required, you can put the Condition as the last statement in a Module:

f[x_List] := Module[{result, test},
                    some calculations here...
                    test = true or false depending on calculations
                    result /; test
             ]

If test is set to False, the Condition fails and Mathematica pretends that the match to f[x_List] never happened, moving on to try other rules for f. This is a little-known tidbit of Mathematica programming. Note that Condition must be the last statement in the Module (or Block or CompoundExpression) for it to work.

A good practice is to use preexisting Mathematica messages. Many messages are used by more than one built-in function, and they are generally defined as though they belonged to the symbol General. When Mathematica looks for a message with name func::tag, it first looks to see if there is a definition for this specific message. If not, it looks for a message of the form General::tag, and if such a message exists it uses its text for the func::tag message. In this way many symbols can share a set of messages. In the example above I used f::list1 to indicate that a list is needed in argument 1. In fact, a message that expresses this is already available, General::list:

General::list

ComposeList[a,b]

Note, though, that the placeholder `1` needs to be the entire f[...] expression. Here's how you can get at this value:

Clear[f]

f[a_List] := something

p:Literal[f[a_]] := holycow /; Message[f::list, HoldForm[p], 1]

Now f behaves just like a built-in function:

f[1]

Tread Lightly on the System

The state of a user's session should not be unnecessarily changed by loading your package or evaluating one of its functions. For example, don't Unprotect and modify system symbols unless absolutely necessary, and never change their Attributes. Similarly, avoid changing global variables (i.e., System` globals, whose names usually begin with a $). Your package should never put symbols in the Global` context or rely on the presence of certain symbols in that context.

If you must Unprotect symbols other than your own, make sure that you restore them to their previous protection state. You can do this by using the idiom introduced by Maeder:

protected = Unprotect[Symbol1, Symbol2]

modify Symbol1 and Symbol2....

Protect[Evaluate[protected]]

If you turn off messages temporarily, make certain that you restore them to their original state, whether that was on or off. This is a particularly common error (you can see several examples in the standard packages). Here is a code fragment that shows how to properly turn messages off and back on:

wasOn = Head[f::msg] =!= $Off;
Off[f::msg];

code that would trigger the f::msg message....

If[wasOn, On[f::msg]];

Testing

Testing is important for any package, no matter how small. Developing a standard test suite for each package will not only help you find most of the bugs (you never find all of them), it will also help you maintain your code and serve as an invaluable final check on the design and implementation. Wolfram Research has a package called Testing.m (available on MathSource) that will help you create and manage a test suite. Ideally, the tests should exercise every branch of every function, including error conditions. Pay special attention to checking that your functions do something sensible with any type of input, no matter how improper.