V3.0: Manifest Destiny

Or, what have we been up to for the last decade or so?

by Theodore W. Gray

The Mathematica front end has undergone many changes over its 9 year life.  In this article we will see how some of the features present in V3.0 developed and changed over time.

The original vision of the front end was to provide a document interface to the Mathematica kernel.  I decided very early that we would create an environment in which you could not only interact with the kernel, but at the same time create a permanent, editable record of what you did.  This would overcome the biggest drawback to using a UNIX terminal interface, which was that when you were done, you had nothing you could easily refer to in the future, or pass on to colleagues.

Since at the time the kernel used only plain ASCII text for input and output, the interface would necessarily concentrate on text.  I ruled out a 2-D drawing type interface (like MathCad) on the grounds that extensive editing of large documents is nearly impossible: Inserting new material in the middle of such a document is impractical.

So, we arrived an an interface that was basically similar to a word processor: A vertically scrolling document in which you could enter text at will.

Cells

One of the very first decisions I made was that the front end would organize text into clearly visible units, cells, rather than provide a seamless, contiguous stream of text.  The strongest motivation for this was the Macintosh MPW programming environment, which does not have a concept like cells.

The advantages of cells are:

• Multi-line input can be evaluated without first manually selecting the range to be evaluated.  Without cells, it would be necessary to manually indicate the range of input to be evaluated each time, as is necessary in MPW.

• When input is evaluated repeatedly, old output from previous evaluations can be deleted automatically.  Without cells, there would be no clear indication of what is output v.s. input, and therefore no way to know what to delete.

Initially, we had only a linear sequence of cells, and instead of cell brackets we had vertical gray bars that indicated the range of text in a cell.  It was due to a suggestion from Steve Jobs that we started using brackets: They graphically reinforce the notion that text within a cell is connected.

It was clear early on that some form of outline capability would be valuable.  Because output from Mathematica can be very long, documents quickly become very, very long and hard to navigate.  The first idea was to collapse ranges of cells, leaving only the first and last cell visible, with a striped area indicating the missing cells.  This quickly evolved into the notion of nested groups of cells, which was implemented very shortly before the release of V1.0.

At first, there were only two types of cells: text and PostScript.  Over time, additional graphics formats were added (PICT, QuickTime, NeXT Bitmap).  In V3.0, we added typeset cells, implemented by Neil Soiffer, which contain editable 2-D mathematical expressions.  This is the most significant addition to Mathematica in many years, because it allows interactive editing of input and output in full typeset notation.  We added OLE cells to support that protocol in Microsoft Windows.

In V3.0 we also introduced inline-cells, which means that cells (typeset or graphics) can be inserted inside of text cells.  This concept is, of course, commonly available in word processors as inline graphics.  It allows formulas like \(TraditionalForm
      \`\(\(-b\) ± \@\(b\^2 - 4 a\ c\)\)\/\(2 a\)\) or graphics like  to be placed inline with text.

Formatting and Styles

The first version of Mathematica allowed only a single font per cell, and a very limited number of formatting options.  Although primitive in this regard, the front end did provide a style sheet mechanism which allowed formatting changes to be made globally.

Over time, more and more formatting options were added: independently adjustable indentation margins, multiple fonts per cell, etc.  Up until V3.0, only a fixed set of style names could be used.  All settings defining the styles (the style sheet) were stored privately in each Notebook: It was not possible for many notebooks to share the same style definitions.

In V3.0, a much more flexible style mechanism has been introduced.  Arbitrary style names can be defined, and style sheets are independent entities that can be stored as separate files and referred to by multiple notebooks.

Perhaps more importantly, in V3.0 all the settings that control the display of objects in the front end have been recast as Mathematica-style options.  For example, to control the appearance of text, there are options such as:

FontFamily -> "Times",
FontSize -> 12,
FontColor->RGBColor[0.5, 0, 1]

You can still use menus to control these things, but because they are defined by underlying option names, you can also use Mathematica programs to control them.  In most other applications, the specification of these sorts of settings is imprecise at best, because there is no clear language design to base the specification on.

All objects in the front end (notebooks, cells, text, graphics, mathematical expressions, etc) are now represented by Mathematica expressions.  For example, a simple cell in Text style is represented by:

Cell["Hello this is a text cell.",  "Text"]

A cell containing multiple fonts is represented by a more complex structure.  For example, this cell would be represented as:

\(Cell[
      TextData[{
\ 
          "A cell containing multiple fonts is represented by a more complex structure.  For example, ", 
\ 
          StyleBox["this", 
\ \ \ \ FontSlant -> "Italic"], 
\ 
          " cell would be represented as:"
}], \ "Text"]\)

File Format

File formats can be divided in two ways: Is the format sequential or pointer based, and is the format binary or text.

In a sequential format, the data (text, graphics, etc) exists in the disk file in the same linear order it is displayed.  In a pointer based file, the data in the file may be in any arbitrary physical order, and a table is maintained that tells how to reassemble the scattered pieces into the proper order.

The biggest difference is in how new material inserted in the middle of a document is handled: In a sequential format, the data is inserted in the analogous position in the file and the data that follows it is shifted down.  In a pointer based format, it is added to the end of the file, eliminating the need to shift potentially large amounts of data on disk.

The problem with pointer based formats is that they are fragile, and completely unintelligable outside of the application that created them.  Because the data is not in order, if you look at the file with a raw data editor you may see nothing more than fragments of text scattered at random.  If the file is damaged, especially if the table portion is effected, the file is likely to be completely destroyed.  If you don't have the application that originally created the file, you have very little chance of being able to make much sense of the file.

The problem with linear formats is that it can be inefficient to shift large amounts of data on disk every time the file is saved.  The file is also not random access, so in order to display data near the end of the file, all the data up to that point must be read in.

In the binary v.s. text format axis, there are three main choices: Binary, 8-bit ASCII-based, and 7-bit ASCII.  (There are many 8-bit encodings such as the ISO Latin family, and several dozen Macintosh script system encodings, all of which use the ASCII standard to define values up to 127, with various assignments for higher values.)

A binary format allows all 256 possible values of each byte in the file.  In some applications, textual data is stored literally with only graphics and pointer data in binary format: If the file is examined in a raw text editor, the text will be visible.  In other applications, the text is compressed (to save disk space), and absolutely nothing is recognizable when the file is viewed in a raw text editor.

In an 8-bit ASCII-based format, the file respects the control characters defined by the ASCII standard (i.e., the file does not contain any byte values less than 32, except when they are being used for their official purpose).  A file in this format can be examined with a raw text editor, and is likely to make at least some sense.  If the text in the file contains accented and other special characters, they are stored as byte values between 128 and 255: Typically the values used depend on the font that is specified for the text in question.

In a 7-bit ASCII format, only byte values between 32 and 127 are used to represent data in the file.  This format is the lowest common denominator understood by essentially all current computers (with the exception of some antique IBM mainframes).  Accented and special characters are stored by translating into multi-byte sequences: A typical solution is to use octal or hexadecimal encoding for these characters.

From the very beginning, we decided that the file format for Notebooks would be linear ASCII text, but that we would not accept all the limitations this might seem to imply.  Because we expected Mathematica to be used by people with a wide variety of different types of computers, we wanted a format that could be transferred very easily.  Because we expected a lot of academic users, we wanted a format that could be sent by email, and at that time email was a text-only format.  (Even today, text is the only format that can be used compatibly between all the available email systems, even those that support one or another binary attachement format.)

Personally, I also felt a certain sense of historical responsibility.  In 500 years, a FrameMaker or Microsoft Word binary document will most likely be completely useless.  The applications will long since have vanished, along with any hardware that could run them, and most likely even the basic concepts of "computer" and "application software" will be unrecognizable.  However, it is fairly safe to assume that the English language will still exist, and I would give even odds that ASCII will still be used at some level to encode it.  Unlike specific binary file formats, ASCII has an air of permanence to it.

In versions before V3.0, we used an 8-bit ASCII format, while in V3.0 we have switched to a pure 7-bit ASCII format.  There are, however, other much larger changes in the file format in V3.0.

Starting with the first pre-release version of Mathematica, the Notebook file format was a very simple, perhaps simplistic, text format.  Each cell was delimited by a line starting with ":[", and contained a series of badly named tokens representing option settings.  Suffice to say that the format did not have room to grow.

In V3.0, we switched to using Mathematica syntax to represent notebooks.  A notebook is simply a list of Cell[] objects (described above).  A notebook containing two cells might look like this:

\(Notebook[{
		Cell["This is the first cell.", "Text"], 
		
        Cell["This is the second cell.", "Text"]
}]\)

Those who have access to V3.0 may want to take a look at an actual notebook document at this point.  Use any text editor other than Mathematica to open a file containing a few cells (BBEdit, Notepad, vi, emacs, Microsoft Word, etc, will work fine).  You will see about a screen full of general comments intended for people who happen upon the file in email, followed by the main Notebook[] expression.

The Mathematica expression format allows for a great deal of flexibility, within a well defined language environment.  It also means notebook documents can be loaded into the kernel, processed, and written back out again, using ordinary Mathematica commands like Put and Get.

The biggest problem with this format is common to any linear, sequential text format: efficiency.  Once you have a notebook document larger than a few pages, it is unacceptably slow to read in the entire contents of the file each time it is opened.  This is especially true if the document contains complex mathematical expressions, or large PostScript graphics.  For acceptable performance, it is necessary to load only those portions of the file that are actually needed to display the currently visible range of the document.  If the user jumps to the end of the document, the program needs a way to load the cells at the end without loading all the cells in between.

This is only possible if some additional information is maintained in the file, basically meta-information about the layout of the main linear sequence.  The solution I chose for V3.0 was to write out a comment containing another Mathematica syntax expression that mirrors the main notebook expression, except that each Cell object contains only a few numbers giving the byte offsets within the file where the actual Cell data is to be found.  Here is an example of this comment for the simple notebook example above (containing two Text style cells):

This comment contains all the information necessary to locate the two full Cell expressions in the main Notebook expression.  When the file is opened in the front end, this comment is read in its entirety.  Then, as the user scrolls around in the docuement, this information is used to incrementally load the actual cell data as needed.

The arguments to Cell in this comment are:

Cell[\(byte\ offset, line\ number, byte\ count, line\ count,         height\ estimate, style\ name, options\)]

The first two numbers say where the Cell expression starts: It is necessary to know the line number (i.e., the number of return characters up to this point in the file), because on DOS/Windows systems, lines are terminated by two characters (return/linefeed), instead of a single character as in the rest of the world.  If the file is transferred to or from a DOS system, the byte offsets within the file will change: By knowing the number of return characters up to each offset, this can be adjusted for automatically.

The next two numbers specify the number of characters or lines in the cell: This allows the data representing a given cell to be shifted around in the file using low level file operations.  The height estimate allows the scroll bar range to be set approximately correctly (the range is the sum of the heights of all the cells in the notebook, including those that have never been displayed).  The estimate will be perfect only when the file is reopened on the same system it was saved on, no fonts have been added or removed from the system, and the window is not resized after opening.

The style name and optional options are used to get the values of those few options whose effects are non-local.  For example, options that effect the automatic grouping behavior of cells must be known for all cells, not just those that are currently visible.

Following the outline comment are additional comments that allow the front end to rapidly locate cells that match a given keyword (CellTag).  This is currently used only for the Help Browser, but in the future it should be useful for other searching/indexing features.

Of course, because the file outline comment contains specific character offsets, if the file is edited outside of Mathematica (in a text editor), the offsets will not be correct anymore.  This is not really a problem, because the front end recognizes that the file has been edited, and simply ignores the file outline, reparsing the main Notebook expression.

Cross Platform Compatibility

There are several separate issues that need to be addressed in order to have full, seamless compatibility between different computer systems.  (As of today, the systems in question are Macintosh, Microsoft Windows (Windows 95, OS-2), UNIX (X-Windows), and NeXT.)

• Graphics formats are different.
• The line termination character is different (return, linefeed, or both).
• The available fonts are different.
• The character codes used to represent accented and special characters are different.

In versions before V3.0 we had effectively solved only the first of these problems, by including a simple PostScript interpreter as part of the front end.  This allowed us to use PostScript as a standard that could be rendered on all systems.  This worked, but could be inefficient because PostScript is a very verbose format for representing graphics.

In V3.0, we solve all these problems quite effectively.

In addition to supporting PostScript, V3.0 introduces a platform independent bitmap format, which allows a compact representation of bitmap images, including caches of rendered PostScript images (PostScript can take a very long time to render, so a bitmap image of each finished image is stored in the notebook file).

The line termination problem is solved by the particularly clever way we manage our file format: See the File Format section above.

The font and character encoding problems are related, and difficult.  The first 127 character codes that make up the ASCII standard are universally compatible, but they include only upper and lower case, unaccented roman letters, arabic numerals, and a very small number of punctuation marks.  Beyond that, there is no standard: Each of the 4 major classes of systems uses a different default character encoding for accented roman letters, and there are dozens of widely used encodings for non-roman character sets (Kanji being the most important example).  This very, very complex issue cannot be ignored, especially not for a system so heavily involved in mathematics, where literally hundreds of special characters are used.

Attempting to bring some order to this chaos is the Unicode character encoding standard.  The Unicode consortium has developed a 16-bit character encoding that aims to define a single unique code to represent each and every character in each and every human language that is or ever has been.  Needless to say, they will never succeed, but they are making a good go at it, and their efforts are, if nothing else, an inspiration.

Mathematica uses the Unicode character encoding internally.  Thus, for example, the character π is represented internally as the code 960, regardless of what platform the front end is running on.  This uniformity is vital if things like this are to work:

N[π]

Without a uniform internal encoding, getting several hundred mathematical symbols to evaluate properly on all platforms would be a nightmare.

As mentioned earlier, in V3.0 we are using a pure 7-bit ASCII file format for notebooks.  To represent 16-bit Unicode characters, we could have chosen to write them in octal or hexadecimal form.  However, while we use Unicode internally, we decided that we would not use it to represent most characters in files.

We decided to give unique, english names to a very large number of characters (almost 800).  These names make the file much more readable to someone looking at it in a raw text editor, and should help to ensure the intelligibility of the file well past the lifetime of Mathematica and Unicode.  The character names are similar to Mathematica function names in that they are long, descriptive, and never abbreviated.

Character names are written as \[name], as in \[ Pi] to represent π.  For example, the following cell:

The Weierstrass ℘ function is an interesting function not related to Fahrvergnügen.

is written in the notebook file as:

Cell[
      "The Weierstrass ℘ function is an interesting function \
not related to Fahrvergnügen.",  "Text"]

This example contains two special characters.  The ü character is one that is in most platform's (non)-standard character encodings, while the ℘ character is available only in a very small number of technical fonts, and is not part of any standard encoding other than Unicode.

Although these names may seem like a very wordy way to write out a single character, they do ensure absolute compatibility, because they define uniquely the character you want, rather than a number that must be interpreted in light of an encoding table that you amy not have access to.  To put it another way, the file is self-contained with regards to character representation.

Although we have 800 character names, including all the western accented characters, and all mathematical and scientific symbols we could identify with extensive research, this is still just a small subset of the 60,000-some unicode characters.  The great bulk of characters we do not have names for are the Japanese, Chinese, and Korean pictographs, with a smaller contribution from the colorful variety of the world's lanuages (Arabic, Hebrew, Tamil, etc).  To handle this overwhelming number of characters, we use the Unicode character codes to store the characters in notebook files.  The codes are written in hexadecimal, using the Mathematica syntax \:xxxx to represent each 16-bit value.  This representation coexists easily with the \[name] notation.  If, in the future, it becomes clear that certain characters should be given names, they can be added without any loss of backwards compatibility.

Having solved the problem of representing characters in a highly compatible cross-platform way, there remains the serious font problem.  It does little good to have a way of representing ℘, or ℵ, or ℷ, or ℏ, if you don't have a font to draw them with.  After considerable discussion and research, we decided that there simply did not exist a coherent, universal, family of fonts to represent the characters commonly used in all the fields in which Mathematica is used.

So, we created a family of over 600 mathematical, scientific, and other special characters, in roman, bold, italic, proportional and mono-spaced versions.  The mono-spaced characters are particularly interesting: No one has ever attempted to make such a complete set of mono-spaced special characters (mainly because no one had any reason to).

In the past, output from Mathematica had to be displayed in a monospaced font for technical reasons: This is no longer the case, and we had initially thought that proportionally spaced fonts would be the automatic choice now that the technical limitation had been removed.  However, we found that for good readability on screen, Courier-like, monospaced characters were considerably superior to any proportionally spaced font.  So, the default output format type (StandardForm) uses Courier and the monospaced version of our fonts for everything except certain characters (large brackets, integral signs, certain infix operators, etc).

Because out family of fonts is a standard part of Mathematica V3.0, and available on all platforms, it is possible to transfer notebooks containing very complex mathematical typesetting with complete compatibilty.  There is literally no other system that is able to do this for more than a very restricted set of characters, or a limited number of platforms.

Integration with the Kernel

The front end and kernel have always been completely separate programs, connected only by a single bi-directional data stream.  Initially we used a primitive protocal known internally as Math Talk.  Later this was replaced with MathLink, which is a more sophisticated, general way of transmitting arbitrary Mathematica expressions across data streams.  (The MathLink protocol has been published, and libraries are available for those who want to write their own programs using it.)

This strict separation has many advantages.  It allows a front end running on one computer to connect to one or more kernels running on different computers, even different types of computers.  A typical case would be a front end running on a small desktop machine with a good user interface (e.g. a Macintosh), connected to a kernel running on a fast, physically distant computer with little or no user interface (e.g. a mainframe, or high end workstation server).

Initially, the operation of the front end did not depend in any way on the kernel.  Unless you actually wanted to evaluate a Mathematica expression, the front end had no use for the kernel.  Over time, more and more features in the front end have come to rely on the presence of a kernel.

The first feature that required a kernel was command completion.  It works by sending the partially entered function name to the kernel, and receiving a list of possible completions.  Since this information exists only in the kernel (and may change if the user defines new symbols), this is clearly necessary.

Next, the Function Browser was implemented by using the kernel to get usage messages for function names, and lists of functions defined in currently loaded packages.  In a slight twist, the V3.0 feature, now called the Help Browser, does not use the kernel any more.  It has direct access to the full text of the Mathematica book, which is much more complete than the usage messages available in the kernel.

With V3.0, two standard menu commands, Convert to TeX and Convert to HTML, are implemented entirely in the kernel.  The front end simply tells the kernel what the user wants, and the kernel reads the front-most notebook across the MathLink connection, asks the front end to put up a Save-As dialog to get a file name, then does the conversion.

The palette mechanism also makes use of the kernel to execute button action functions, when required.  Many simple action functions can be executed directly in the front end (paste text, evaluate selection, change attributes, move selection, open/close files, are some of the operations that can be executed directly in the front end).  However, any action functions that required testing conditions or doing computations of any sort must be evaluated in the kernel.

In the future, it is likely that more and more features in the front end will come to rely on a kernel.  The main force limiting this trend is the large amount of memory required to run a kernel.  As of today, it would be unacceptable to require a kernel for the front end to operate at all.  However, the day is not too far off when this will be reasonable.  At this point, the front end will probably incorporate its own personal kernel, which it will use to coordinate many internal operations, allowing for a great deal of flexibility and user programmability.

Cross Platform Code Sharing

As discussed above, the front end and kernel are completely separate programs.  The original motivation for this architecture was not the advantage of allowing them to run on separate computers.  Instead, the idea was that the kernel would be a purely platform independent program, while the front end would encapsulate all the messy platform dependent interactions with particular window systems.

The kernel was to be a pure, text-in-text-out program that would be compiled from almost identical sources on all platforms.  The front end, we thought, would be a relatively small, disposable piece of code that would be written from scratch for each different window system.

The kernel was originally developed on UNIX systems, and it clearly shows this bias.  It does, in fact, run almost identically on all platforms, but only because it turns out that all platforms implement UNIX compatibility libraries.  So, this part of the original plan has worked out pretty much as expected, mainly by luck.

The front end part of the plan turned out to be completely mistaken.  There were two basic errors in our thinking: First, the front end is a much larger and more sophisticated program than we thought.  Second, window systems have a lot more in common than we thought.

I originally wrote the front end as a Macintosh only program, with no thought what so ever to porting it to any other system.  When Steve Jobs was developing his NeXT computer, he learned about Mathematica, and Wolfram Research ended up making a deal to bundle Mathematica with all NeXT systems.

Menus

The earliest, Macintosh-only, version of the front end did not allow any rearrangement of menu commands: The function of each item was determined by its position within the menu.

With the first cross-platform version of the front end (Macintosh-NeXT), it was necessary to introduce a more flexible way of associating items in menus with their corresponding functions.  Each internal function was assigned a unique number that did not change regardless of which menu or position an item was in.  In theory, this allowed users to rearrange menus at will, but because the menu layout was stored using platform specific system formats (MENU resources and nib files), this was not practical for most users.

In V3.0, each internal function has been given a name instead of just a number, and the layout of the menus is defined by a Mathematica expression format text file.  The format of the file allows it to be edited fairly easily by someone familiar with the Mathematica language, and perhaps more importantly allows it to be manipulated by Mathematica programs.

The file containing the menu layout is called MenuSetup.tr,  A separate copy for each platform is in the main Mathematica installation directory, inside SystemFiles/FrontEnd/TextResources/platform, where platform is one of Macintosh, NeXT, X, or Windows.  The file contains a nested set of Menu[] and Item[] expressions: Here is the beginning of the file for the X version:

\(\(Menu["Menu for the X Front End", 
      
{
	Menu["File", \ 

          	{
		Item["New", \ "New", 
              MenuKey["n", \ Modifiers -> {Command}]], 
		
            Item["Open...", \ "Open", 
              MenuKey["o", \ Modifiers -> {Command}]], 
		
            Item["Close", \ "Close", MenuKey["w", \ Modifiers -> {Command}]], 
            
		Item["Save", \ "Save", 
              MenuKey["s", \ Modifiers -> {Command}]], 
		
            Item["Save As...", \ "SaveRename", 
              MenuKey["S", \ Modifiers -> {Command, \ Shift}]], 
		
            Menu["Save As Special...", 

              		{
			
                Item["Pre-3.0 Notebook", SaveRenameSpecial["V20Notebook"]], 

                			Item["Text", \ SaveRenameSpecial["Text"]], 
			
                Item["Cell Expression", SaveRenameSpecial["CellExpression"]], 
                
			Item["Notebook Expression", 
                  SaveRenameSpecial["NotebookExpression"]], 
			
                Item["Package Format", SaveRenameSpecial["Package"]], 
			
                Delimiter, 
			
                Item["TeX", \ 
				
                  FrontEndExecute[ToExpression["FrontEnd`DoTeXSave[]"]], 

                  				MenuEvaluator -> Automatic], 
			
                Item["HTML", \ 
				
                  FrontEndExecute[ToExpression["FrontEnd`DoHTMLSave[]"]], 

                  				MenuEvaluator -> Automatic]
		}], 
		
		
            \(etc.\)\)\)

This defines the File menu, with its "Save As Special..." submenu.  The first argument to Item is the text to be displayed in the menu.  The second argument is the token that uniquely identifies the function carried out by the menu item.  Notice that some of the items specify command key equivalents using the optional third argument.

Two items at the end demonstrate that the second argument can also be a function to be executed in the front end, or evaluated in the kernel.  Menus handle their action functions in the same way ButtonBox objects do, with the same set of available functions (documented in the Mathematica book, and online documentation).

Conclusion

There have been three major trends in the developement of the Mathematica front end over its 9 year life.