Front End Frontiers

Commands and Tokens

by Louis J. D’Andria

The Mathematica front end is a MathLink application, inside and out. With every menu selection and keystroke, the front end performs some MathLink operation. This column will explore the various methods of controlling the front end by sending it MathLink commands, such as writing a kernel program to perform a sequence of actions that would typically be invoked from the menus or from the keyboard.

Commands

The usual method for scripting some front end actions is to use the notebook commands that are built into the kernel. These commands include NotebookApply, NotebookOpen, and NotebookPut, which are documented in the Mathematica book [Wol] and the help browser. Here is a list of the most often used notebook functions

({{ButtonNotebook, NotebookFind, NotebookSave}, {CellPrint, NotebookGet, NotebookWrite}, {EvaluationNotebook, NotebookLocate, Options}, {InputNotebook, NotebookOpen, SelectedNotebook}, {NotebookApply, NotebookPrint, SelectionMove}, {NotebookClose, NotebookPut, SetOptions}, {NotebookCreate, NotebookRead, SetSelectedNotebook}, {NotebookDelete, Notebooks, StylePrint}})

Given a handful of examples [Wol §2.10.3], it is not hard to see how these functions work. There is typically a target notebook — usually one of ButtonNotebook, EvaluationNotebook, InputNotebook, or SelectedNotebook — and the function either performs some action on that notebook or reads information from it. Here is a simple script that copies all of the input cells from one notebook into a new, untitled notebook.

nb=NotebookOpen["myNotebook.nb"];
NotebookFind[nb, "Input", All, CellStyle];
inputCells=NotebookRead[nb];
nb2=NotebookCreate[];
NotebookWrite[nb2,inputCells];

For many applications, this set of commands is sufficient. Indeed, books and articles are starting to appear which exhibit uses of notebook commands ([Abb], [GD], [GG], [Mae]). However, notice that this scripting language does not draw upon existing knowledge of using the front end with a keyboard and mouse. In order to do the above task by hand, one would simply use copy and paste from the Edit menu, or the keyboard shortcuts for those menu commands.

There is another approach to notebook programming that uses familiarity of the menus and keyboard in order to make scripting more natural: tokens.

Tokens

A token is a string which the front end associates with some action. They are sometimes referred to as command tokens to distinguish them from parsing tokens [Wol §2.8.14]. Most menu commands and keyboard commands have a command token associated with them. When a menu item is selected, or a set of keys is depressed, the front end initiates the token’s action. A token’s action can also be initiated by sending it from the kernel with this template,

FrontEndTokenExecute["TokenName"]

or even direct a token’s action to a particular notebook with this template.

FrontEndTokenExecute[nbObject,"TokenName"]

It only remains to determine which tokens correspond to the desired menu and keyboard commands. Starting with Mathematica version 3, there is a resource file — MenuSetup.tr — containing the front end menu tree in Mathematica expression form, which is read every time Mathematica launches. Embedded in that menu tree are the command tokens associated with each menu command, as well as the keyboard equivalents for those commands. Here are a few lines from the Windows version of that resource file:

Item["&New", "New", MenuKey["n", Modifiers->{Control}]]
Item["&Copy", "Copy", MenuKey["c", Modifiers->{Control}]]
Item["&Paste", Paste[After], MenuKey["v", Modifiers->{Control}]]

The relevant aspects of this syntax are: each Item expression contains information for a single menu item, and the second argument is the token or other expression that the front end will execute when the menu item is selected. (Further examples of the syntax of Item can be found in [Abb].) Thus, the token "New" corresponds to the File ⊳ New menu item, and "Copy" corresponds to Edit ⊳ Copy. To imitate these menu items, call the tokens directly from the kernel following the above template.

FrontEndTokenExecute["Copy"];
FrontEndTokenExecute["New"];

As for paste, Paste[After] is an example of a compound token, or a token with a parameter. To call a compound token from the kernel, specify the target notebook as the first argument, and the token and its parameter as second and third arguments, to FrontEndTokenExecute:

FrontEndTokenExecute[nbObject,"Paste", "After"];

That's all there is to it. Knowledge of tokens will allow a kernel program to imitate selecting over 90% of the menu items in Mathematica.

Not all tokens are listed in MenuSetup.tr. There is another resource file — KeyEventTranslations.tr — which defines keyboard shortcuts not associated with a menu item, or additional shortcuts for a menu item. For reference, lists of tokens extracted from these resource files appear at the end of this column. There are other tokens not listed in either of these resource files. As of this writing, there is no way to query the front end for a complete listing of tokens which it accepts.

The kernel command FrontEndTokenExecute is actually a shorthand for the following, more verbose command:

FrontEndExecute[{FrontEnd`FrontEndToken[args__]}]

To call tokens in buttons, use the above form as the button’s function. Such buttons do not require the kernel and so can have ButtonEvaluator->None, as discussed last time. For example, evaluating the following code will create a button that, when clicked, causes any initialization cells in its notebook to evaluate:

ButtonBox["Evaluate Init Cells",
ButtonFunction:>FrontEndExecute[{FrontEnd`FrontEndToken["EvaluateInitialization"]}],
ButtonEvaluator->None,
Active->True]//DisplayForm

See [DW] for another example of tokens in buttons.

When using tokens for automation, it is important to note that several of them are unsuitable because they bring up dialog boxes. For example, the "Open" token brings up a file open dialog, which requires user interaction. There is currently no way to interact with such a dialog using Mathematica or MathLink commands, so this is effectively a dead end for automation. It is sometimes possible to perform the same action in another way that can be automated — in the case of opening files, the NotebookOpen command can be used instead of the "Open" token.

Another common use for tokens is to set cell options and styles. For example, when a cell is selected, choosing the menu item Format ⊳ TextAlignment ⊳ Align Center activates the compound token {"TextAlignment", "Center"}, which is essentially a shorthand for SetOptions[NotebookSelection[InputNotebook[]], TextAlignment -> Center]. On the other hand, there is no simple command to replace the compound token which changes cell styles, such as {"Style", "Title"}.

Most menu items are identical on all platforms that Mathematica supports. Likewise, the tokens which underlie those menu items are identical, and can be used in platform independent programs. However, there are a small number of tokens which are platform specific, and so should be used with caution when developing solutions for multiple platforms. Tokens such as "XInfoDialog" and {"SelectionConvert", "Metafile"} rely on functionality built into the OS, which differs from platform to platform.

Packets

For the most part, knowledge of commands and tokens will be enough to use the notebook programming capabilities of Mathematica to the fullest. But this isn’t quite the whole story. Underlying both of these concepts is the MathLink communication that makes it all work.

The documentation for most MathLink programs begins by describing how to open a link between the kernel and the program. Other commands are needed which write to and read from the link when necessary. The front end is no different. Thankfully, little or no configuration is necessary in order to get a front end and kernel communicating via MathLink — provided they both reside on the same machine — and one rarely needs to resort to sending data across the link manually.

Even so, link commands are at the heart of the definition for the notebook commands discussed earlier. Consider the command Notebooks[]. Evaluating this command causes the kernel to send a MathLink packet to the front end, and then it waits for a response. Looking at Trace[Notebooks[]], the central idea is obfuscated by error checking. Here is the crux:

(LinkWrite[$ParentLink,FrontEnd`Notebooks[]];
LinkRead[$ParentLink])

The packet in this case is FrontEnd`Notebooks. Notice that it is distinguished from the command Notebooks by residing in a different context. The front end responds to this packet by sending information over the link back to the kernel. The kernel reads the data from the link, packages it up as appropriate, and then returns it as output. All built-in notebook commands are evaluated in approximately the same way:

• The user types a notebook command into the front end and presses ⇧|⏎;
• The front end sends the input to the kernel to be evaluated;
• The kernel translates the notebook command into a packet;
• The kernel sends the packet back to the front end;
• The front end interprets the packet and performs the specified action, if any;
• The front end sends the packet's return value, if any, back to the kernel;
• The kernel sends the formatted result of evaluation to the front end;
• The front end displays the result for the user.

This explains why front end commands continue to work even when the kernel is on a remote machine. Each command is ultimately interpreted in the front end. It doesn’t matter where the kernel is physically. This also explains the error message that is issued from these commands when Mathematica is not running through the front end.

Mathematica 4.0 Kernel for Power Macintosh
	Copyright 1988-1999 Wolfram Research, Inc.
	 -- Terminal graphics initialized -- 
	
	In[1]:= Notebooks[]
	
	FrontEndObject::notavail: 
	   A front end is not available; certain operations require a front end.
	
	Out[1]= Notebooks[]

Because all packet names reside in the FrontEnd` context, a list of known packets can be generated with Names["FrontEnd`*"]. For the most part, a command and its corresponding packet have the same name, differing only in context.

Some packets, such as FrontEnd`Notebooks, will return information over the link to the kernel, and some, such as FrontEnd`SetOptions, will not. Reading from a link when there’s nothing there, or leaving information on the link instead of reading it immediately, will cause the front end and kernel to become out of sync. If this happens, start a fresh Mathematica session.

Tokens also are evaluated by way of MathLink packet traffic. Note that tokens never return information to the link, so there's no need to call LinkRead.

LinkWrite[$ParentLink,FrontEnd`FrontEndExecute[{FrontEnd`FrontEndToken[nbObject,"Token","Parameter"]}]]

Token Listing

Here is a list of command tokens in Mathematica version 4.0.2, categorized by the menu in which they appear. It includes tokens from the resource files MenuSetup.tr and KeyEventTranslations.tr from every supported platform. Compound tokens are listed as ordered pairs of strings.

Comp: Format the following as you wish to make best use of space — multi column, font size, etc.

File Menu Tokens

"New"
"Open"
"Close"
"Save"
"SaveRename"
{"SaveRenameSpecial","V20Notebook"}
{"SaveRenameSpecial","Text"}
{"SaveRenameSpecial","CellExpression"}
{"SaveRenameSpecial","NotebookExpression"}
{"SaveRenameSpecial","Package"}
"OpenSpecial"
"Import"
"NotebookMail"
"NotebookMailSelection"
"ModifyNotebooksMenu"
"GeneratePalette"
"GenerateNotebook"
"SystemPrintOptionsDialog"
"PrintOptionsDialog"
"HeadersFootersDialog"
"PrintDialog"
"PrintSelectionDialog"
"FrontEndQuit"

Edit Menu Tokens

"Undo"
"Cut"
"Copy"
{"Paste","After"}
"Clear"
{"CopySpecial","PlainText"}
{"CopySpecial","FormattedText"}
{"CopySpecial","InputText"}
{"CopySpecial","CellExpression"}
{"CopySpecial","NotebookExpression"}
{"CopySpecial","CompleteNotebook"}
{"CopySpecial","PICT"}
{"CopySpecial","PICTBitmap"}
{"CopySpecial","PICTEmbeddedPS"}
{"CopySpecial","QuickTime"}
{"CopySpecial","EMF"}
{"CopySpecial","EPS"}
{"CopySpecial","MGF"}
{"CopySpecial","RTF"}
{"CopySpecial","Text"}
{"CopySpecial","WAV"}
{"PasteSpecial","PlainText"}
{"PasteSpecial","FormattedText"}
{"PasteSpecial","InputText"}
{"PasteSpecial","InlineCell"}
{"PasteApply","After"}
{"PasteDiscard","After"}
{"SelectionSaveSpecial","Text"}
{"SelectionSaveSpecial","InputText"}
{"SelectionSaveSpecial","CellExpression"}
{"SelectionSaveSpecial","NotebookExpression"}
{"SelectionSaveSpecial","CompleteNotebook"}
{"SelectionSaveSpecial","Package"}
{"SelectionSaveSpecial","EPS"}
{"SelectionSaveSpecial","MGF"}
{"SelectionSaveSpecial","PICT"}
{"SelectionSaveSpecial","PICTBitmap"}
{"SelectionSaveSpecial","PICTEmbeddedPS"}
{"SelectionSaveSpecial","PICS"}
{"SelectionSaveSpecial","QuickTime"}
{"SelectionSaveSpecial","EMF"}
{"SelectionSaveSpecial","Illustrator"}
{"SelectionSaveSpecial","RTF"}
{"SelectionSaveSpecial","WAV"}
{"SelectionSaveSpecial","WMF"}
"SelectAll"
"InsertObject"
"MoveNext"
"MovePrevious"
"DeleteNext"
"DeletePrevious"
"MoveNextWord"
"MovePreviousWord"
"DeleteNextWord"
"DeletePreviousWord"
"MoveLineEnd"
"MoveLineBeginning"
"MoveNextLine"
"MovePreviousLine"
"Superscript"
"Subscript"
"Above"
"Below"
"Otherscript"
"Fraction"
"Radical"
"NewRow"
"NewColumn"
"MoveExpressionEnd"
"ExpandSelection"
"CreateInlineCell"
"MoveNextCell"
"InsertRawExpression"
"AddInterpretation"
{"SpanMaxSize","Infinity"}
{"SpanMaxSize","1"}
{"SpanMaxSize",etc…}
{"SpanMaxSize","Automatic"}
"NudgeLeft"
"NudgeRight"
"NudgeDown"
"NudgeUp"
"RemoveAdjustments"
"Make2D"
"Balance"
"FindNextMisspelling"
"OptionsDialog"

Cell Menu Tokens

{"SelectionConvert","InputForm"}
{"SelectionConvert","OutputForm"}
{"SelectionConvert","PostScript"}
{"SelectionConvert","Bitmap"}
{"SelectionConvert","PICT"}
{"SelectionConvert","PICTBitmap"}
{"SelectionConvert","PICTEmbeddedPS"}
{"SelectionConvert","QuickTime"}
{"SelectionConvert","Metafile"}
{"SelectionDisplayAs","TextForm"}
{"SelectionDisplayAs","InputForm"}
{"CommonDefaultFormatTypesInput","InputForm"}
{"CommonDefaultFormatTypesOutput","OutputForm"}
{"CommonDefaultFormatTypesOutput","InputForm"}
{"CellOpen","Toggle"}
{"CellOpen","True"}
{"CellOpen","False"}
{"Editable","Toggle"}
{"Editable","True"}
{"Editable","False"}
{"CellEditDuplicate","Toggle"}
{"CellEditDuplicate","True"}
{"CellEditDuplicate","False"}
{"Evaluatable","Toggle"}
{"Evaluatable","True"}
{"Evaluatable","False"}
{"Active","Toggle"}
{"Active","True"}
{"Active","False"}
{"InitializationCell","Toggle"}
{"InitializationCell","True"}
{"InitializationCell","False"}
"CellGroup"
"CellUngroup"
{"CellGrouping","Manual"}
{"CellGrouping","Automatic"}
"SelectionOpenAllGroups"
"SelectionCloseAllGroups"
"OpenCloseGroup"
"CellSplit"
"CellMerge"
"SelectionAnimate"
"SoundPlay"
"GraphicsRender"
"GenerateImageCaches"
"GraphicsOriginalSize"
"GraphicsAlign"
"NotebookStatisticsDialog"

Format Menu Tokens

{"Style","Tile"}
{"Style","Subtitle"}
{"Style","Section"}
{"Style",etc…}
"StyleOther"
"ToggleShowExpression"
"OptionsDialog"
"ClearCellOptions"
"StyleDefinitionsOther"
"EditStyleDefinitions"
"FontPanel"
{"FontFamily","Courier"}
{"FontFamily","Times"}
{"FontFamily",etc…}
"PlainFont"
{"FontWeight","Toggle"}
{"FontWeight","Plain"}
{"FontWeight","Bold"}
{"FontWeight",etc…}
{"FontSlant","Toggle"}
{"FontSlant","Plain"}
{"FontSlant","Italic"}
{"FontSlant",etc…}
{"FontVariationsUnderline","Toggle"}
{"FontVariationsUnderline","True"}
{"FontVariationsUnderline","False"}
{"FontVariationsOutline","Toggle"}
{"FontVariationsOutline","True"}
{"FontVariationsOutline","False"}
{"FontVariationsShadow","Toggle"}
{"FontVariationsShadow","True"}
{"FontVariationsShadow","False"}
{"FontTracking","Condensed"}
{"FontTracking","Extended"}
{"FontSize","Smaller"}
{"FontSize","Larger"}
{"FontSize","9"}
{"FontSize","10"}
{"FontSize","12"}
{"FontSize","14"}
{"FontSize",etc…}
"FontSizeDialog"
{"FontColor","GrayLevel[0]"}
{"FontColor","GrayLevel[1]"}
{"FontColor","GrayLevel[0.1]"}
{"FontColor","RGBColor[1, 0, 0]"}
{"FontColor","RGBColor[0, 1, 0]"}
{"FontColor",etc…}
"FontColorDialog"
"ToggleGrayBox"
{"Background","None"}
{"Background","GrayLevel[0]"}
{"Background","GrayLevel[1]"}
{"Background","GrayLevel[0.1]"}
{"Background","RGBColor[1, 0, 0]"}
{"Background","RGBColor[0, 1, 0]"}
{"Background",etc…}
"BackgroundDialog"
{"TextAlignment","Left"}
{"TextAlignment","-0.5"}
{"TextAlignment","Center"}
{"TextAlignment","0.5"}
{"TextAlignment","Right"}
{"TextAlignment","AlignmentMarker"}
{"TextJustification","0."}
{"TextJustification","0.25"}
{"TextJustification","0.5"}
{"TextJustification","0.75"}
{"TextJustification","1."}
{"PageWidth","Infinity"}
{"PageWidth","PaperWidth"}
{"PageWidth","WindowWidth"}
{"PageWidth",etc…}
"FixCellWidth"
{"CellDingbat","None"}
{"CellDingbat","■"}
{"CellDingbat"etc…}
{"CellFrame","Inherited"}
{"CellFrame","{{0, 0}, {0, 0.5}}"}
{"CellFrame","{{0, 0}, {0, 2}}"}
{"CellFrame","{{0, 0}, {0, 3}}"}
{"CellFrame","{{0, 0}, {0.5, 0}}"}
{"CellFrame","{{0, 0}, {2, 0}}"}
{"CellFrame","{{0, 0}, {3, 0}}"}
{"ShowPageBreaks","Toggle"}
{"ShowPageBreaks","True"}
{"ShowPageBreaks","False"}
{"Magnification","0.75"}
{"Magnification","1."}
{"Magnification","1.25"}
{"Magnification",etc…}

Input Menu Tokens

"GraphicsCoordinatesDialog"
"ViewPointSelectorDialog"
"ColorSelectorDialog"
"RecordSoundDialog"
"FileNameDialog"
"CreateGridBoxDialog"
"CreateCustomButtonBox"
"EditButtonBoxDialog"
"CreateHyperlinkDialog"
"CreateCounterBoxDialog"
"CreateValueBoxDialog"
{"DuplicatePreviousInput","After"}
{"DuplicatePreviousOutput","After"}
"SimilarCellBelow"
{"CompleteSelection","True"}
"TemplateSelection"

Kernel Menu Tokens

"EvaluateCells"
{"Evaluate","All"}
"EvaluateNextCell"
"SubsessionEvaluateCells"
"EvaluateNotebook"
"EvaluateInitialization"
{"EnterSubsession","Automatic"}
{"ExitSubsession","Automatic"}
"RemoveFromEvaluationQueue"
{"EvaluatorInterrupt","Automatic"}
{"EvaluatorAbort","Automatic"}
"ModifyEvaluatorNames"
{"ShowCellLabel","Toggle"}
{"ShowCellLabel","True"}
{"ShowCellLabel","False"}
"DeleteGeneratedCells"

Find Menu Tokens

"FindDialog"
"SelectionSetFind"
"Replace"
"ReplaceFind"
"ReplaceAll"
"OpenSelectionParents"
"SelectionScroll"
"HyperlinkGoBack"
"CellTagsEditDialog"
{"ShowCellTags","Toggle"}
{"ShowCellTags","True"}
{"ShowCellTags","False"}
"CellLabelsToTags"
"CreateIndexDialog"

Window Menu Tokens

"StackWindows"
"TileWindowsWide"
"TileWindowsTall"

Help Menu Tokens

"AboutBoxDialog"
"XInfoDialog"
"HelpDialog"
"SelectionHelpDialog"
"ExplainBeepDialog"
"RebuildHelpIndex"

Key Event Tokens

"Above"
"Below"
"Copy"
"CreateInlineCell"
"Cut"
"DeleteNext"
"DeleteNextExpression"
"DeletePrevious"
"DeletePreviousExpression"
"EvaluateCells"
"EvaluateNextCell"
"ExpandSelection"
"Fraction"
"InsertRawExpression"
"Linebreak"
"MoveExpressionEnd"
"MoveLineBeginning"
"MoveLineEnd"
"MoveNext"
"MoveNextCell"
"MoveNextExpression"
"MoveNextLine"
"MoveNextWord"
"MovePrevious"
"MovePreviousExpression"
"MovePreviousLine"
"MovePreviousPlaceHolder"
"MovePreviousWord"
"NewColumn"
"NewRow"
"NudgeDown"
"NudgeLeft"
"NudgeRight"
"NudgeUp"
"Otherscript"
"Radical"
"ScrollNotebookEnd"
"ScrollNotebookStart"
"ScrollPageDown"
"ScrollPageUp"
"SelectionHelpDialog"
"SelectionSpeak"
"SelectionSpeakSummary"
"SelectLineBeginning"
"SelectLineEnd"
"SelectNext"
"SelectNextLine"
"SelectNextWord"
"SelectPrevious"
"SelectPreviousLine"
"SelectPreviousWord"
"ShortNameDelimiter"
"SimilarCellBelow"
"Subscript"
"Superscript"
"Tab"
"Undo"
{"CompleteSelection","False"}
{"Evaluate","All"}
{"Paste","After"}

References

[Abb]	Abbott, Paul, ed. “Tricks of the Trade” The Mathematica Journal . Vol. 7, No. 2, pp 105–127. 1998.

[DW]	D’Andria, Louis J. and Stan Wagon. “Front End Grab Bag”, Mathematica in Education and Research. Vol. 6, No. 2, pp 54–57. 1997.

[GD]	Gaylord, Richard J. and Louis J. D’Andria. Simulating Society: A Mathematica Toolkit for Modeling Socioeconomic Behavior. TELOS/Springer-Verlag, New York, 1998.

[GG]	Glynn, Jerry and Theodore Gray. The Beginner’s Guide to Mathematica Version 4. Cambridge University Press, 2000.

[Mae]	Maeder, Roman. Programming in Mathematica, Third Edition. Addison-Wesley, 1996.

[Wol]	Wolfram, Stephen. The Mathematica Book, Fourth Edition. Wolfram Media, Cambridge University Press, 1999.

About the Author

Comp: I assume you still have my "about the author" blurb from the "old" column. I’ll review it at proof time, to make sure it’s still valid.