To Manipulate And Beyond

Programming the New Interface Paradigm

Lou D'Andria
Wolfram Research

The forthcoming version of Mathematica includes high-level functions like Manipulate which let users immediately and effortlessly leverage the new interface paradigm. However, the highly powerful dynamic primitives from which Manipulate is built are also accessible, and can be used directly in Mathematica's symbolic programming language to build arbitrary interfaces. The unique dynamic tracking mechanism and accessible syntax allow developers to produce complete interfaces in a few lines of simple code. In this talk, we will discuss these primitives, and show examples of using them to build structures like, and unlike, Manipulate.

Why Manipulate Is Unique

Manipulate's syntax is extraordinarily simple: if you know how to write a Table or a Do loop, you can leverage the power of Manipulate.

Manipulate[Plot[Sin[n x],{x,0,10}], {n,1,5}]

But how can you leverage the power behind  Manipulate?

Why Manipulate Is Not Unique

Every bit of every feature of Manipulate is accessible to every Mathematica developer. In fact, Manipulate is written entierly in the Mathematica programming language.

There are many other features in Mathematica that are built up from the same primitives as Manipulate.

Functions

Animate
ListAnimate
ControllerManipulate
SystemInformation
Input
InputString
MessageDialog
ChoiceDialog
Monitor

Other  features

Debugger
About Box
Preferences Dialog
Install Dialog
Options Dialog
Notebook History

Large-output suppression

Range[10^5]

Historical Background

So, How Did We Get to Manipulate?

Historical Background

In Versions 1-2, We Had Evaluation

The journey of a thousand miles begins with a single step.

Who here has run Version 1?
Version 2?

Who still runs it?

Historical Background

In Versions 3-5, We Had Boxes

The fact that Mathematica notebook documents were represented by a Mathematica expression allowed several key technologies, like typesetting and structured editing.

However, even though the box level was very powerful, it was quite awkward to use, and so it was rare for people to use boxes to do interesting things.

GridBox
ButtonBox
StyleBox
Cell
BoxData

DisplayForm
MakeBoxes
CellPrint
NotebookPut

Still, there was a lot of power in moving to structured documents,

Example from MiER, 9:1

label="Smallest Seven Digit Prime";
f[] := Block[{c=10^6}, While[!PrimeQ[++c]]; Print[c]]

ButtonBox[label,ButtonFunction:>f[],ButtonEvaluator->Automatic,Active->True]//DisplayForm

Historical Background

Now, Well, We Still Have Boxes

Because they are still very powerful. But we have something new: dedicated box generators.

Programmatically creating boxes in output is just a matter of evaluating one of the dozens of box generators, Grid or Button or Style.

Grid
Button
Style
ExpressionCell

MiER example refactored

Button[label, f[]]

And We Have Lots of New Boxes, and Combinations of Boxes

Luckily, they all have box generators

We will look at a few of these generators in detail. But from here on out, for the most part, you can essentially forget about boxes, and think instead in terms of functions that produce the structures you want, like Grid or Button or Style.

Layout and views

Manipulate
Grid
Column
Row
Subscript
Superscript
Underscript
Overscript
TabView
OpenerView
FlipView
SlideView
PopupView
MenuView

Styling and annotations

Style
Item
Deploy
Tooltip
Mouseover
StatusArea
Annotation
Panel
Magnify
Framed
Labeled
Invisible
Pane
RawBoxes

Notebook structure

DocumentNotebook
PaletteNotebook
DialogNotebook
PaletteNotebook
ExpressionCell
TextCell
CellGroup

Standard controls

LIke Button, other controls are implemented directly as boxes in the front end.

Button
Slider
VerticalSlider
Checkbox
RadioButton
PopupMenu
InputField
ProgressIndicator
Hyperlink
Opener

Not-so-standard controls

Animator
ActionMenu
Manipulator
PasteButton
DefaultButton
CancelButton
Slider2D
PaneSelector
Setter
SetterBar
Toggler
TogglerBar
RadioButtonBar
CheckboxBar
FileNameSetter
ColorSetter
ColorSlider

Locator
SliderPane
ClickPane
EventHandler

Details: Grid

Create arbitrary two-dimensional layouts.

data=CellularAutomaton[30,{{1},0},15];

Dimensions[data]

Grid[data]

Grid[data //. {0 -> "□", 1->"■"}, Spacings -> {0,-.2}]

Example from yesterday's Manipulate talk

Style[Grid[{
{Magnify[Style["ControlTypes",Bold],1.2], SpanFromLeft},
{Style["Variable Specification", Bold], Style["Default Control Type", Bold]},
{{var,xmin,∞}, Animator},
{{var, xmin, xmax}, Manipulator},
{{var,xmin,xmax,Δx},Manipulator},
{{var,{xmin,ymin},{xmax,ymax}}, Slider2D},
{{var,{xmin,ymin},{xmax,ymax},{Δx,Δy}}, Slider2D},
{{var,{True, False}}, Checkbox},
{{var, list}, "PopupMenu or SetterBar"},
{{var, color}, ColorSlider},
{{var, other}, InputField}
}, ItemSize -> {"Columns" -> {{Scaled[0.5]}}},Background-> {"Rows" -> {White, {White, GrayLevel[0.9]}}}, Dividers->{"ColumnsIndexed"->{2 -> True},"RowsIndexed"->{3 -> True}}], FontFamily->"Helvetica", TextAlignment -> Center]

Dividers example from Theo Gray

Manipulate[Style[Grid[
  ReplacePart[Table[a,{9},{9}],
   {{2,9}->SpanFromLeft,
    {8,4}->SpanFromLeft,
    {9,3}->SpanFromAbove,
    {9,4}->SpanFromBoth}],
GridBoxDividers->{{
Table[Directive[{Hue[i],Opacity[.5]}],{i,0,1,0.1}],
{9->Directive[Thickness[2],Dashing[{2,2}]],
{2,4}->None,
{2,5}->None,
{4,4}->None,
{4,5}->None,
{3,1}->Green,
{3,4}->Thickness[5],
{1,1}->{Blue,Thickness[th]},
{2,1}->{Green,Thickness[10],Opacity[1]},
Apply[Sequence,Table[{i,7}->Thickness[3 i],{i,1,7}]]}
},{
Table[Directive[Hue[3.5 i]],{i,0,1,0.1}],
{
5->Directive[Blue,Dashing[{4,4}]],
6->Thickness[2],
 Apply[Sequence,Table[{8,i}->Thickness[3 i],{i,1,7}]],
{3,3}->None,
{4,3}->None,
{3,5}->None,
{4,5}->None,
{7,7}->{Red,Thickness[5]},
{2,1}->{Green,Thickness[10],Opacity[1]}}
}}],Thickness[10],Dashing[{}]],{th,0,20}]

Details: Style

It used to be very difficult to get anything but 12pt Courier in output. Style is the main function that make's it easy to get other things.

Style takes an expression, followed by any number of styles or options which are used to style the expression.

Style[Grid[{{"This", "is"},{"a","grid."}}], FontSize -> 24, FontFamily -> "Times"]

Note that almost all box generators persist through evaluation and reevaluation, including Style.

Map[If[PrimeQ[#], Style[#,Orange, Bold], #]&, Range[100]]

Grid[Partition[%,10]]

The Single Most Important Construct in the New Interface Paradigm Is…

No, it's not Manipulate

Manipulate goes to some lengths to hide this construct from the casual user…

The Single Most Important Construct in the New Interface Paradigm Is…

Dynamic

Dynamic is the function that allows you to connect front end controls to kernel variables, to make structures which depend on the state of other structures, and ultimately to build interfaces like Manipulate.

Dynamic has two main features.

Displaying the current value of an expression

The output of Dynamic[expr] always looks like it was evaluated right now, no matter when it was actually evaluated.

Dynamic[expr]

expr = Plot[Sin[x],{x,0,6}];

expr = Range[10];

expr = Plot[Cos[x],{x,0,6}];

Note that Dynamic[expr] in the kernel evaluates to itself, just like Hold[expr]. The evaluation only evaluates when the output is displayed in the front end.

Connecting expressions to controls

Instead of specifying the behavior controls by handling events as is typical for professional interface programmers, we use Dynamic do attach controls to individual variables or expressions. From then on, the control will reflect the value of that variable, and vice versa.

Dynamic[x]

Slider[Dynamic[x]]

Scoping Dynamic Variables

In order to build Manipulate, we need a scoping construct that is aware of Dynamic's delayed evaluation behavior.

DynamicModule

DynamicModule is such a scoping construct.

DynamicModule[{x=0},{Slider[Dynamic[x]], Dynamic[x]}]

DynamicModule is also the construct that remembers variable values.

Building Manipulate

Build a simple version of Manipulate

Beginnings

{Slider[Dynamic[x]],Dynamic[x]}

Aesthetics

Panel[Column[{Slider[Dynamic[x]], Item[Dynamic[x], Background -> White, FrameStyle -> GrayLevel[0.8]]}], FrameMargins -> 10, DefaultBaseStyle -> {}]

Scoping

DynamicModule[{x}, Panel[Column[{Slider[Dynamic[x]], Item[Dynamic[x], Background -> White, FrameStyle -> GrayLevel[0.8]]}], FrameMargins -> 10, DefaultBaseStyle -> {}]]

Etc.

Manipulate has lots of features beyond this simple example, and all of them are built in top-level code.

SetAttributes[aMenu, HoldFirst];
aMenu[x_]:=Tooltip[ActionMenu[Style["★", Blue,24],{"random":>(x=Random[]), Delimiter,"top":>(x=1),"mid":>(x=0.5),"bottom":>(x=0)}, Appearance -> None],"presets"]

Deploy[DynamicModule[{x=0}, Panel[Column[{Row[{Slider[Dynamic[x]],aMenu[x]},Spacer[10]], Item[Dynamic[x], Background -> White, FrameStyle -> GrayLevel[0.8]]}], FrameMargins -> 10, DefaultBaseStyle -> {}]]]

Details: Dynamic

Dynamic can appear as a primitive almost anywhere in a Mathematica expression that will be displayed in a notebook, and it will just work. For example…

Outisde graphics

Column[{
Row[{Checkbox[Dynamic[axes]], "Axes"}],
Row[{Checkbox[Dynamic[boxed]], "Boxed" }],
Dynamic[Plot3D[Sin[x y],{x,0,3},{y,0,3}, Axes -> axes,Boxed -> boxed]]
}]//Panel

Inside graphics

Column[{
Row[{Checkbox[Dynamic[axes]], "Axes"}],
Row[{Checkbox[Dynamic[boxed]], "Boxed" }],
Plot3D[Sin[x y],{x,0,3},{y,0,3}, Axes -> Dynamic[axes],Boxed -> Dynamic[boxed]]
}]//Panel

Nested Inside other Dynamics

Dynamic updating keeps track of which dynamics depend on which variables, and updates only those it needs to. This not updating 'outer' dynamics in favor of 'inner' dynamics when there is nesting.

Column[{
Row[{Checkbox[Dynamic[axes]], "Axes"}],
Row[{Checkbox[Dynamic[boxed]], "Boxed" }],
Dynamic[Plot3D[Sin[x y],{x,0,3},{y,0,3}, Axes -> Dynamic[axes],Boxed -> Dynamic[boxed]]]
}]//Panel

Elsewhere

You can use Dynamic almost anywhere in a box generator and it will just work. If it doesn't, move it up.

• boxes
• box options
• stylesheets
• cell groups
• window options
• global options
• …

Beyond Manipulate: Views

Function names that end in View all have the same purpose: to provide a pre-built interface for viewing a set of things. TabView will be familiar to most computer users.

funcs = {Sin, Cos, Tan, Sec, Csc, Cot};

TabView[Plot[#[x],{x,0,10}]& /@ funcs]

You can specify arbitrary labels in all view functions.

TabView[# -> Plot[#[x],{x,0,10}]& /@ funcs]

x=.;
TabView[((#[x])^2) -> Plot[((#[x])^2),{x,0,10}]& /@ funcs]//TraditionalForm

Lists are everywhere in Mathematica

Views can come in very handy as a quick, one-off interface.

Solve[a x^3+b (x^2)+c x + d==0,x]//TabView

Other Views

?*View

funcs = {Abs,AiryAi,AiryAiPrime,AiryBi,AiryBiPrime,ArcCos,ArcCosh,ArcCot,ArcCsc,ArcCsch,ArcSec,ArcSech,ArcSin,ArcSinh,ArcTan,Arg,Ceiling,Clip,Conjugate,Cos,Cosh,Cot,Coth,Csc,Csch,EllipticE,Erf,Erfc,Erfi,Exp,ExpIntegralEi,Factorial,Factorial2,Fibonacci,Floor,FractionalPart,Gamma,HeavisideTheta,Im,IntegerPart,KelvinBei,KelvinBer,KelvinKei,KelvinKer,Log,LogGamma,Max,Min,Minus,Multinomial,Plus,PolyGamma,Power,Re,RiemannSiegelTheta,RiemannSiegelZ,Round,Sec,Sech,Sign,Sin,Sinc,Sinh,Sqrt,Tan,Tanh,Times,UnitStep};

MenuView[# -> Plot[#[x],{x,0,10}]& /@ funcs]//Timing

Delaying evaluation by using Dynamic, and specifying an automatic ImageSize, are two ways to speed up the time required by the initial evaluation.

MenuView[# -> Dynamic[Plot[#[x],{x,0,10}]]& /@ funcs, ImageSize -> Automatic]//Timing

Of course, by the time you do this, you might as well be using Manipulate…

Manipulate[Plot[f[x],{x,0,10}], {f,funcs}]

Beyond Manipulate: Singleton Interfaces

Interfaces can be as simple as a single checkbox or slider.

Plot[Sin[x],{x,0,10}, Axes -> Checkbox[], Frame -> Checkbox[]]

Plot[Sin[x],{x,0,10},Axes -> DynamicSetting[Checkbox[False]],Frame -> DynamicSetting[Checkbox[True]]]

This concept of generalized input can be taken to much futher extremes. See the advanced dynamic tutorial for examples.

In short…

DynamicSetting is a forethough; Setting is an afterthought.

Beyond Manipulate: Experimental Features

Interesting features that won't be polished in time for release, but still interesting. (You didn't hear about these from me…)

Manipulate`Control

Here is a syntax for accessing controls as easily outside Manipulate as inside.

Manipulate`Control[{x,1,5,1}]

Manipulate`Control[{x,Range[5]}]

Manipulate`Control[{x,Range[10]}]

Manipulate`Control[{x,{True,False}}]

Experimental`ControlBlock

This is a way to hide specific sets of controls behind a labeled Opener.

Manipulate[
  Plot[amp f[freq x],{x,0,6},PlotStyle->{color,Dashing[dashing],Thickness[thickness]},Axes->axes,Frame->frame,AxesOrigin->axesorigin,PlotLabel->f],
  {f, {Sin, Cos, Tan}},

  Experimental`ControlBlock["Function Variations",
  {freq, 1, 5},
  {amp, 1, 5}],

  Experimental`ControlBlock["Axes and Frames",
  {axes, {True, False}},
  {frame, {False, True}},
  {axesorigin, {0, 0}, {6, 1}}],

  Experimental`ControlBlock["Plot Style",
  {color, Blue},
  {dashing, 0, 0.1},
  {thickness, 0.001, 0.1}] ]

Beyond Manipulate: Controllers

Mathematica allows full access to the state of any controllers which are attached to your computer. ControllerInformation provides an overview of all available controllers, and ControllerState can be used to extract individual pieces of information.

SystemInformation[]

ControllerInformation[]

Dynamic[
Framed[Graphics[Disk[{0,0}+#], PlotRange->2]]& /@ ControllerState[{"XY1","XY2"}]]

Motion sensor level example

Graphics[{Gray,Rectangle[{-3,-1},{3,1}],Rectangle[{-1,-3},{1,3}],White,Disk[],Black,Circle[{0,0},0.2],Opacity[0.4],Dynamic[{x,y,z}=ControllerState["Sudden Motion Sensor",{"X","Y","Z"}];{Red,Disk[{-2 x,0}],Green,Disk[-2 {y,y}],Blue,Disk[{0,-2 z}],Black,Disk[{-2 x,-2 z},0.2],Opacity[1],Inset[Grid[{{Style["●",Red],"x",x},{Style["●",Green],"y",y},{Style["●",Blue],"z",z},{Style["●",Black],"(x,z)",""}}],{2,2}]}]}]

Localizing controller actions

The main way that controller actions can be localized is via selections. That is how Manipulate works by default. A Manipulate output is only affected by a controller if the output is selected. How is that done?

Dynamic[If[CurrentValue["SelectionOver"],x=ControllerState["X"]];x]

Dynamic[If[CurrentValue["NotebookSelected"],x=ControllerState["X"]];x]

Beyond Manipulate: Dynamic's Second Argument

Custom tracking functions

In the output of Slider[Dynamic[x]], the location of a slider's thumb reflects the current value of x, and dragging the slider thumb to newval essentially evaluates x = newval.

You can override this default behavior by giving a second argument to Dynamic, specifying up to three custom tracking functions.

Panel[Grid[{
{"", "Mouse Down", "Mouse Dragged", "Mouse Up"},
{InputForm[Slider[Dynamic[x]]], Defer[x=newval], Defer[x=newval], Defer[x=newval]},
{InputForm[Slider[Dynamic[x, f  | {f}]]], Defer[f[newval,x]], Defer[f[newval,x]], Defer[f[newval,x]]},
{InputForm[Slider[Dynamic[x,{f,h}]]], Defer[f[newval,x]], Defer[f[newval,x]], Defer[h[newval,x]]},
{InputForm[Slider[Dynamic[x,{f,g,h}]]], Defer[f[newval,x]], Defer[g[newval,x]], Defer[h[newval,x]]}
}, Dividers->{"ColumnsIndexed"->{2 -> True},"RowsIndexed"->{2 -> True}}, Spacings -> {2,1}], DefaultBaseStyle -> "Text"]

Of course, this goes for all controls, not just Slider.

Dynamic[x]

Slider[Dynamic[x,(x=#)&]]

Slider[Dynamic[1-x,(x=1-#)&]]

Restricting example

Here's a slider that only sets the variable to prime numbers.

{Slider[Dynamic[p, If[PrimeQ[#], p=#]&], {2,100,1}], Dynamic[p]}

Locator example from Arnoud Buzing

p={-0.5,-0.5};
c1={0,0};r1=1;
c2={.25,.25};r2=.5;
c3={-.5,0};r3=.2;
c4={0,-.5};r4=.2;
color1=White;
color2=Pink;
Graphics[{{color1,Disk[c1,r1]},{color2,Disk[c2,r2]},{color2,Disk[c3,r3]},{color2,Disk[c4,r4]},Locator[Dynamic[p,(p=Which[Norm[#]<1∧Norm[#-c2]>r2∧Norm[#-c3]>r3∧Norm[#-c4]>r4,#,Norm[#]≥1,#/Norm[#],Norm[#-c2]≤r2,c2+r2(#-c2)/(Norm[#-c2]),Norm[#-c3]≤r3,c3+r3(#-c3)/(Norm[#-c3]),Norm[#-c4]≤r4,c4+r4(#-c4)/(Norm[#-c4])])&]]},PlotRange->1,PlotRangePadding->.25,Background->color2]

Beyond Manipulate: Event Handling

Mathematica provides EventHandler for those situations where you want to alter the typical mouse and keyboard interaction. Applications involving EventHandler are usually written at a very low level, and the resulting code may be familiar to those with more traditional user interface experience.

ZoomPlot, inspired by Rob Morris

x=.;

ZoomPlot[f_,{x_,xmin0_,xmax0_},opts___]:=DynamicModule[{dragdisp,xmin,xmax,xrange,yrange,theclick},dragdisp={};{xrange,yrange}={{xmin0,xmax0},Automatic};Column[{EventHandler[Dynamic[{xmin,xmax}=xrange;Show[Plot[f,{x,xmin,xmax},Epilog->Dynamic[dragdisp],opts,ImageSize->Medium],PlotRange->Dynamic[{xrange,yrange}]]],{"MouseDown":>(theclick=MousePosition["Graphics"]),"MouseDragged":>(dragdisp={Opacity[0.1],Rectangle[theclick,MousePosition["Graphics"]]}),"MouseUp":>({xrange,yrange}=Sort/@Transpose[{theclick,MousePosition["Graphics"]}];dragdisp={})}],Button["reset",{xrange,yrange}={{xmin0,xmax0},Automatic},ImageSize->Automatic]}]]

ZoomPlot[Sin[1/x],{x,-1,1},PlotRange->All,Axes->True]

Managing Dynamics

There are several measures in place that allow you to tweak the dynamic updating mechanism

Dynamic timeout

By default, dynamic evaluation will timeout after six seconds. You can adjust this via the DynamicEvaluationTimeout front end option.

Disabling dynamic updating

You can globally turn off dynamic updating in the Evaluation menu. This is most useful if you are developing dynamic content which is running amok.

Refresh

If you know ahead of time that you want a particular Dynamic's contents to update in some non-standard way, you can wrap the contents with Refresh.

Options[Refresh]

{Slider[Dynamic[x]], Slider[Dynamic[y]],Dynamic[Refresh[{x,y}, TrackedSymbols :> {x}]]}

Synchronicity

Only preemptive link evaluations are subject to the dynamic timeout. You can instruct Dynamic to use the main link instead by setting SynchronousUpdating -> False.

More About Dynamic

Tutorials and Examples

There are very useful tutorials on Dynamic. which are linked from Dynamic's documentation.

?Dynamic

Working with Windows

Sometimes having your interfaces appear as output in your notebook is sufficient, but sometimes you want them to be in a separate window.

Notebook creation

CreatePalette
CreateDialog
DialogInput

Pre-built dialogs

Input[] and InputString[] are examples of non-modal dialog windows that block in the kernel.

Integrate[Input[], t]

ChoiceDialog[] is another blocking, non-modal dialog.

Plot[Sin[x],{x,0,10}, ImageSize -> ChoiceDialog["Choose the plot's size:", {Tiny,Small,Medium,Large}]]

MessageDialog[] is an example of a non-modal, non-blocking dialog.

MessageDialog["Your operation is complete"]

Blocking vs Modal

When building an interface that will be deployed in its own notebook dialog window, there are two important questions:

1) Should the kernel's main evaluation thread wait for the user to interact with the dialog or not?
2) Should the user be allowed to interact with other notebook windows while the dialog is open or not?

Question 1 determines whether to use DialogInput or CreateDialog.
Question 2 determines the setting for the Modal option to those functions.

Be careful with modal windows!!

Using Modal -> True will really create a modal window. Really. There's no way to interact with the front end until you've dealt with the modal window.

During development, do not set Modal -> True until you are sure that you have a working DefaultButton or CancelButton. A modal window that can't be dismissed will end your ability to interact with that front end session, and you'll have to kill Mathematica to get control back.

Beyond Manipulate: Word Data

I suggested to someone yesterday that they write an interface to WordData, and possibly all the other data paclets, to take the hassle (typing)  out of discovery. They said "that's something that you would do, isn't it?"

No! The point of this talk is to empower you to do these things quickly and easily.

Implementation

Unprotect[WordData];
WordData[]:=CreateDialog[
DynamicModule[{possibleTypes = WordData["the","Properties"]//Reverse, word="set"},
Column[{
InputField[Dynamic[word],String], Dynamic @ TabView[MapThread[(Row[{#1," (",Length[#2],")"}]->If[#2==={},"None",Column[#2]])&,{possibleTypes, (If[!ListQ[#], {}, #]& @ WordData[word, #])& /@ possibleTypes}], ControlPlacement -> {Top,Left}] 
}]], NotebookEventActions->{}, WindowTitle  -> "WordData", WindowSize -> FitAll, WindowMargins -> {{Automatic,Automatic},{Automatic,0}}]
Protect[WordData];

Use

WordData[]

Manipulate version

Yep, it's shorter.

Manipulate[Style[TabView[MapThread[(Row[{#1," (",Length[#2],")"}]->If[#2==={},"None",Column[#2]])&,{possibleTypes, (If[!ListQ[#], {}, #]& @ WordData[word, #])& /@ possibleTypes}], ControlPlacement -> {Top,Left}], "DialogStyle"],{{word,"set"},InputField[#, String]&}, Initialization:>(possibleTypes = WordData["the","Properties"]//Reverse)]

Conclusion

In summary:

• Manipulate combines dynamic primitives and box generators is a natural and effective way.

• Manipulate is written entirely in top-level Mathematica code.

• Dynamic is the main framework on which the new interface paradigm has been built.

• Box generators take the power of boxes and put it in the hands of even novice users.

• Views can serve as powerful one-off interfaces.

• There is a rich new language of interface creation, from EventHandler to Manipulate.

• Things like Manipulate can be written by people like you very easily.

So get to work!