DebugTrace – an alternative debugger

In search of the optimal debug experience

David Bailey

http://www.dbaileyconsultancy.co.uk/

Abstract

Mathematica has had a built-in debugger since version 6.0, however, in this paper I will present a free alternative debugger, DebugTrace, based on rather different principles, that are more attuned to the absolute beginner user, and it also contains features helpful in very complex debugging scenarios. DebugTrace operates on whole notebooks (or multi-cell selections) – which accommodates typical beginner coding style – and uses line numbers rather than code-highlighting to denote program location. This, in turn means that a debug session can be recorded for future inspection. Steps in the program are also numbered, so that a subsequent debug session can be instructed to skip at once to a point of interest. Other features include:
•  An awareness of high level concepts, such as functions, rather than presenting a purely kernel-oriented viewpoint.
•  A mechanism to find the values of active local variables, without knowing their $n suffix. This is particularly useful to construct conditional breakpoints, where the value of n may vary, as, for example, when a module is repeatedly entered.
•  The ability to print out the values of pattern arguments to functions.
•  Some ability to debug dynamic constructs.
•  Breakpoints can be toggled on or off by merely clicking the relevant line number in a special 'listing' notebook created on the fly.
•  DebugTrace is written as a normal user package and is not encoded, so it could be extended in various ways.
•  Full documentation is provided, in the new help system format.

The talk will conclude with a discussion of how DebugTrace is implemented, and will hopefully initiate a debate about desirable features in a Mathematica debugger.

A trivial program to debug

Here is a trivial program to debug. It calculates Legendre polynomials using the 3-term recurrence relation. It is deliberately written in a naive way using a loop that processes an array – since that is what a beginner is likely to do, particularly if he/she has had experience of other programming languages.

poly=Table[0,{10}];

CalculateLegendre[]:=Module[{n},
poly[[0+1]]=1;
poly[[1+1]]=x;
For[n=2,n≤10,n++,
(* This applies the recurrence relation *)
poly[[n+1]]=Expand[Simplify[1/n((2n-1)x poly[[n-1+1]]-(n-1)poly[[n-2+1]])]];
];
];

CalculateLegendre[];
Print[poly];
Print[poly/.x->0]

It can be debugged with the built-in debugger. To use the built-in debugger on this code, you must:

•  Switch on the debugger from the evaluation menu.

•  Execute the first cell, which is essentially just definitions.

•  Select the first step in the second cell (CalculateLegendre[]) and use the palette to set this as a break point.

•  Execute the second cell so that the debugger stops on the initial break point and then use the Step in button (say) to advance the program through the code (or maybe set other breakpoints and let the program run).

This process is easy to get wrong – for example, perhaps the user simply executes the two cells (without setting an initial break location), and complains that the debugger has done nothing! Furthermore, 'real' programs often contain hundreds of cells, containing a jumble of evaluations and definitions.

Who needs a debugger?

When experienced Mathematica programmers write their own programs, they may have little need of a debugger. However, inevitably some users are beginners, and others only use Mathematica occasionally, never learning the most effective ways to code. It is important to help such people get their code to work.

Another group of potential debugger users are consultants (and others) who find themselves presented with large notebooks – often 1000 lines or more developed by a client. Understanding someone else's code can sometimes be a daunting task! For this reason, DebugTrace provides an easy robust interface for beginners, but also contains features that help in understanding and debugging much larger programs.

Introducing DebugTrace

•  DebugTrace is a (free) package that comes with its own palette. It does not use the built-in debugger, which should be turned off when DebugTrace is used.

•  DebugTrace works primarily on whole notebooks or package files. This avoids confusion for beginners – who often consider a notebook and a program to be the same thing – it caters nicely for debugging large collections of structure-less code stored in a notebook and scattered across many cells.

•  DebugTrace is based around the idea of line numbers. Stepping, break point setting, trace-back information, etc. trace-back relate to the line number on which a particular expression starts. One immediate consequence of this, is that it is possible to create a permanent record of where the debugger stopped, and what happened. As a result, well laid out code is easier to debug than code written in the "one-liner" style!

•  Starting from the notebook you wish to debug, you press the "Debug Notebook" or "Debug Selection" button to initiate the process. For convenience, these buttons also ensure the DebugTrace package is loaded, using Needs. A copy of the user's notebook (or the relevant selection) is then created with line numbers attached. These numbers also act as buttons, and clicking on them turns them into breakpoints where this is possible. A further notebook is created to contain both the stepping information, and the results as the program runs. This notebook can, of course, be saved as a permanent record of the debugging session. Both these generated notebooks are given a distinctive non-scrolling header, so that these should never get confused with user notebooks.

•  Typically the user then proceeds to debug the code using the other buttons to single step, perform a number of steps (with or without intermediate print out) , run to a break point, or a diagnostic message, or run to a specific step number (see below).

Accessing local variables

Suppose we break the above program at line 7, and wish to know the value of 'n'. The problem is that the variable 'n' will be represented by a temporary variable with a name such as n$456. This is tedious to work out for a one-off examination, but even more troublesome if you wish to set a conditional break point – breaking as soon as n==9 (say). DebugTrace contains a function to resolve a variable to the most recent local of that name:

DTLocalValue[n]

This function also reports the value of pattern variables to functions (which are also displayed in the same way by the FrontEnd syntax colourer).

Sometimes it is more convenient to see the temporary names assigned to local symbols – particularly when more than one local symbol with the same name is in use. For example:

DTLocalSymbols[tt]

Tackling larger problems

•  DebugTrace is designed to perform realistic debugging tasks on large codes. In particular, a complex application is rarely debugged in one shot, and it is vital to be able to use information gleaned from one run, in a subsequent run. This is why each step is numbered – whether the debugger stops there or not. (A given line in a loop will have one line number, but a different step number each time it is reached). These step numbers correspond from run to run (unless, of course the computational environment changes - source code, associated files, relevant random number generators, etc.), and if, say, you establish that whatever goes wrong happens after step 15231, then on the next run of the debugger, you can run directly to this step and work from there.

•  Step numbers also make it possible to home in on problems that crash or hang the kernel using a binary chop approach. Suppose you establish that if you run the program till step 5000, it never gets there, but you can run it to step 3000. The next step might be to run the program to step 4000, and continue halving the interval until you establish exactly where the program fails. Note that this is not really possible using the built-in debugger, because there is no concept of step numbering. Here is an artificial example of the kind of problem that can be tackled in this way:

j=1;
While[j < 1000000,
If[j==90000,Quit[]];
j++;
];

Here the loop is supposed to represent a complicated process that will run for many steps, and the Quit operation is supposed to represent a step that crashes or hangs the kernel for some reason. You can use a rough binary chop approach with DebugTrace to determine the point at which this code is about to destroy the kernel. Using the built-in debugger, it would be necessary to step through the loop 90000 times to discover this information! Thus we find:

•  300000 steps fail to complete

• 150000 steps complete with j=50000

• 230000 steps complete with j=76666

• 270000 steps complete with j=90000

• 285000 steps fail to complete

• 275000 steps fail to complete

• Continue. until process converges.

Note that although the DebugTrace control panel shows the step number, this value cannot be updated at every step for efficiency reasons, so in the event of a kernel crash, the displayed value will typically underestimate the point of failure.

Debugging dynamic constructs

Prior to version 6.0, almost all Mathematica calculations were initiated by the user executing one or more cells, or an entire notebook. Now we also have dynamic calculations – initiated by end-user interaction with a GUI object constructed previously. A notebook can come to life as soon as it is loaded, if it contains dynamic constructions. Users constructing these objects, are probably more in need of a debugger than anyone – yet as far as I can tell, the built-in debugger does not cater for their needs at all.

Dynamic constructions work using a rapid interchange between the front end and the kernel – an interchange that cannot be safely interrupted without generating a variety of error messages, or even hanging Mathematica. Therefore when executing code dynamically, DebugTrace treats break points as requests to store information, rather than requests to pause and enter a dialog with the user.

Simple dynamic constructs are often built using Manipulate. In most cases, such programs are best debugged by testing the functions being manipulated in a non-dynamic context, so we will consider instead a lower level Dynamic construction that illustrates a well known theorem in Euclidean geometry. Here is the correct version:

Clear[fixOnCircle1];
SetAttributes[fixOnCircle1,HoldRest];
fixOnCircle1[newpos_,variable_]:=Module[{r=Sqrt[Dot[pt1,pt1]]},
variable=newpos/r;
];


pt1={0,1};
Dynamic[Graphics[{
Circle[{0,0},1],Line[{{-1,0},{1,0}}],Red,Line[{pt1,{-1,0}}],Line[{pt1,{1,0}}],
Locator[Dynamic[pt1,fixOnCircle1]]
},PlotRange->{{-1.2,1.2},{-1.2,1.2}}]
]

Here is a faulty version of the same construction, in which the Sqrt has been omitted.

Clear[fixOnCircle];
SetAttributes[fixOnCircle,HoldRest];
fixOnCircle[newpos_,variable_]:=Module[{r=Dot[pt,pt]},
variable=newpos/r;
];


pt={0,1};
Dynamic[Graphics[{
Circle[{0,0},1],Line[{{-1,0},{1,0}}],Red,Line[{pt,{-1,0}}],Line[{pt,{1,0}}],
Locator[Dynamic[pt,fixOnCircle]]
},PlotRange->{{-1.2,1.2},{-1.2,1.2}}]
]

It is instructive to explore the flawed example, because it shows how difficult it can be to understand a bug in dynamic code – notice how the locator appears to oscillate when it is moved, and can be dragged off the circle. To explore what is going on  we select the relevant cells, and run DebugTrace on it. Pressing "Debug Selection" followed by "Run until Break". The code that prepares the interactive diagram, runs to completion – there are no breaks.

We now Shift-Click on line 4, which activates line 4 as a break point, and brings up a dialog box to enable us to enter an expression to be evaluated each time this line is reached. Obviously, we would like to see the value of newpos, but since this is a pattern variable, we enter the following into the dialog box:

DTLocalValue[newpos]

We now press the "Record N dynamics" button, to store information relating to line 4 (we could, of course, have collected data from several locations in the program).

Now when we move the Locator, N (default 100) pieces of break point data are recorded. This data can be displayed thus:

DTDynamicResults[]

Looking at the actual numbers, it is much easier to determine what has gone wrong.

Although not really relevant in this context, the break point information also contains timing information. This can be useful in certain situations in which a dynamic construct is responding in a slow or erratic fashion. The step numbers are counted from the time the "Record N dynamics" button was pressed.

The reason why the dynamic recording is limited to N items (but N could be set to a very large number), is that formatting this information consumes enough time to force some updates to a crawl, and possibly cause dynamic processes to run out of their allotted time.

DebugTrace internals

•  The built-in debugger does not come with a published API to control it, and it not clear in any case that this would have been useful to create DebugTrace, because the two debuggers operate in substantially different ways. Therefore DebugTrace works entirely using the standard Mathematica command set, making no use of the built-in debugger.

•  When DebugTrace is activated, it executes a NotebookGet (or NotebookRead, if debugging a selection) to obtain the code in BoxForm. This information contains the newline characters that mark successive lines (as opposed to lines that are merely broken by the layout mechanism to fit the window size). This information is massaged to generate the (read only) 'listing notebook' containing the break point buttons. It is also split into a list of BoxForms representing successive cells (more precisely, executable expressions, since some cells contain multiple inputs, but I will refer to 'cells' in what follows). The line number information is encoded within these structures. The line corresponding to the start of each cell is activated as a potential break point (coloured blue), but additional breakpoints within each cell cannot be determined at this stage. This is because the BoxForm represents symbol names as strings, and these can only be converted to symbols in the appropriate context. The context required to translate cell N, can only be determined after the previous N-1 cells have been executed – think of notebooks containing Needs[...], or notebooks containing a whole package.

•  A large listing notebook can contain hundreds or even thousands of line numbers – each of which is a button. I discovered the hard way that dynamic constructs inside these buttons overload the FrontEnd/Kernel interface and tend to result in hangs and crashes of various sorts. For this reason the button colours are updated using pre-version 6 methods!

•  When the user steps the  program to the point where cell N is to be executed, the annotated BoxForm structure described above, is converted into a held expression, which is recursively manipulated to identify many common constructions – such as CompoundExpression, If, Module, While, For, etc.. Extra function calls, invoking DebugTrace routines with the relevant line-number information are added to the code, before the Hold is finally removed, to permit actual execution of the modified code. Thus, in summary, DebugTrace operates by transforming the program into a form which contains extra function calls that allow its progress to be tracked. For example:

({{({{CalculateLegendre[]:=DebugTrace`Private`Function˘Tag[Module[{n},DebugTrace`Private`Debug˘Tag[2,"In Module with local variables ","ExtraInfo"->"HoldForm"[{n}]];(DebugTrace`Private`Debug˘Tag[3,""];(poly[[0+1]]=1;(DebugTrace`Private`Debug˘Tag[4,""];poly[[1+1]]=x);For[n=2,n≤10,n++,DebugTrace`Private`Debug˘Tag[5,"Executing body of For"];(DebugTrace`Private`Debug˘Tag[7,""];(poly[[n+1]]=Expand[Simplify[(((2 n-1) x poly[[n-1+1]]-(n-1) poly[[n-2+1]])/n)]];))];))],"CalculateLegendre",2]}})}})

Conclusions

The task of debugging Mathematica code – actually making it work – is a very important one. The built-in debugger certainly does not represent the last work in tools for this purpose, because it is tricky to operate correctly, and does not cater well for larger problems. The DebugTrace package should certainly be useful in itself, and will hopefully stimulate further design effort to create the ideal debugger.