CleanSlate.m, version 1.1, Package Documentation.

This document supplements the explanation and comments in the 
CleanSlate.m file itself (all of that information is excerpted from 
this more complete discussion). The information is provided for the 
interested reader. It is certainly not necessary to understand, or even 
read, this file to use CleanSlate effectively.

PURPOSE

The purpose of CleanSlate is to provide an easy and complete way to 
accomplish two goals: 1) free memory, and 2) clear values of symbols, 
so that you need not worry about tripping over some preexisting 
definition for a symbol. The basic command exported from the package, 
CleanSlate[], tries to do everything possible to return the kernel to 
the state it was in when the CleanSlate.m package was initially read in 
(usually, this is at the end of the startup process, but, as discussed 
below, it can be read in at other times as well). Of course, short of 
actually restarting, there is no way to do this, but I hope that 
CleanSlate comes as close as possible. I think it will be adequate for 
most user's needs. 

BRIEF SUMMARY

There are 3 functions exported from the package: CleanSlate, 
CleanSlateExcept, and ClearInOut.

ClearInOut[] simply clears the In[] and Out[] values, and resets the 
$Line number to 1 (so new input begins as In[1]). It is called 
internally by CleanSlate and CleanSlateExcept. Once this function has 
been executed, you can no longer refer to older input or output (if 
ClearInOut[] is executed as In[32], then you cannot refer to %30, for 
example). It does not affect the values of any symbols, though, so it 
is a relatively "nondestructive" attempt to free memory. By itself, it 
usually results in only a minimal recovery of memory, but in some cases 
(e.g., graphics) the savings can be large.

CleanSlate and CleanSlateExcept share the same basic purging engine 
(the private function CleanSlateEngine), differing only in the way they 
calculate which contexts to send to this engine. These functions will 
be discussed in much greater detail below, but their basic use is as 
follows. CleanSlate[] tries to purge everything that has happened since 
the CleanSlate package was read in. You can also specify specific 
contexts for purging with CleanSlate["Context1`","Context2`", ...]. 
Only the listed contexts, along with all of their subcontexts, will be 
affected. Thus, if you don't specify a context or contexts, CleanSlate 
will assume you want the complete job. 
CleanSlateExcept["Context1`","Context2`", ...] allows you to specify a 
set of contexts to be spared from purging. Everything other than what 
you list will be purged. At the end of the process, the functions print 
a list of the contexts purged and the approximate amount of memory 
freed. The return value is the new $ContextPath.

CleanSlate and CleanSlateExcept take one option, Verbose, which can be
set to True or False. The default is Verbose->True, which specifies that
they print their usual diagnostic messages.

CleanSlate and CleanSlateExcept have some basic error-checking code 
built into them, to prevent incorrect use. In particular, they catch 
any invalid parameters (such as a misspelled context). For consistency, 
they take their input in the same form as the Mathematica functions that
take contexts as parameters (Begin and BeginPackage): a sequence (not a 
list) of strings, each specifying a context name.

It should be noted that CleanSlate is sometimes not very effective at 
freeing memory. CleanSlate can free memory used for storing values and 
definitions for symbols, but not memory used internally by Mathematica 
for calculations (which is the responsibility of Mathematica's internal
memory-management mechanisms). If you read in a huge package that takes
up a megabyte of memory, though, you can usually recover virtually all
of it with CleanSlate. 

CleanSlate and Share: Mathematica version 2.1 has a command, Share[],
which can free significant amounts of memory. Share and CleanSlate do
not conflict, and in fact they are ideally used together. Run Share
after CleanSlate to produce the maximum recovery of memory. Share
generally executes much more slowly than CleanSlate, however, so you
might not want to use it routinely.


EXAMPLES

  CleanSlate[]                       -- The works
  CleanSlate["Packagename`"]         -- Just strip out this one context
  CleanSlate["Global`"]              -- Get rid of user-defined stuff,
                                          but leave all packages
  CleanSlateExcept["Global`"]        -- Leave all user-defined stuff,
                                          but get rid of all packages
  CleanSlateExcept["Packagename`"]   -- Get rid of everything but this
                                          one package
  CleanSlate[Verbose->False]         -- Same as CleanSlate[], but don't
                                          print diagnostic output
  ClearInOut[]                       -- Just clear In and Out values

WHAT IS MEANT BY "PURGING"?

Essentially, "purging" means wiping out all trace of the context's 
existence. This is basically a 3-step process. Step 1 is to map 
Unprotect, ClearAll, and Remove over all symbols in the context (and 
any subcontexts). Step 2 is to try to remove any rules that the context 
may have defined for System symbols. Step 3 is to remove the context 
from $ContextPath and $Packages (if it is present in $Packages). The 
Global` context, however, is not removed from $ContextPath or 
$Packages.

Some packages "overload" System functions (i.e., those in the System` 
context) with additional rules. A good example is the package 
Algebra`ReIm`, which adds new rules for Re, Im, Abs, Conjugate, and 
Args. To effectively remove this package, we would need to remove these 
additional rules as well. CleanSlate uses a clever (I think) scheme 
that enables it to strip out rules a package adds for System functions. 
This will be discussed below, but the basic mechanism involves 
substituting my own function for Unprotect. In this way, we can 
intercept all attempts to Unprotect system symbols (a necessary prelude 
to adding rules), noting which symbols are being unprotected and which 
context is doing it. After this information has been recorded, the 
built-in Unprotect is called. This "patching" technique will be 
familiar to programmers.

HOW TO USE IT

CleanSlate.m is designed to be read in at the end of the startup 
process. This is best accomplished by putting it as the last thing in 
the file init.m. The code can be simply pasted into this file, or you 
can just put <<CleanSlate.m into the file. The point at which 
CleanSlate.m is read in is fundamentally important to its operation. In 
effect, it takes a snapshot of the system at this point, so that later 
when the function is executed, it will not affect anything that was 
initially present. This snapshot consists of two things: the contexts 
on $ContextPath, and any symbols already defined in the Global` 
context. When you execute CleanSlate[], for example, all contexts on 
the current $ContextPath that were not on the initial $ContextPath will 
be purged (along with all of their subcontexts). The exception is the 
Global` context; it is present initially, but it will be purged anyway. 
However, any symbols that were originally defined in the Global` 
context will be spared. There is no way to purge a context that was on 
the initial path (except Global`) -- any attempts to specify such a 
context will generate an error message.

Although CleanSlate.m is intended to be read in at the start of a 
session, it can actually be read in at any time during a session, or 
even more than once during a session. It will always respect the 
contexts on the $ContextPath at that time, as well as all symbols in 
the Global` context at that time. This allows you to effectively set a 
"mark" at any point in the session, so that executing CleanSlate[] will 
take you back to that point.

HISTORY

CleanSlate was originally written for my own use in the spring of 1991. 
It has been enormously elaborated since then, but it retains the same 
basic core purging code (about 10 lines' worth). I had hopes to 
distribute it back then, but I knew it was not robust and I despaired 
at the complexity of the issues involved and the lack of documentation 
surrounding some low-level details. Then in spring of 1992, Courtney 
Johnson of WRI gave me a copy of a function called Restart, written by 
Matthew Markert. Restart was remarkably similar to CleanSlate, although 
obviously an independent creation. Inspired by the discovery that 
someone else had hit on the same basic method (and the implication that 
there probably wasn't a better way), I began improving CleanSlate.

BACKGROUND ON CONTEXTS AND PACKAGES

To understand the CleanSlate code, you need a basic understanding of 
contexts and packages in general, and the System symbols $ContextPath 
and $Packages in particular. (An excellent reference is the invaluable 
Programming in Mathematica, by Roman Maeder.) When Mathematica
encounters a symbol, it of course checks to see if it has already been
defined. Where does it look? First, it looks in the present context; if
it is not found there, it looks in all the contexts listed in
$ContextPath; if it is not found anywhere in there, it is created as a
new symbol in the present context. Thus, the $ContextPath controls the
visibility of symbols. The System` context is always on the $ContextPath
(otherwise Mathematica wouldn't be able to find its own built-in
functions!) The Global` context, which is generally the current context
when the user is working, is usually on the $ContextPath (except after a
BeginPackage statement; see below). Additionally, there may be many
other contexts on the $ContextPath, corresponding to packages that have
been read in by the user, or by the startup process.

One of the first statements in any package is 
BeginPackage["Packagename`"]. This has the effect of initiating a new 
context with name Packagename`, and causing the $ContextPath to be 
shortened to just {"Packagename`", "System`"}. Thus, there is no danger 
that a symbol name you use in a package will be mistaken for a 
previously existing symbol (because all other contexts are no longer 
visible). In addition, "Packagename`" is added to the list $Packages, 
which is simply a list of all packages that heve been loaded. The 
EndPackage[] statement returns you to the context you were in before 
BeginPackage[] (most likely Global`) and returns $ContextPath to its 
former value, but with "Packagename`" prepended to it. Therefore, the 
symbols defined in the Packagename` context will be visible to the user 
for the duration of the session. Thus, by looking at the current 
$ContextPath and comparing it to the $ContextPath when CleanSlate.m was 
read in, we can see what contexts have been added and are fair game for 
purging.

After clearing the symbols in a context, we then want to remove the 
name of the context from $ContextPath. We also need to remove it from 
$Packages (if it is a package name), because the command Needs looks in 
this list to see if it is necessary to load a given package.

HOW DOES IT WORK?

In what follows, I will give a brief walk-through of the code. I will 
touch on only the main features. If you have any questions or comments 
about anything you find, please contact me. I would be more than happy 
to discuss specific points in more detail. I hope the correspondence 
between this text and the relevant lines of code is clear.

As noted above, when Cleanslate.m is read in, it needs to record two 
things: the current $ContextPath, and the names of any symbols already 
defined in the Global` context. Both of these pieces of info need to be 
obtained before the BeginPackage command, however. This is because 
BeginPackage changes $ContextPath and it makes the Global` context 
invisible. Thus, we create two symbols in the System` context to hold 
this info, System`startupPath and System`startupGlobals. Once inside 
the package's `Private` context, we can transfer this info to two 
"static" variables, initialPath and initialGlobals, and then Remove the 
two System` symbols.

The next section is devoted to the modified Unprotect function. The 
list alteredSystemSymbols is the data structure that will accumulate 
information about who is modifying what System functions. It will have 
the following structure: {{"Guiltycontext1`", {Hold[symbol1], 
Hold[Symbol2]}}, {"Guiltycontext2`", {Hold[symbol3]}},...}. We only add 
System symbols to the list if the "unprotecting context" is not 
System`, for reasons I won't go into. There are several interesting 
points in the code. We have to call the built-in Unprotect function 
inside our own Unprotect. To accomplish this, we need to change the 
DownValues of Unprotect from within Unprotect itself. This is 
complicated by the fact that we need to Unprotect Unprotect itself to 
do any of this. The modified Unprotect function here will not intercept
calls to Unprotect where the argument is a string, so that
Unprotect["Unprotect"] will not trigger the patched code. I use this
form for Unprotect throughout the CleanSlate package. String arguments
to Unprotect are rarely, if ever, used in packages (never in the
standard packages), so this method does not compromise CleanSlate's
operation. This is an important fact to remember for advanced users of
CleanSlate, and can come in handy if for some reason you want to avoid
CleanSlate's watchful eye on system symbols. The idea of "patching" a
System symbol with your own code that calls the original definition
inside it is a programming technique with many interesting
possibilities, and the Unprotect function here can serve as a basic
model.

Another complication is that we need to prevent the symbols passed to
Unprotect from being evaluated at any time inside our function. Thus, we
have to wrap some things in Hold, and the pure function needs to have
the attribute HoldAll.

Next is the straightforward function ClearInOut[], which is really just 
inline code. This function was discussed above. It is called by 
CleanSlate and CleanSlateExcept.

The two main functions, CleanSlate and CleanSlateExcept, are next. They 
both work the same way: based on what parameters they are given, they 
compute the value of contextsToPurge, which is the list of contexts 
that will be sent to CleanSlateEngine for purging. By the method with 
which contextsToPurge is calculated, and by later error-checking, the 
following two constraints are enforced on contextsToPurge:
1) it can only consist of contexts on $ContextPath.
2) it can never contain the contexts System`, CleanSlate`, or anything 
that was on the initial $ContextPath (other than Global`).

The function ErrorChecking is shared by both CleanSlate and 
CleanSlateExcept. There are two types of errors that are trapped here. 
First, every element of the list contextsToPurge must be a member of 
$ContextPath. Second, the current $ContextPath must completely contain 
the initial $ContextPath (from when CleanSlate was read in). The most 
likely reason for this to fail is if CleanSlate is being called from 
within another package (between BeginPackage..EndPackage pairs). It is 
also possible that something has been manipulating the $ContextPath 
directly. CleanSlate will not work as desired if either of these 
conditions are not met.

CleanSlateEngine, a private function, is the real meat of the process. 
First, ClearInOut is called. The next few lines are the main purging
code. They are straightforward, if a bit difficult to read. We simply
map Unprotect, ClearAll, and Remove over every symbol in every context
in contextsToPurge (and their subcontexts). The Global` context has to
be done separately, because we need to preserve all symbols that were 
present in this context when CleanSlate.m was initially read in.

The next part is where we try to remove any extra rules defined for 
System symbols. Before I walk through the code in this section, I 
should explain the basic method used. Let's say you read in the 
following package:

BeginPackage["Foo`"];
Unprotect[Power];
Power[x_] := garbage;
Protect[Power];
EndPackage[];

The modified Unprotect function will have recorded that the context 
Foo` unprotected the System symbol Power.

Now let's say you want to see this new rule. You could type ??Power, or 
you could use

In[]:=  DownValues[Power]
Out[]=  {Literal[Power[x_]] :> garbage}

Now let's say you do a Remove["Foo`*"], which is part of the CleanSlate 
process, then look at Power again:

In[]:=  DownValues[Power]
Out[]=  {Literal[Power[Removed[x]_]] :> Removed[garbage]}

We see that Removed appears as a head of any symbol in 
the rule. Thus, its existence can serve as a flag to indicate that this 
particular rule was defined in a context which has now been purged. All 
we have to do is delete all DownValues (and UpValues) for Power in 
which the head Removed appears on the left-hand side (we could also 
check the right-hand side, but this will rarely catch any additional 
cases). It turns out that Removed is not an ordinary head, and we can't 
detect its presence with, for example, FreeQ[#,Removed]&. But we can 
convert the LHS of the rule to a string and use string-matching to look 
for "Removed[*".

There is one type of rule that will not be detected by this method: 
rules that do not have a named pattern or variable on the LHS, for 
example Power[_] := garbage. But it is relatively unlikely that any 
packages will be overloading System functions in this way. If you are 
writing a package and you want to be sure that CleanSlate will find all 
the System rules you define, just name all patterns even if the RHS 
doesn't use the name.

Now, back to the code itself. First, we accumulate a list, called 
systemSymbolsToCheck, of all System symbols that will need to be 
checked for removable rules, based on what contexts are being purged. 
We also want to remove the records from alteredSystemSymbols 
corresponding to purged contexts and their affected symbols. The 
systemSymbolsToCheck is accumulated as a side-effect of deleting 
records from alteredSystemSymbols. 

Then, we scan a function over systemSymbolsToCheck which performs the 
actual deleting of UpValues, DownValues, SubValues, and FormatValues. 
Again, we need to prevent evaluation of any of the symbols, so we give 
the pure function the attribute HoldAll.

Next, we print out some useful information if the Verbose option is not
set to False: the actual (top-level) contexts purged, and the
approximate amount of memory recovered. 

Finally, we need to adjust $ContextPath and $Packages to reflect the 
removed contexts. Note that we cannot use Complement here, as we have 
done before with lists of contexts, because this would result in a 
reordering of the lists. We also restore the modified Unprotect 
function. The return value is the new $ContextPath.

MODIFYING CLEANSLATE

I have not had time to go into all the details of CleanSlate's design 
and operation. If anyone has more specialized needs, or wants to modify
the code in some way, I would be happy to help. However, there are some
strange things in CleanSlate that are there for a reason. I don't 
recommend changing anything in the code, or even cannibalizing portions 
of it.

BUGS, LIMITATIONS, AND ADVANCED ISSUES

1) There is a bug in some versions of the package ComplexExpand.m
(Mathematica version 2.1 only) that prevents CleanSlate from working in
"thin kernel" systems (such as Mac and Windows) once the ComplexExpand
package has been read in. This is an easy bug to fix, and you will want
to fix it, whether or not it affects your use of CleanSlate, so read on.
   The ComplexExpand.m package is read in "invisibly" by many other
packages, so you may not notice it happening.(for example, when the
integration packages are loaded). It also deletes itself (incorrectly)
from $ContextPath and $Packages, which is where the problem lies.
After ComplexExpand.m has been read, if you execute one of the
CleanSlate functions, it will abort with a CleanSlate::cntxtpth 
error.
   The bug is a simple typographical mistake, and you should fix it
right now if it afflicts your system. Here's how: open the
ComplexExpand.m file with a text editor (you can use Mathematica itself
if you have a Mac or Windows PC). Near the end of the file you may find
a line that looks like:

   $ContextPath = DeleteCases[$Packages, "System`ComplexExpand`"]

Oops! $ContextPath is being restored from $Packages! Change it to read:

   $ContextPath = DeleteCases[$ContextPath, "System`ComplexExpand`"]

That's it.

2) In its printout of memory recovered, CleanSlate fails to count the 
memory occupied by the most recent Out[] value. This memory _is_ freed, 
but it does not show up in the amount CleanSlate prints. This means 
that if you execute CleanSlate immediately after generating a huge 
graphic, you will not see its contribution in the memory reported 
freed. 

3) Contexts begun with Begin["context`"] and ended with End[] are not 
put on the $ContextPath. Thus, CleanSlate will never purge such 
contexts (unless they are subcontexts of a context that is on 
$ContextPath, as is the case with subcontexts in packages). If you are 
in the habit of creating contexts while you work, and you want 
CleanSlate to be able to purge them, make them subcontexts of Global`, 
as with Begin["`context`"]. Alternatively, use EndAdd[], which puts the 
context on $ContextPath.

4) CleanSlate will not be able to remove rules added for System 
functions unless the LHS of the rule has a named pattern or variable. 
For example, Re[x_Integer] := x will be stripped out, but Re[_] := 
Print["attempting to call Re"] will not. This should rarely be a 
problem since you will almost always want to use the pattern on the RHS 
and so will need to name it. You can always go into a package and give 
arbitrary names to such unnamed pattern variables to avoid this 
problem.

5) If a package modifies properties of System symbols other than 
UpValues, DownValues, SubValues or FormatValues (e.g., Attributes, 
Options, Messages), these changes will not be undone by CleanSlate.

6) The modified Unprotect function will not intercept calls to 
Unprotect when the parameter is a string, for example 
Unprotect["System`*"] to unprotect every System symbol. This should not 
be a problem because this syntax is probably never used by package 
writers. Its only use is to pass a string pattern, as in this example, 
but few packages will be unprotecting System symbols in such a broad 
way.

7) CleanSlate will abort if called from within another package. This is 
a design decision relating to assumptions CleanSlate makes about the 
$ContextPath.

8) On machines that use a so-called "thin system" setup, not all System 
functions are loaded during startup. Rather, they are "stubbed" in in 
such a way that the first reference to the function causes the relevant 
file to be read in. Some packages refer to stubbed System functions, 
causing these additional files to be read in, usually unbeknownst to 
the user. The symbols are put in the System` context, though, and 
nothing is added to $ContextPath. CleanSlate cannot take out these 
definitions or free the memory they occupy.

9) When you execute CleanSlate, it clears its record of modified system
symbols. This means that until you Unprotect a symbol again, subsequent
calls to CleanSlate will not wipe out newly added rules for that symbol.
For example, you might Unprotect a system symbol like Power, add some
rules for it, and leave it unprotected. Later you execute CleanSlate,
then add some new rules for Power. But since you left Power
unprotected, you don't bother to Unprotect it again. Now if you execute
CleanSlate again, it will not wipe out the new rules. You must
explicitly call Unprotect[Power] to re-register it with
CleanSlate's list of modified system symbols. 

10) The discussion so far has dealt with kernel memory. Memory used by
the notebook front end (on machines where it is available) is a separate
issue that has nothing to do with CleanSlate. If you generate a
complicated plot, the kernel memory required to hold the Graphics object
can be freed with CleanSlate, but the front end may still require a lot
of memory to hold the PostScript code and bitmapped representation.

11) CleanSlate cannot do anything for symbols with the attribute Locked.
It it best to not use CleanSlate on a context that locks any symbols.
None of the standard packages lock any symbols

-------------------------------------------
November, 1992 (last modified May 25, 1993)
Todd Gayley
internet: tgayley@wri.com
compuserve: 72560,1165