(************** Content-type: application/mathematica **************
Mathematica-Compatible Notebook
This notebook can be used with any Mathematica-compatible
application, such as Mathematica, MathReader or Publicon. The data
for the notebook starts with the line containing stars above.
To get the notebook into a Mathematica-compatible application, do
one of the following:
* Save the data starting with the line of stars above into a file
with a name ending in .nb, then open the file inside the
application;
* Copy the data starting with the line of stars above to the
clipboard, then use the Paste menu command inside the application.
Data for notebooks contains only printable 7-bit ASCII and can be
sent directly in email or through ftp in text mode. Newlines can be
CR, LF or CRLF (Unix, Macintosh or MS-DOS style).
NOTE: If you modify the data for this notebook not in a Mathematica-
compatible application, you must delete the line below containing
the word CacheID, otherwise Mathematica-compatible applications may
try to use invalid cache data.
For more information on notebooks and Mathematica-compatible
applications, contact Wolfram Research:
web: http://www.wolfram.com
email: info@wolfram.com
phone: +1-217-398-0700 (U.S.)
Notebook reader applications are available free of charge from
Wolfram Research.
*******************************************************************)
(*CacheID: 232*)
(*NotebookFileLineBreakTest
NotebookFileLineBreakTest*)
(*NotebookOptionsPosition[ 36771, 1139]*)
(*NotebookOutlinePosition[ 38851, 1208]*)
(* CellTagsIndexPosition[ 38597, 1195]*)
(*WindowFrame->Normal*)
Notebook[{
Cell[CellGroupData[{
Cell[TextData[{
"Numerical Optimization in ",
StyleBox["Mathematica",
FontSlant->"Italic"],
": an insider's view of NMinimize"
}], "Title"],
Cell["Brett Champion", "Author"],
Cell["Wolfram Research, Inc.", "Address"],
Cell[TextData[{
StyleBox["NumericalMath`NMinimize", "MR"],
" is a new standard add-on package for finding solutions to constrained and \
unconstrained global optimization problems. The package provides several \
derivative\[Hyphen]free methods for finding global minima. This allows us to \
solve problems where the objective function is not differentiable (or even \
continuous). The methods are sufficiently robust to find global minima \
insofar as they are not readily trapped by local minima. We will discuss the \
user interface to ",
StyleBox["NMinimize", "MR"],
" and also provide details of the methods and technologies utilized. \
Technical issues such as how initial regions are chosen, integer variables \
handled, and the like will also be covered."
}], "Abstract",
TextAlignment->Left,
TextJustification->1],
Cell[CellGroupData[{
Cell["A Quick Introduction to NMinimize", "SectionFirst"],
Cell[TextData[{
StyleBox["NMinimize", "MR"],
" is a new function in ",
StyleBox["Mathematica",
FontSlant->"Italic"],
" 4.2 for solving a wide range of global optimization problems. In order to \
use this function, we need first to load one of the Standard AddOn Packages:"
}], "Text",
TextAlignment->Left,
TextJustification->1],
Cell[BoxData[
\(<< NumericalMath`NMinimize`\)], "NumberedEquation"],
Cell["The basic syntax for unconstrained problems is", "Text",
TextAlignment->Left,
TextJustification->1],
Cell[BoxData[
\(NMinimize[f, \ vars]\)], "NumberedEquation"],
Cell[TextData[{
"where ",
StyleBox["f",
FontSlant->"Italic"],
" is the function to minimized, and ",
StyleBox["vars",
FontSlant->"Italic"],
" is a list of variables, possibly specifying initial search regions. For \
constrained problems, we have the extended syntax"
}], "Text"],
Cell[BoxData[
\(NMinimize[{f, \ cons}, \ vars]\)], "NumberedEquation"],
Cell[TextData[{
"where ",
StyleBox["cons",
FontSlant->"Italic"],
" is the set of constraints represented as either a list or a logical \
combination of inequalities, equalities, and variable domains. We can also \
specify different algorithms for minimization, most of which treat the \
objective function as a black box. Each of the methods has various suboptions \
that allow fine tuning of the algorithms. The syntax for specifying methods \
and method options is, respectively"
}], "Text"],
Cell[BoxData[
\(NMinimize[f, \ vars, \ Method \[Rule] method]\)], "NumberedEquation"],
Cell[BoxData[
\(NMinimize[f, \ vars, \
Method \[Rule] {method, \ opt\_1, \ opt\_2 ... }]\)], "NumberedEquation"],
Cell[TextData[{
"where ",
StyleBox["method",
FontSlant->"Italic"],
" is an acceptable method, and the ",
Cell[BoxData[
\(TraditionalForm\`opt\_i\)],
FontSlant->"Italic"],
"are method specific options (those in ",
StyleBox["Options[", "MR"],
StyleBox["method", "TT",
FontSlant->"Italic"],
StyleBox["]", "MR"],
"). Normal ",
StyleBox["NMinimize", "MR"],
" options (those in ",
StyleBox["Options[NMinimize]", "MR"],
") are specified at the same level as the ",
StyleBox["Method", "MR"],
" option. For more information on ",
StyleBox["NMinimize", "MR"],
" syntax, and examples using ",
StyleBox["NMinimize", "MR"],
", please see the ",
StyleBox["NMinimize", "MR"],
" entry in the ",
StyleBox["Mathematica",
FontSlant->"Italic"],
" Help Browser."
}], "Text"],
Cell[TextData[{
"It should be noted that this paper discusses very technical aspects of the \
implementation of ",
StyleBox["NMinimize", "MR"],
" in ",
StyleBox["Mathematica",
FontSlant->"Italic"],
" 4.2, and some of the implementation details or interfaces may change in \
future versions of ",
StyleBox["Mathematica",
FontSlant->"Italic"],
"."
}], "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell["Methods", "Section"],
Cell[CellGroupData[{
Cell["DifferentialEvolution", "Subsection",
CellDingbat->None],
Cell[TextData[{
StyleBox["DifferentialEvolution", "MR"],
" is a genetic algorithm developed by K. Price and R. Storn [1]. This \
method maintains a population of specimens ",
Cell[BoxData[
\(TraditionalForm\`x\_1, \[Ellipsis], x\_n\)]],
", represented as vectors of ",
StyleBox["d",
FontSlant->"Italic"],
"\[Hyphen]dimensional real numbers (\[OpenCurlyDoubleQuote]genes\
\[CloseCurlyDoubleQuote]), where ",
StyleBox["d",
FontSlant->"Italic"],
" is the number of variables. Every iteration, for each vector ",
Cell[BoxData[
\(TraditionalForm\`x\_i\)]],
", it chooses random integers ",
Cell[BoxData[
\(TraditionalForm\`1 \[LessEqual] a, b, c \[LessEqual] n\)]],
" and constructs the mate ",
Cell[BoxData[
\(TraditionalForm\`y\_i =
x\_i + \[Gamma](x\_a + \((x\_b - x\_c)\))\)]],
", where ",
Cell[BoxData[
\(TraditionalForm\`\[Gamma]\)]],
" is the value of ",
StyleBox["ScalingFactor", "MR"],
". Then ",
Cell[BoxData[
\(TraditionalForm\`x\_i\)]],
" is mated with ",
Cell[BoxData[
\(TraditionalForm\`y\_i\)]],
" according to the value of ",
StyleBox["CrossProbability", "MR"],
", giving us the child ",
Cell[BoxData[
\(TraditionalForm\`z\_i\)]],
". At this point ",
Cell[BoxData[
\(TraditionalForm\`x\_i\)]],
" competes against ",
Cell[BoxData[
\(TraditionalForm\`z\_i\)]],
" for position ",
Cell[BoxData[
\(TraditionalForm\`i\)]],
" in the population. The default value of ",
StyleBox["SearchPoints", "MR"],
" is ",
StyleBox["Automatic", "MR"],
", which is ",
StyleBox["Min[10*d,", "MR"],
" ",
StyleBox["50]", "MR"],
"."
}], "Text",
CellTags->{"S5.90.1", "1.24"}]
}, Open ]],
Cell[CellGroupData[{
Cell["NelderMead", "Subsection",
CellDingbat->None],
Cell[TextData[{
"NelderMead is an implementation of J. A. Nelder and R. Mead's simplex \
algorithm [2]. For simplicity, we assume here that minimization is being \
done. We start with the highest vertex ",
Cell[BoxData[
\(TraditionalForm\`v\_h\)]],
" of a simplex, and reflect it across the centroid ",
Cell[BoxData[
\(TraditionalForm\`v\_c\)]],
" of the remaining points to a new point ",
Cell[BoxData[
\(TraditionalForm\`v\_a\)]],
" such that ",
Cell[BoxData[
\(TraditionalForm\`\[LeftDoubleBracketingBar]v\_a -
v\_c\[RightDoubleBracketingBar]/\[LeftDoubleBracketingBar]v\_h -
v\_c\[RightDoubleBracketingBar]\)]],
" equals the ",
StyleBox["ReflectRatio", "MR"],
". If ",
Cell[BoxData[
\(TraditionalForm\`v\_a\)]],
" is lower than all other vertices, we expand reflection by finding the \
point ",
Cell[BoxData[
\(TraditionalForm\`v\_b\)]],
" such that ",
Cell[BoxData[
\(TraditionalForm\`\[LeftDoubleBracketingBar]v\_b -
v\_a\[RightDoubleBracketingBar]/\[LeftDoubleBracketingBar]v\_a -
v\_c\[RightDoubleBracketingBar]\)]],
" equals the ",
StyleBox["ExpandRatio", "MR"],
". Then the lower of ",
Cell[BoxData[
\(TraditionalForm\`v\_a\)]],
" and ",
Cell[BoxData[
\(TraditionalForm\`v\_b\)]],
" replaces ",
Cell[BoxData[
\(TraditionalForm\`v\_h\)]],
" in the simplex and the process starts over. If ",
Cell[BoxData[
\(TraditionalForm\`v\_a\)]],
" is lower than the highest vertex but not lower than the lowest vertex, ",
Cell[BoxData[
\(TraditionalForm\`v\_a\)]],
" replaces ",
Cell[BoxData[
\(TraditionalForm\`v\_h\)]],
" and the process starts over. Finally, if ",
Cell[BoxData[
\(TraditionalForm\`v\_a\)]],
" is the highest vertex, we construct ",
Cell[BoxData[
\(TraditionalForm\`v\_b\)]],
" so that ",
Cell[BoxData[
\(TraditionalForm\`\[LeftDoubleBracketingBar]v\_c -
v\_b\[RightDoubleBracketingBar]/\[LeftDoubleBracketingBar]v\_c -
v\_h\[RightDoubleBracketingBar]\)]],
" equals the ",
StyleBox["ContractRatio", "MR"],
". If ",
Cell[BoxData[
\(TraditionalForm\`v\_b\)]],
" is lower than ",
Cell[BoxData[
\(TraditionalForm\`v\_h\)]],
", we replace ",
Cell[BoxData[
\(TraditionalForm\`v\_h\)]],
" with ",
Cell[BoxData[
\(TraditionalForm\`v\_b\)]],
" and start over; otherwise, we move each of the simplex vertices ",
Cell[BoxData[
\(TraditionalForm\`v\_i\)]],
" toward the lowest vertex ",
Cell[BoxData[
\(TraditionalForm\`v\_l\)]],
", giving new vertices ",
Cell[BoxData[
\(TraditionalForm\`v\_i\%\[Prime]\)]],
" where ",
Cell[BoxData[
\(TraditionalForm\`\[LeftDoubleBracketingBar]v\_i -
v\_i\%\[Prime]\[RightDoubleBracketingBar]/\
\[LeftDoubleBracketingBar]v\_i - v\_l\[RightDoubleBracketingBar]\)]],
" matches the value of ",
StyleBox["ShrinkRatio", "MR"],
", and start over. "
}], "Text",
CellTags->{"S5.90.1", "1.30"}],
Cell[TextData[{
StyleBox["NelderMead", "MR"],
" also has a mechanism for restarting if it converges to a point that \
doesn't satisfy the constraints. In this case, it will attempt to follow \
constraint gradients to find a point that does satisfy the constraints, \
generate a new simplex around this point, and start again. Each restart has \
the full ",
StyleBox["MaxIterations", "MR"],
" available for use. The maximum number of restarts is three if ",
StyleBox["NelderMead", "MR"],
" is being used as a result of ",
StyleBox["Method\[Rule]Automatic", "MR"],
", and five if ",
StyleBox["NelderMead", "MR"],
" was specified by the ",
StyleBox["Method", "MR"],
" option."
}], "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell["RandomSearch", "Subsection",
CellDingbat->None],
Cell[TextData[{
StyleBox["RandomSearch", "MR"],
" is a simple method that generates a large set of points and uses each one \
as the starting point for ",
StyleBox["FindMinimum", "MR"],
". Since ",
StyleBox["FindMinimum", "MR"],
" doesn't allow constraints, a modified objective function is used. The \
default value of ",
StyleBox["SearchPoints", "MR"],
" is ",
StyleBox["Automatic", "MR"],
", which is ",
StyleBox["Min[100*d,", "MR"],
" ",
StyleBox["1024]", "MR"],
", where ",
Cell[BoxData[
\(TraditionalForm\`d\)]],
" is the number of variables. "
}], "Text",
CellTags->{"S5.90.1", "1.36"}]
}, Open ]],
Cell[CellGroupData[{
Cell["SimulatedAnnealing", "Subsection",
CellDingbat->None],
Cell[TextData[{
StyleBox["SimulatedAnnealing", "MR"],
" is an implementation of a biased random\[Hyphen]walk search method [3]. \
It generates a random set of starting points, and for each starting point \
picks a random direction in which to move. If the move is a better point then \
it is accepted. If the move is not a better point, we calculate a probability \
",
Cell[BoxData[
\(TraditionalForm\`\[Rho]\)]],
" and compare it against a random value ",
Cell[BoxData[
\(TraditionalForm\`r \[Element] \((0, 1)\)\)]],
", and if ",
Cell[BoxData[
\(TraditionalForm\`r < \[Rho]\)]],
" we accept the new point despite it not being an improvement. The \
probability ",
Cell[BoxData[
\(TraditionalForm\`\[Rho]\)]],
" is given by ",
Cell[BoxData[
\(TraditionalForm\`\(\(\[Rho] =
e\^\(b \(\(\[InvisibleSpace]\)\([i \(\(,\)\(\ \)\) \[CapitalDelta] f \
\(\(,\)\(\ \)\) f\_0]\)\)\)\)\(,\)\)\)]],
" where ",
Cell[BoxData[
\(TraditionalForm\`b\)]],
" is the function defined by ",
StyleBox["BoltzmannExponent", "MR"],
", ",
Cell[BoxData[
\(TraditionalForm\`i\)]],
" is the current iteration, ",
Cell[BoxData[
\(TraditionalForm\`\[CapitalDelta] f\)]],
" is the change in objective function value, and ",
Cell[BoxData[
\(TraditionalForm\`f\_0\)]],
" is the value of the objective function from the previous iteration. This \
is repeated for each starting point until the maximum number of iterations is \
reached, the method converges to a point based on the accuracy or precision \
goals, or the method stays at the same point consecutively for the number of \
iterations given by ",
StyleBox["LevelIterations", "MR"],
". When ",
StyleBox["BoltzmannExponent", "MR"],
Cell[BoxData[
\(TraditionalForm\` \[Rule] \)]],
StyleBox["Automatic", "MR"],
" the function used is ",
StyleBox["Function[", "MR"],
Cell[BoxData[
FormBox[
RowBox[{"{",
RowBox[{
StyleBox["i",
"MR"],
StyleBox[",",
"MR"], " ",
StyleBox["df",
"MR"],
StyleBox[",",
"MR"], " ",
StyleBox["f0",
"MR"]}],
FormBox["}",
"TraditionalForm"]}], TraditionalForm]]],
StyleBox[",", "MR"],
" ",
StyleBox["-df*Log[i+1]/10]", "MR"],
". The default value of ",
StyleBox["SearchPoints", "MR"],
" is ",
StyleBox["Automatic", "MR"],
", which is ",
StyleBox["Min[2*d,", "MR"],
" ",
StyleBox["50]", "MR"],
", where ",
Cell[BoxData[
\(TraditionalForm\`d\)]],
" is the number of variables."
}], "Text",
CellTags->{"S5.90.1", "1.43"}]
}, Open ]],
Cell[CellGroupData[{
Cell["Automatic", "Subsection",
CellDingbat->None],
Cell[TextData[{
"With ",
StyleBox["Method", "MR"],
Cell[BoxData[
\(TraditionalForm\` \[Rule] \)]],
StyleBox["Automatic", "MR"],
", ",
StyleBox["NMinimize", "MR"],
" picks which method to use based on the type of problem. If the objective \
function and constraints are linear then a pure linear programming routine is \
called. If there are integer variables, or if the head of the objective \
function isn't a numeric function, ",
StyleBox["DifferentialEvolution", "MR"],
" is used. For everything else, it uses ",
StyleBox["NelderMead", "MR"],
", and if that does poorly it tries ",
StyleBox["DifferentialEvolution", "MR"],
". "
}], "Text",
CellTags->{"S5.90.1", "1.20"}]
}, Open ]]
}, Open ]],
Cell[CellGroupData[{
Cell["Handling of Constraints", "Section"],
Cell[TextData[{
StyleBox["NMinimize", "MR"],
" handles constraints in a number of ways depending on the their type. The \
first step is to expand the constraints to the Form ",
StyleBox["Or[And[..], And[..], ..]", "MR"],
" so that we can solve the problem over each of the ",
StyleBox["Or", "MR"],
" branches. Inequalities are changed to the form ",
StyleBox["g\[LessEqual]0", "DisplayFormula",
FontSlant->"Italic"],
", equalities to ",
StyleBox["h\[Equal]0",
FontSlant->"Italic"],
", and domain constraints are reduced to a list of integer variables. "
}], "Text"],
Cell[TextData[{
"If linear equalities are present, ",
StyleBox["NMinimize", "MR"],
" solves the equations exactly, and plugs the solution into the other \
constraints, reducing the dimension of the problem. It will not solve for \
integer variables, although solutions for non-integer variables may contain \
the integer variables."
}], "Text"],
Cell["\<\
Linear inequalities are currently handled differently depending on \
whether they give a bound on a variable. If so, then after a new point is \
generated, the point is checked against the bound constraints, and violations \
are truncated to satisfy the constraints. Other linear inequalities are \
treated the same as nonlinear inequality constraints, as next \
described.\
\>", "Text"],
Cell[TextData[{
"Nonlinear constraints, including certain linear inequalities as just \
noted, are handled with penalty functions applied outside the feasible \
region. Penalty functions are functions of two variables, with the first \
variable being the positive constraint violation, and the second variable \
being the current iteration number. The default penalty function doubles the \
penalty term multiplier every ten iterations and uses quadratic penalty \
terms. This can be changed using the ",
StyleBox["PenaltyFunction", "MR"],
" option. In the following example we double the multiplier every \
iteration, with quadratic penalty terms:"
}], "Text"],
Cell[BoxData[
\(NMinimize[{obj, \ cons}, \ vars, \
PenaltyFunction \[Rule]
Function[{d, i}, \(d\^2\) 2\^i]]\)], "NumberedEquation"],
Cell["\<\
Domains are given by adding Element[x, Integers] or Element[x, \
Reals] to the constraints. All variables must be either integers or real \
numbers at this point.\
\>", "Text"],
Cell[TextData[{
StyleBox["NMinimize", "MR"],
" handles integer variables ",
Cell[BoxData[
\(TraditionalForm\`x\_i\)]],
" by substituting ",
Cell[BoxData[
FormBox[
RowBox[{
StyleBox["Round",
"MR"], "[", \(x\_i\), "]"}], TraditionalForm]]],
" for ",
Cell[BoxData[
\(TraditionalForm\`x\_i\)]],
" in the objective function and all constraints. This is made possible by \
using methods that don't rely on derivatives. It is also possible for users \
to handle integer variables on their own, e.g. by adding terms like ",
Cell[BoxData[
FormBox[
SuperscriptBox[
RowBox[{"(",
RowBox[{"x", "-",
RowBox[{
StyleBox["Round",
"MR"], "[", "x", "]"}]}], ")"}], "2"], TraditionalForm]]],
"to the objective function or by using a mapping from real numbers to \
integers in the objective function."
}], "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell["Choosing an Initial Region", "Section"],
Cell[TextData[{
"Since ",
StyleBox["NMinimize", "MR"],
" methods generally need several initial points, ",
StyleBox["NMinimize", "MR"],
" takes a starting region as input, analogous to the way ",
StyleBox["FindMinimum", "MR"],
" and ",
StyleBox["FindRoot", "MR"],
" take starting points. The initial regions are calculated from the \
variables and constraints passed into ",
StyleBox["NMinimize", "MR"],
". Each variable can be given as either the variable by itself, or in a \
list with the upper and lower bounds of the initial region for that variable. \
If no initial bound is given in the variable list, we look to the constraints \
for information, and if nothing there can be used, the default range for the \
variable is the interval [-1,1]. Here are several examples:"
}], "Text"],
Cell[BoxData[
\(NMinimize[f[x], \ {x}]\)], "NumberedEquation"],
Cell["\<\
Since there are no constraints, and no explicit initial region, the \
default initial region [-1,1] is used.\
\>", "Text"],
Cell[BoxData[
\(NMinimize[f[x], \ {{x, \ \(-5\), \ 5}}]\)], "NumberedEquation"],
Cell["\<\
Here there still are no constraints but there is an explicit \
initial region, [-5,5], that is used.\
\>", "Text"],
Cell[BoxData[
\(NMinimize[{f[x], \
0 \[LessEqual] x \[LessEqual] 5}, \ {x}]\)], "NumberedEquation"],
Cell["\<\
This example has upper and lower bounds on the variable, but no \
explicit initial region, so the region defined by the constraints, [0,5], is \
used.\
\>", "Text"],
Cell[BoxData[
\(NMinimize[{f[x], \
0 \[LessEqual] x \[LessEqual] 5}, \ {{x, \ 1, \
6}}]\)], "NumberedEquation"],
Cell["\<\
This example has an explicit initial region, as well as bounds on \
the variable. We use the intersection of the explicit initial region and the \
constraint region, giving [1,5] as the actual initial region.\
\>", "Text"],
Cell[BoxData[
\(NMinimize[{f[x], \ x \[LessEqual] 3}, \ {x}]\)], "NumberedEquation"],
Cell[TextData[{
"Here there is an upper bound, but no lower bound, and no explicit initial \
region. We use ",
StyleBox["Min[-1, UpperBound-2]", "MR"],
" as the lower bound, resulting in [-1,3] as the initial region."
}], "Text"],
Cell[BoxData[
\(NMinimize[{f[x], \ x \[GreaterEqual] 0}, \ {x}]\)], "NumberedEquation"],
Cell[TextData[{
"This example is the opposite of the previous one, with a lower bound, and \
no upper bound, but still no explicit initial region. In this case we use ",
StyleBox["Max[1, LowerBound+2]", "MR"],
" as the upper bound, which gives us [0,2] as the initial region."
}], "Text"],
Cell[TextData[{
"The initial region is just that\[LongDash]a place to start. The \
minimization process is able to move outside the region, so long as \
constraints aren't violated. It is also possible to specify an explicit set \
of starting points via the ",
StyleBox["InitialPoints", "MR"],
" option. This takes a list of starting points, whose ordering is the same \
as the ordering of the variables given to ",
StyleBox["NMinimize", "MR"],
". If a method needs more starting points than given by ",
StyleBox["InitialPoints", "MR"],
", it randomly generates points inside the initial region as specified \
above. If fewer points are needed, it takes points sequentially from the list \
until enough points are selected. "
}], "Text"],
Cell[TextData[{
StyleBox["NMinimize", "MR"],
" will generate up to ten times the number of needed points in an attempt \
to find points that additionally satisfy the non-rectangular inequality \
constraints. If it cannot find enough initial points that satisfy the \
constraints it issues a message and uses those generated points that have the \
smallest constraint violations. Equality constraints are ignored at this \
point."
}], "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell["When to Say When", "Section"],
Cell[TextData[{
StyleBox["NMinimize", "MR"],
" uses several criteria in determining when to stop. ",
StyleBox["MaxIterations", "MR"],
" gives the maximum number of iterations to be used. How this is \
interpreted may vary from method to method. For example, ",
StyleBox["RandomSearch", "MR"],
" will use the ",
StyleBox["MaxIterations", "MR"],
" value as the maximum number of iterations for each of the individual ",
StyleBox["FindMinimum", "MR"],
" calls. ",
StyleBox["AccuracyGoal", "MR"],
" and ",
StyleBox["PrecisionGoal", "MR"],
" behave much as they do in other functions\[LongDash]they provide a way to \
stop when either the absolute or relative change in function evaluations \
becomes smaller than a given amount. Most ",
StyleBox["NMinimize", "MR"],
" methods also look at the sum of squares of the constraint violations, and \
compare this to the value of the ",
StyleBox["Tolerance", "MR"],
" option."
}], "Text"],
Cell[TextData[{
StyleBox["NMinimize", "MR"],
" will run until it reaches the maximum number of iterations, unless the \
sum of squares of the constraint violations is below ",
StyleBox["Tolerance", "MR"],
" and either the absolute change in function values drops below ",
Cell[BoxData[
FormBox[
SuperscriptBox["10",
RowBox[{"-",
StyleBox["AccuracyGoal",
"MR"]}]], TraditionalForm]]],
"or the relative change drops below ",
Cell[BoxData[
FormBox[
SuperscriptBox["10",
RowBox[{"-",
StyleBox["PrecisionGoal",
"MR"]}]], TraditionalForm]]],
". For problems with integer variables, ",
StyleBox["NMinimize", "MR"],
" runs until the maximum number of iterations are used. One can make ",
StyleBox["NMinimize", "MR"],
" use the full ",
StyleBox["MaxIterations", "MR"],
" by setting both ",
StyleBox["AccuracyGoal\[Rule]\[Infinity]", "MR"],
" and ",
StyleBox["PrecisionGoal\[Rule]\[Infinity]", "MR"],
". For ",
StyleBox["NelderMead", "MR"],
" and ",
StyleBox["RandomSearch", "MR"],
", ",
StyleBox["AccuracyGoal", "MR"],
" and ",
StyleBox["PrecisionGoal", "MR"],
" are checked every iteration, and for ",
StyleBox["DifferentialEvolution", "MR"],
" and ",
StyleBox["SimulatedAnnealing,", "MR"],
" every ten iterations."
}], "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell["Post-Processing", "Section"],
Cell[TextData[{
StyleBox["NMinimize", "MR"],
" generally does some post-processing of the results using ",
StyleBox["FindMinimum", "MR"],
". This frequently adds a good deal of local refinement once we are in the \
general area of the global minimum. This is configurable in all methods \
except ",
StyleBox["Automatic", "MR"],
" by changing the value of the ",
StyleBox["PostProcess", "MR"],
" method-option. Possible values for ",
StyleBox["PostProcess", "MR"],
" are ",
StyleBox["True", "MR"],
", ",
StyleBox["False", "MR"],
", ",
StyleBox["Automatic", "MR"],
", \"",
StyleBox["FindMinimum\"", "MR"],
", or a list of the form ",
StyleBox["{\"FindMinimum\", }", "MR"],
". At present the values ",
StyleBox["Automatic", "MR"],
" and ",
StyleBox["FindMinimum", "MR"],
" have the same effect as ",
StyleBox["True", "MR"],
". If ",
StyleBox["{\"FindMinimum\", }", "MR"],
" is used, then the options listed are passed to ",
StyleBox["FindMinimum", "MR"],
" in the post-processing stage in place of the default values used by ",
StyleBox["NMinimize", "MR"],
"."
}], "Text"],
Cell[TextData[{
"Since ",
StyleBox["FindMinimum", "MR"],
" does not accept constraints we need to use penalty functions to enforce \
the constraints in post-processing. For constraints that are not satisfied \
when post-processing begins, we use penalty terms based on the function given \
by the ",
StyleBox["PenaltyFunction", "MR"],
" option to encourage the solution to move inside the feasible region. For \
the constraints that are already satisfied, we use inverse barrier terms to \
keep the solution inside the feasible region."
}], "Text"],
Cell[TextData[{
"Since ",
StyleBox["FindMinimum", "MR"],
" uses descent methods which do not work well with integer variables, we do \
not try to improve integer components of solutions. Instead we fix the \
integer variables at the rounded value, and proceed to post-process with the \
remaining variables (if any). This, of course, may give less than optimal \
results."
}], "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell["And the Winner Is...", "Section"],
Cell[TextData[{
"Once we reach the end, we typically have several potential solutions, from \
which we need to choose the \"best\" solution. The solutions are first ranked \
according to the sum of squares of the constraint violations, and we discard \
any that violate the constraints by more than the value of ",
StyleBox["Tolerance", "MR"],
" plus the smallest constraint violation. The remaining solutions are then \
sorted by objective function value, and the best is returned. If this \
solution is not the one with the smallest constraint violation, a message is \
issued that notes the candidate solution with smallest constraint violation."
}], "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell["Examples", "Section"],
Cell[TextData[{
"Many elementary examples may be found in the on\[Hyphen]line documentation \
for ",
StyleBox["NMinimize", "MR"],
", which is available at "
}], "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell[" http://documents.wolfram.com/v4", "Address"],
Cell[TextData[{
"by following the links ",
StyleBox["Standard Packages\[Rule]NumericalMath\[Rule]NMinimize", "SR"],
". More examples may be found in talks from the 2001 ",
StyleBox["Mathematica",
FontSlant->"Italic"],
" Developer Conference at the following URLs:"
}], "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell["\<\
\
http://library.wolfram.com/conferences/devconf2001/champion/champion.sm.nb\
\>\
", "Address"],
Cell["\<\
\
http://library.wolfram.com/conferences/devconf2001/lichtblau/lichtblau.nb\
\>\
", "Address"],
Cell[CellGroupData[{
Cell["Permutation Example", "Subsection",
CellDingbat->None],
Cell[TextData[{
"Here is a simple example that demonstrates a technique that works well for \
permutation problems. We use ",
StyleBox["NMinimize", "MR"],
" to find the permutation that minimizes a dot product. We define a \
function on a list of real numbers that turns the list numbers into a \
permutation using ",
StyleBox["Ordering", "MR"],
", and takes the dot product of the numbers one through four with the \
permutation. The answer from ",
StyleBox["NMinimize", "MR"],
" is returned in terms of real numbers, and so a bit of work is done to get \
the answer back in terms of integers."
}], "Text"],
Cell[CellGroupData[{
Cell[BoxData[{
\(\(f[x : {_Real .. }] :=
Range[4] . Ordering[x];\)\), "\[IndentingNewLine]",
\(\(sol =
NMinimize[f[{a, b, c, d}], \ {a, b, c, d}, \
Compiled \[Rule] False];\)\), "\[IndentingNewLine]",
\(\(sol\[LeftDoubleBracket]2\[RightDoubleBracket] =
Thread[{a, b, c, d} \[Rule]
Ordering[{a, b, c, d} /.
Last[sol]]];\)\), "\[IndentingNewLine]",
\(sol\)}], "Input",
CellLabel->"In[14]:="],
Cell[BoxData[
\({20.`, {a \[Rule] 4, b \[Rule] 3, c \[Rule] 2,
d \[Rule] 1}}\)], "Output",
CellLabel->"Out[17]="]
}, Open ]]
}, Open ]],
Cell[CellGroupData[{
Cell["SIAM $100 Challenge Example", "Subsection",
CellDingbat->None],
Cell[TextData[{
"Here is a solution to the fourth problem in the SIAM $100 Challenge [4]. \
We use ",
StyleBox["DifferentialEvolution", "MR"],
" since it tends to be a bit more robust, albeit slower, than the default \
method."
}], "Text"],
Cell[CellGroupData[{
Cell[BoxData[{
\(\(f = \[ExponentialE]\^Sin[50\ x] + Sin[60\ \[ExponentialE]\^y] +
Sin[70\ Sin[x]] + Sin[Sin[80\ y]] - Sin[10\ \((x + y)\)] +
1\/4\ \((x\^2 + y\^2)\);\)\), "\[IndentingNewLine]",
\(NMinimize[f, \ {x, y}, \ Method \[Rule] "\", \
WorkingPrecision \[Rule] 32]\)}], "Input",
CellLabel->"In[18]:="],
Cell[BoxData[
\({\(-3.3068686474752372800761137708985118524102845`31.5606\), {x \[Rule] \
\(-0.02440307969437517113212592302992253453205967545`32\),
y \[Rule]
0.2106124271553557699442785449689184282472448`32}}\)], "Output",
CellLabel->"Out[19]="]
}, Open ]]
}, Open ]],
Cell[CellGroupData[{
Cell["Polynomial Example with Constraints", "Subsection",
CellDingbat->None],
Cell["\<\
Here is an example with nonlinear constraints. This problem is \
tough to solve numerically because multiple constraints are active at the \
solution [5]. The exact solution is -44 when at {0, 1, 2, -1}.\
\>", "Text"],
Cell[BoxData[{
\(\(f[w_Real, x_Real, y_Real, \ z_Real] :=
w\^2 + x\^2 + 2\ y\^2 + z\^2 - 5\ w - 5\ x - 21\ y + 7\ z +
Random[NormalDistribution[]];\)\), "\[IndentingNewLine]",
\(\(c1 =
2\ w\^2 + x\^2 + y\^2 + 2\ w - x - z - 5 \[LessEqual] 0;\)\), "\n",
\(\(c2 =
w\^2 + x\^2 + y\^2 + z\^2 + w - x + y - z - 8 \[LessEqual]
0;\)\), "\n",
\(\(c3 =
w\^2 + 2\ x\^2 + y\^2 + 2\ z\^2 - w - x - 10 \[LessEqual]
0;\)\)}], "Input",
CellLabel->"In[281]:="],
Cell[CellGroupData[{
Cell[BoxData[
\(NMinimize[{f[w, x, y, z], c1 \[And] c2 \[And] c3}, {w, x, y, z},
Method \[Rule] DifferentialEvolution, \
PrecisionGoal \[Rule] \[Infinity], \
AccuracyGoal \[Rule] \[Infinity], \
MaxIterations \[Rule] 500]\)], "Input",
CellLabel->"In[26]:="],
Cell[BoxData[
\({\(-43.99992156842104`\), {w \[Rule] \(-0.0015141488397362028`\),
x \[Rule] 0.9998441881976146`, y \[Rule] 2.001124844623207`,
z \[Rule] \(-0.9986686775638975`\)}}\)], "Output",
CellLabel->"Out[26]="]
}, Open ]]
}, Open ]],
Cell[CellGroupData[{
Cell["Stochastic Objective Function Example", "Subsection",
CellDingbat->None],
Cell["\<\
Next we will add some Gaussian noise to the objective function from \
the previous example.\
\>", "Text"],
Cell[BoxData[{
\(\(<< "\";\)\), "\[IndentingNewLine]",
\(\(fr[w_Real, x_Real, y_Real, \ z_Real] :=
w\^2 + x\^2 + 2\ y\^2 + z\^2 - 5\ w - 5\ x - 21\ y + 7\ z +
Random[NormalDistribution[]];\)\)}], "Input"],
Cell["\<\
We will use multiple objective function evaluations for improved \
performance, although this will be slower. We still get solutions near the \
noise\[Hyphen]free minimum, but not as accurately. Our results appear to be \
comparable in quality to those of dedicated stochastic solvers [6].\
\>", \
"Text"],
Cell[BoxData[
\(\(f\_5[w_Real, x_Real, y_Real, \ z_Real] :=
Sum[fr[w, x, y, z], {i, 5}]/5;\)\)], "Input",
CellLabel->"In[30]:="],
Cell[CellGroupData[{
Cell[BoxData[
\(NMinimize[{f\_5[w, x, y, z], c1 \[And] c2 \[And] c3}, {w, x, y, z},
Method \[Rule] DifferentialEvolution, \ MaxIterations \[Rule] 500, \
PrecisionGoal \[Rule] \[Infinity], \
AccuracyGoal \[Rule] \[Infinity]]\)], "Input",
CellLabel->"In[33]:="],
Cell[BoxData[
\({\(-44.15150032255648`\), {w \[Rule] 0.05301557767307735`,
x \[Rule] 0.8078489680458282`, y \[Rule] 2.004806957275463`,
z \[Rule] \(-0.9754812534695265`\)}}\)], "Output",
CellLabel->"Out[33]="]
}, Open ]]
}, Open ]],
Cell[CellGroupData[{
Cell["References", "Section"],
Cell[TextData[{
Cell[BoxData[
\(TraditionalForm\`\(\(\[InvisibleSpace]\)\([1]\)\)\)]],
" K. Price and R. Storn, \[OpenCurlyDoubleQuote]Differential Evolution,\
\[CloseCurlyDoubleQuote] ",
StyleBox["Dr. Dobb's Journal,", "TI"],
" April 1997, pp. 18\[Dash]24, 78. \n",
Cell[BoxData[
\(TraditionalForm\`\(\(\[InvisibleSpace]\)\([2]\)\)\)]],
" R. Mead and J. A. Nelder, \[OpenCurlyDoubleQuote]A Simplex Method for \
Function Minimization,\[CloseCurlyDoubleQuote] ",
StyleBox["Computer Journal,", "TI"],
" ",
StyleBox["7", "TB"],
" (4), 1965 pp. 308\[Dash]312. \n",
Cell[BoxData[
\(TraditionalForm\`\(\(\[InvisibleSpace]\)\([3]\)\)\)]],
" L. Ingber, \[OpenCurlyDoubleQuote]Simulated annealing: Practice versus \
theory,\[CloseCurlyDoubleQuote] ",
StyleBox["Mathematical Computer Modelling,", "TI"],
" ",
StyleBox["18", "TB"],
" (11), 1993 pp. 29\[Dash]57. \n",
Cell[BoxData[
\(TraditionalForm\`\(\(\[InvisibleSpace]\)\([4]\)\)\)]],
" N. Trefethen, \"A Hundred-dollar, Hundred-digit Challenge,\"",
StyleBox["SIAM News",
FontSlant->"Italic"],
", ",
StyleBox["35",
FontWeight->"Bold"],
" (1), 2002 p. ??. \n",
Cell[BoxData[
\(TraditionalForm\`\(\(\[InvisibleSpace]\)\([5]\)\)\)]],
" H.-P. Shwefel, ",
StyleBox["Evolution and Optimum Seeking",
FontSlant->"Italic"],
", New York: John Wiley and Sons, 1995. \n",
Cell[BoxData[
\(TraditionalForm\`\(\(\[InvisibleSpace]\)\([6]\)\)\)]],
" I.-J. Wang and J. C. Spall, \[OpenCurlyDoubleQuote]A Constrained \
Simultaneous Perturbation Stochastic Approximation Algorithm Based on Penalty \
Functions,\[CloseCurlyDoubleQuote] in ",
StyleBox["Proceedings of the American Control Conference,", "TI"],
" ",
StyleBox["San Diego, CA.", "TB",
FontWeight->"Plain"],
"1999 pp. 393\[Dash]399. "
}], "Text",
CellTags->{"NMinimize", "1.77"}]
}, Open ]]
}, Open ]]
}, Open ]]
},
FrontEndVersion->"4.2 for Macintosh",
ScreenRectangle->{{0, 1240}, {0, 1002}},
WindowToolbars->"EditBar",
WindowSize->{628, 768},
WindowMargins->{{Automatic, 175}, {Automatic, 0}},
PrintingCopies->1,
PrintingPageRange->{1, Automatic},
Magnification->1,
StyleDefinitions -> "ArticleClassic.nb"
]
(*******************************************************************
Cached data follows. If you edit this Notebook file directly, not
using Mathematica, you must remove the line containing CacheID at
the top of the file. The cache data will then be recreated when
you save this file from within Mathematica.
*******************************************************************)
(*CellTagsOutline
CellTagsIndex->{
"S5.90.1"->{
Cell[5934, 193, 1725, 58, 106, "Text",
CellTags->{"S5.90.1", "1.24"}],
Cell[7752, 259, 3075, 94, 154, "Text",
CellTags->{"S5.90.1", "1.30"}],
Cell[11634, 380, 633, 21, 58, "Text",
CellTags->{"S5.90.1", "1.36"}],
Cell[12368, 409, 2708, 83, 186, "Text",
CellTags->{"S5.90.1", "1.43"}],
Cell[15168, 500, 707, 19, 74, "Text",
CellTags->{"S5.90.1", "1.20"}]},
"1.24"->{
Cell[5934, 193, 1725, 58, 106, "Text",
CellTags->{"S5.90.1", "1.24"}]},
"1.30"->{
Cell[7752, 259, 3075, 94, 154, "Text",
CellTags->{"S5.90.1", "1.30"}]},
"1.36"->{
Cell[11634, 380, 633, 21, 58, "Text",
CellTags->{"S5.90.1", "1.36"}]},
"1.43"->{
Cell[12368, 409, 2708, 83, 186, "Text",
CellTags->{"S5.90.1", "1.43"}]},
"1.20"->{
Cell[15168, 500, 707, 19, 74, "Text",
CellTags->{"S5.90.1", "1.20"}]},
"NMinimize"->{
Cell[34846, 1085, 1885, 49, 170, "Text",
CellTags->{"NMinimize", "1.77"}]},
"1.77"->{
Cell[34846, 1085, 1885, 49, 170, "Text",
CellTags->{"NMinimize", "1.77"}]}
}
*)
(*CellTagsIndex
CellTagsIndex->{
{"S5.90.1", 37497, 1159},
{"1.24", 37914, 1170},
{"1.30", 38008, 1173},
{"1.36", 38102, 1176},
{"1.43", 38195, 1179},
{"1.20", 38290, 1182},
{"NMinimize", 38388, 1185},
{"1.77", 38486, 1188}
}
*)
(*NotebookFileOutline
Notebook[{
Cell[CellGroupData[{
Cell[1727, 52, 149, 5, 171, "Title"],
Cell[1879, 59, 32, 0, 40, "Author"],
Cell[1914, 61, 41, 0, 20, "Address"],
Cell[1958, 63, 834, 15, 146, "Abstract"],
Cell[CellGroupData[{
Cell[2817, 82, 57, 0, 63, "SectionFirst"],
Cell[2877, 84, 343, 9, 42, "Text"],
Cell[3223, 95, 71, 1, 25, "NumberedEquation"],
Cell[3297, 98, 109, 2, 26, "Text"],
Cell[3409, 102, 64, 1, 25, "NumberedEquation"],
Cell[3476, 105, 296, 9, 42, "Text"],
Cell[3775, 116, 74, 1, 25, "NumberedEquation"],
Cell[3852, 119, 503, 10, 74, "Text"],
Cell[4358, 131, 89, 1, 25, "NumberedEquation"],
Cell[4450, 134, 123, 2, 26, "NumberedEquation"],
Cell[4576, 138, 821, 29, 75, "Text"],
Cell[5400, 169, 379, 12, 42, "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell[5816, 186, 26, 0, 57, "Section"],
Cell[CellGroupData[{
Cell[5867, 190, 64, 1, 42, "Subsection"],
Cell[5934, 193, 1725, 58, 106, "Text",
CellTags->{"S5.90.1", "1.24"}]
}, Open ]],
Cell[CellGroupData[{
Cell[7696, 256, 53, 1, 42, "Subsection"],
Cell[7752, 259, 3075, 94, 154, "Text",
CellTags->{"S5.90.1", "1.30"}],
Cell[10830, 355, 709, 17, 90, "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell[11576, 377, 55, 1, 42, "Subsection"],
Cell[11634, 380, 633, 21, 58, "Text",
CellTags->{"S5.90.1", "1.36"}]
}, Open ]],
Cell[CellGroupData[{
Cell[12304, 406, 61, 1, 42, "Subsection"],
Cell[12368, 409, 2708, 83, 186, "Text",
CellTags->{"S5.90.1", "1.43"}]
}, Open ]],
Cell[CellGroupData[{
Cell[15113, 497, 52, 1, 42, "Subsection"],
Cell[15168, 500, 707, 19, 74, "Text",
CellTags->{"S5.90.1", "1.20"}]
}, Open ]]
}, Open ]],
Cell[CellGroupData[{
Cell[15924, 525, 42, 0, 57, "Section"],
Cell[15969, 527, 593, 14, 74, "Text"],
Cell[16565, 543, 350, 7, 58, "Text"],
Cell[16918, 552, 397, 7, 58, "Text"],
Cell[17318, 561, 667, 11, 90, "Text"],
Cell[17988, 574, 151, 3, 27, "NumberedEquation"],
Cell[18142, 579, 186, 4, 42, "Text"],
Cell[18331, 585, 946, 27, 74, "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell[19314, 617, 45, 0, 57, "Section"],
Cell[19362, 619, 811, 17, 106, "Text"],
Cell[20176, 638, 66, 1, 25, "NumberedEquation"],
Cell[20245, 641, 132, 3, 26, "Text"],
Cell[20380, 646, 83, 1, 25, "NumberedEquation"],
Cell[20466, 649, 124, 3, 26, "Text"],
Cell[20593, 654, 112, 2, 25, "NumberedEquation"],
Cell[20708, 658, 175, 4, 42, "Text"],
Cell[20886, 664, 135, 3, 25, "NumberedEquation"],
Cell[21024, 669, 232, 4, 42, "Text"],
Cell[21259, 675, 88, 1, 25, "NumberedEquation"],
Cell[21350, 678, 235, 5, 42, "Text"],
Cell[21588, 685, 91, 1, 25, "NumberedEquation"],
Cell[21682, 688, 295, 5, 42, "Text"],
Cell[21980, 695, 751, 14, 106, "Text"],
Cell[22734, 711, 445, 8, 74, "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell[23216, 724, 35, 0, 57, "Section"],
Cell[23254, 726, 963, 23, 122, "Text"],
Cell[24220, 751, 1383, 42, 122, "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell[25640, 798, 34, 0, 57, "Section"],
Cell[25677, 800, 1151, 35, 122, "Text"],
Cell[26831, 837, 558, 11, 74, "Text"],
Cell[27392, 850, 390, 8, 58, "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell[27819, 863, 39, 0, 57, "Section"],
Cell[27861, 865, 667, 10, 90, "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell[28565, 880, 27, 0, 57, "Section"],
Cell[28595, 882, 174, 5, 26, "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell[28806, 892, 51, 0, 20, "Address"],
Cell[28860, 894, 290, 7, 42, "Text"]
}, Open ]],
Cell[CellGroupData[{
Cell[29187, 906, 106, 4, 20, "Address"],
Cell[29296, 912, 105, 4, 20, "Address"],
Cell[CellGroupData[{
Cell[29426, 920, 62, 1, 42, "Subsection"],
Cell[29491, 923, 621, 13, 90, "Text"],
Cell[CellGroupData[{
Cell[30137, 940, 478, 11, 76, "Input"],
Cell[30618, 953, 128, 3, 28, "Output"]
}, Open ]]
}, Open ]],
Cell[CellGroupData[{
Cell[30795, 962, 70, 1, 42, "Subsection"],
Cell[30868, 965, 245, 6, 42, "Text"],
Cell[CellGroupData[{
Cell[31138, 975, 373, 6, 97, "Input"],
Cell[31514, 983, 272, 5, 60, "Output"]
}, Open ]]
}, Open ]],
Cell[CellGroupData[{
Cell[31835, 994, 78, 1, 42, "Subsection"],
Cell[31916, 997, 227, 4, 42, "Text"],
Cell[32146, 1003, 534, 12, 100, "Input"],
Cell[CellGroupData[{
Cell[32705, 1019, 290, 6, 60, "Input"],
Cell[32998, 1027, 240, 4, 28, "Output"]
}, Open ]]
}, Open ]],
Cell[CellGroupData[{
Cell[33287, 1037, 80, 1, 42, "Subsection"],
Cell[33370, 1040, 115, 3, 26, "Text"],
Cell[33488, 1045, 266, 4, 62, "Input"],
Cell[33757, 1051, 315, 6, 58, "Text"],
Cell[34075, 1059, 143, 3, 28, "Input"],
Cell[CellGroupData[{
Cell[34243, 1066, 286, 5, 60, "Input"],
Cell[34532, 1073, 233, 4, 28, "Output"]
}, Open ]]
}, Open ]],
Cell[CellGroupData[{
Cell[34814, 1083, 29, 0, 57, "Section"],
Cell[34846, 1085, 1885, 49, 170, "Text",
CellTags->{"NMinimize", "1.77"}]
}, Open ]]
}, Open ]]
}, Open ]]
}
]
*)
(*******************************************************************
End of Mathematica Notebook file.
*******************************************************************)