The problems in this notebook are from the Mathematica 4.2 Help file under NMinimize (top sections) and other simple test problems.  In all cases, Global Optimization functions are faster or get a better solution.  GlobalPenaltyFn is slower but is able to handle very complex and even nonanalytic constraints.

<< NumericalMath`NMinimize`

NMaximize[Sin[x + y] - (x^2) - (y^2), {x, y}] // Timing

GlobalSearch[-(Sin[x + y] - x^2 - y^2), , , {{x, -2, 
            2}, {y, -2, 2}},  .00001, Starts -> 1] // 
      Timing

GlobalPenaltyFn[-(Sin[x + y] - x^2 - y^2), , , {{x, -2, 
            2}, {y, -2, 2}},  .000001, Starts -> 1] // 
      Timing

Note that GlobalMinima is very very fast on this problem:

N[GlobalMinima[-(Sin[x + y] - x^2 - y^2), , {{x, -2, 
              2}, {y, -2, 2}}, 4,  .00001,  .3,  .0000001, 
          ShowProgress -> False] // Timing]

======================================

Here we restrict a variable x to be Integer.

NMinimize[{(((x - 1/3))^2) + (((y - 1/3))^2), 
          x ∈ Integers}, {x, y}] // Timing

In the Global Optimization package, only MultiStartMin can use Integer variables.  It is much faster on this problem.

MultiStartMin[(((x - 1/3))^2) + (((y - 1/3))^2), , , {{x, 0, 2, 
            Integer}, {y, 0, 1}},  .0000000001, Starts -> 1] // 
      Timing

======================================

NMinimize[
        (x^4) + ((((y^2) - π))^2), {{x, -2, 2}, {y, -2, 2}}, 
        Method -> "DifferentialEvolution"] // Timing

NMinimize[
        (x^4) + ((((y^2) - π))^2), {{x, -2, 2}, {y, -2, 2}}, 
        Method -> {"DifferentialEvolution", "RandomSeed" -> 
              2002}] // Timing

GlobalSearch[
        (x^4) + ((((y^2) - π))^2), {-y, -y + x + 
            1}, , {{x, -2, 2}, {y, -2, 2}},  .000000000001, 
        Starts -> 1] // Timing

======================================

Here is an example where the NMinimize default penalty function is not quite strong enough to force the constraint to be satisfied.

NMinimize[{(x^2) + (y^2), (((y - 1))^3) ≥ (x^2)}, {{x, 0, 
            5}, {y, 0, 5}}] // Timing

Next, the PenaltyFunction is increased:

NMinimize[{(x^2) + (y^2), (((y - 1))^3) ≥ (x^2)}, {{x, 0, 
            5}, {y, 0, 5}}, 
        PenaltyFunction -> Function[{d, i}, (2^i) (d^2)]] // 
      Timing

The default for both GlobalSearch and GlobalPenaltyFn find the exact solution.

GlobalSearch[
        (x^2) + (y^2), {-(((y - 1))^3) + (x^2)}, , {{x, -2, 
            2}, {y, -2, 2}},  .0000001, Starts -> 1] // 
      Timing

======================================

NMinimize[{(x^2) + (((y -  .5))^2), 
          y ≥ 0 && y ≥ x + 1}, {x, y}] // 
      Timing

GlobalSearch[
        x^2 + (y -  .5)^2, {-y, -y + x + 1}, , {{x, -2, 
            2}, {y, -2, 2}},  .0000001, Starts -> 1] // 
      Timing

GlobalPenaltyFn[
        x^2 + (y -  .5)^2, {-y, -y + x + 1}, , {{x, -2, 
            2}, {y, -2, 2}},  .000001, Starts -> 1, 
        ShowProgress -> False] // Timing

======================================

NMinimize[{x1^2 + x2^2 + x3^2 + x4^2, 
          x1 ≥ 1 && x2 > 1}, {{x1, -2, 2}, {x2, -2, 
            2}, {x3, -2, 2}, {x4, -2, 2}}] // Timing

GlobalSearch[
        x1^2 + x2^2 + x3^2 + 
          x4^2, {-x1 + 1, -x2 + 1}, , {{x1, -2, 2}, {x2, -2, 
            2}, {x3, -2, 2}, {x4, -2, 2}},  .00001, 
        Starts -> 1] // Timing

GlobalPenaltyFn[
        x1^2 + x2^2 + x3^2 + 
          x4^2, {-x1 + 1, -x2 + 1}, , {{x1, -2, 2}, {x2, -2, 
            2}, {x3, -2, 2}, {x4, -2, 2}},  .00001, Starts -> 1, 
        ShowProgress -> False] // Timing

MultiStartMin[
        x1^2 + x2^2 + x3^2 + 
          x4^2, {-x1 + 1, -x2 + 1}, , {{x1, -2, 2}, {x2, -2, 
            2}, {x3, -2, 2}, {x4, -2, 2}},  .00001, 
        Starts -> 1] // Timing

res = 
      IntervalMin[
          x1^2 + x2^2 + x3^2 + 
            x4^2, {-x1 + 1, -x2 + 1}, , {Interval[{-2, 2}], 
            Interval[{-2, 2}], Interval[{-2, 2}], 
            Interval[{-2, 2}]}, {x1, x2, x3, x4},  .00000001, 
          ShowProgress -> False] // Timing

N[res]

======================================

NMinimize[{x1^2 + x2^2 + x3^2 + x4^2, 
          x1 + x2 == 1 && x3 > 1}, {{x1, -2, 2}, {x2, -2, 
            2}, {x3, -2, 2}, {x4, -2, 2}}] // Timing

GlobalSearch[
        x1^2 + x2^2 + x3^2 + 
          x4^2, {-x3 + 1}, {x1 + x2 - 1}, {{x1, -2, 2}, {x2, -2, 
            2}, {x3, -2, 2}, {x4, -2, 2}},  .000001, 
        Starts -> 1] // Timing

GlobalPenaltyFn[
        x1^2 + x2^2 + x3^2 + 
          x4^2, {-x3 + 1}, {x1 + x2 - 1}, {{x1, -2, 2}, {x2, -2, 
            2}, {x3, -2, 2}, {x4, -2, 2}},  .0000001, 
        Starts -> 1] // Timing

MultiStartMin[
        x1^2 + x2^2 + x3^2 + 
          x4^2, {-x3 + 1}, {x1 + x2 - 1}, {{x1, -2, 2}, {x2, -2, 
            2}, {x3, -2, 2}, {x4, -2, 2}},  .00001, 
        Starts -> 1] // Timing

======================================

NMinimize[{(x^2),  (x^4) + (x^3) ≥ 1}, {{x, 0, 5}}] // 
      Timing

(x^4) + (x^3) /. {x -> 0.8191725133961645`}

GlobalSearch finds the solution.

GlobalSearch[x^2, {-x^4 - x^3 + 1}, , {{x, 0, 5}},  .00000001, 
        Starts -> 2, ShowProgress -> False] // Timing

GlobalPenaltyFn obtains the true min solution in less time:

GlobalPenaltyFn[x^2, {-x^4 - x^3 + 1}, , {{x, 0, 2}},  .00000001, 
        Starts -> 1, ShowProgress -> False] // Timing

MultiStartMin is the fastest here:

MultiStartMin[x^2, {-x^4 - x^3 + 1}, , {{x, 0, 2}},  .00000001, 
        Starts -> 2, ShowProgress -> False] // Timing

======================================

NMinimize[{100 (((y - (x^2)))^2) + (((1 - x))^2), -2.084 ≤ x ≤ 2.084 && -2.084 ≤ y ≤ 
              2.084}, {{x, -2, 2}, {y, -2, 2}}] // Timing

GlobalSearch[
        100 (((y - (x^2)))^2) + (((1 - x))^2), {}, {}, {{x, -2, 
            2}, {y, -2, 2}},  .0000001, Starts -> 1] // 
      Timing

MultiStartMin[
        100 (((y - (x^2)))^2) + (((1 - x))^2), {}, {}, {{x, -2, 
            2}, {y, -2, 2}},  .0000001, Starts -> 1] // 
      Timing

res = 
      GlobalMinima[
          100 (((y - (x^2)))^2) + (((1 - x))^2), , {{x, -2, 
              2}, {y, -2, 2}}, 4,  .00001,  .2,  .000000001, 
          ShowProgress -> False] // Timing

N[res]

======================================

soln= -210

Clear[x, f, vars]

x = 
        Table[ToExpression[StringJoin["x", ToString[i]]], {i, 1, 
            10}];

f = 
        Sum[(((x[[i]] - 1))^2), {i, 1, 10}] - 
          Sum[x[[i]] x[[i - 1]], {i, 2, 10}];

vars = Table[x[[i]], {i, 1, Length[x]}]

vardefs = Table[{x[[i]], -10, 10}, {i, 1, 10}];

The default does not find the solution quite:

NMinimize[f, vars, 
      Method -> "DifferentialEvolution"]

Using a range of values for the variables does solve it:

NMinimize[f, vardefs, Method -> "DifferentialEvolution"] // 
      Timing

\(NMinimize[f, vars, 
        Method -> {"DifferentialEvolution", "CrossProbability" -> 0.1}] // Timing\)

The GlobalSearch and MultiStartMin functions are much faster:

GlobalSearch[f, {}, , vardefs,  .00001, Starts -> 1] // 
      Timing

MultiStartMin[f, {}, , vardefs,  .00001, Starts -> 1] // 
      Timing

======================================

Plot3D[
      20  Sin[Pi/2 (x - 2  Pi)] + 
        20  Sin[Pi/2 (y - 2  Pi)] + (x - 2  Pi)^2 + (y - 
              2  Pi)^2, {x, 0, 10}, {y, 0, 10}]

true solution is: {{x->5.322168876695626`,y->5.322168876695626`},-38.07792316073022`}

NMinimize default fails

ClearAll[x]

NMinimize[
        20  Sin[Pi/2 (x - 2  Pi)] + 
          20  Sin[Pi/2 (y - 2  Pi)] + (x - 2  Pi)^2 + (y - 
                2  Pi)^2, {x, y}] // Timing

\(NMinimize[
        20  Sin[Pi/2 \((x - 2  Pi)\)] + 
          20  Sin[Pi/2 \((y - 2  Pi)\)] + \((x - 2  Pi)\)^2 + \((y - 
                2  Pi)\)^2, {x, y}, 
        Method -> {"SimulatedAnnealing", "PerturbationScale" -> 3}] // Timing\)

GlobalSearch solves this on the default with 4 starts:

ClearAll[x]

GlobalSearch[
        20  Sin[Pi/2 (x - 2  Pi)] + 
          20  Sin[Pi/2 (y - 2  Pi)] + (x - 2  Pi)^2 + (y - 
                2  Pi)^2, {}, , {{x, 0, 10}, {y, 0, 10}},  .00001, 
        Starts -> 4] // Timing

Because we know the range and it is 2D, GlobalMinima is the fastest:

res = 
      GlobalMinima[
          20  Sin[Pi/2 (x - 2  Pi)] + 
            20  Sin[Pi/2 (y - 2  Pi)] + (x - 2  Pi)^2 + (y - 
                  2  Pi)^2, , {{x, 0, 10}, {y, 0, 10}}, 
          4,  .00001,  .3,  .0000001, ShowProgress -> False] // 
        Timing

N[res]

ClearAll[x]

We see next that IntervalMin is slower but reliable on this problem:

res = 
      IntervalMin[
          20  Sin[Pi/2 (x - 2  Pi)] + 
            20  Sin[Pi/2 (y - 2  Pi)] + (x - 2  Pi)^2 + (y - 
                  2  Pi)^2, {-x1 + 1, -x2 + 
              1}, , {Interval[{0, 10}], Interval[{0, 10}]}, {x, 
            y},  .00000001, ShowProgress -> False] // Timing

N[res]

======================================

Next is shekel's foxholes, a very difficult problem.

Clear[a, shekel]; 
    a = Reverse /@ 
        Distribute[{{-32, -16, 0, 16, 32}, {-32, -16, 0, 16, 
              32}}, List]; 
    shekel = 1/(0.002 + 
            Plus @@ MapIndexed[
                1/(#2[[1]] + 
                        Plus @@ (({x, y} - #1)^6)) &, a]); 
    Plot3D[shekel, {x, -50, 50}, {y, -50, 50}, PlotPoints -> 50, 
      Mesh -> False];

shekel

With the default, NMinimize does not do so well.

Do[Print[
          NMinimize[shekel, {{x, -50, 50}, {y, -50, 50}}, 
            Method -> {"NelderMead", "RandomSeed" -> 
                  i}]], {i, 5}] // Timing

\(Do[Print[
          NMinimize[shekel, {{x, \(-50\), 50}, {y, \(-50\), 50}}, 

            Method -> {"NelderMead", "RandomSeed" -> 
                  i, 
"ShrinkRatio" ->  .95, 
"ContractRatio" ->  .95, 
"ReflectRatio" -> 1.5}]], {i, 5}] // 
      Timing\)

Do[Print[
          NMinimize[shekel, {{x, -50, 50}, {y, -50, 50}}, 
            Method -> {"RandomSearch", "RandomSeed" -> 
                  i, "SearchPoints" -> 2000}]], {i, 5}] // 
      Timing

GlobalSearch finds the best solution with 8 starts in much less time.

GlobalSearch[
        shekel, {}, , {{x, -50, 50}, {y, -50, 50}},  .00001, 
        Starts -> 12] // Timing

======================================

The solution of the next problem is any number ≤ 0 for x and y.  FindMinimum can't solve this.  GlobalSearch is much faster on this problem.

FindMinimum[Max[0, x] + Max[0, y], {x, 3, 4}, {y, 3, 4}]

GlobalSearch[
        Max[0, x] + Max[0, y], {}, , {{x, 2, 3}, {y, 2, 3}},  .0000001, 
        Starts -> 1, CompileOption -> False] // Timing

NMinimize[Max[0, x] + Max[0, y], {x, y}] // Timing

======================================

In this example, Solve fails because the functions are non-algebraic.  GlobalPenaltyFn solves this problem faster than NMinimze.

Solve[Floor[(x^2)] == 196 && Mod[x, 5] == 4, {x}]

NMinimize[{1, 
          Floor[(x^2)] == 196 && Mod[x, 5] == 4}, {{x, -50, 
            50}}] // Timing

GlobalPenaltyFn[
        1, , {Floor[(x^2)] - 196, Mod[x, 5] - 4}, {{x, -50, 50}},  .00001, 
        Starts -> 1] // Timing

======================================

For the following simple constrained problem, GlobalSearch solves this problem faster than NMinimze.

NMinimize[{x^2, x > 1}, {{x, 3, 4}}] // Timing

GlobalSearch[(x^2), {-x + 1}, , {{x, 3, 4}},  .0000001, 
        Starts -> 1] // Timing