Approximation

machine learning framework for Mathematica

While we have presented the general task of supervised analysis, we will now focus on creating computational models from data. For that purpose we will show how supervised methods can be used to solve numeric approximation methods. While traditional analytical solutions might provide higher accuracy, the models obtained from the methods shown in this chapter are very easy to interpret, even by nonexperts.

Initialization of the system

Needs["Graphics`Graphics`"]
Needs["mlf`"]

A simple 1-dimensional example

In this example we will show how a one-dimensional function can be approximated using various methods.

Setup data

First we get the sample data. To do so, we define the first coordinate to contain the x, and the second coordinate to contain the y = f(x) values.

f[x_] := N@Sin[Sqrt@x];
range = 100;
Plot[f[x], {x, 0, range}];

We will use 1000 samples for training.

samples = 1000;
trainData = 
        Table[{x*range/samples, f[x*range/samples]}, {x, 0, 
            samples}];

trainDataSet = 
        Def["DataSet", 
          DataSet[trainData, {"x", "y"}]];

testData = List /@ Range[0, 100];

Simple controller

As a first attempt, we will define a simple controller by hand.

We define a set of 10 predicates.

\(\(testParts = 
        CreatePartition[trainDataSet, 1, 
          10, 
Overlap -> 
            1.0, 
PredType -> "ISEX", 
SetType -> "EXP", 
Border -> 
            False, 
Linguistic -> 
            False
];\)\)
testPreds = testParts[[2]];
testVars = DefPredicateVars[testPreds];

PlotFuzzySet[testParts];

Sugeno controller

Now we can define a rule for each predicates as

(Subscript[p, i]) -> f (CoG ((Subscript[p, i])))

where CoG((Subscript[p, i])) is the center of gravity of the i-th predicate.

testParts[[1, 1]]

interpolRules = 
        Table[
FRule[
            testVars[[i]], 
NVal@f@EvalObject@
                  testParts[[1, 1, i, 2]]], 
{i, 
            Length@testPreds}
];

PrintRules[interpolRules]

For faster computation we will define a temporary object SimpleControlSug containing this rule set as FRuleSetSugeno[].

simpleControlSug = 
        Def["SimpleControlSug", 
          FRuleSetSugeno[interpolRules]];

We can now define the controller as the result of the evaluation of the inference system (FuzzyInference[]) defined by the rule set.

inferenceSystem[x_] := 
        EvalObject@FuzzyInference[simpleControlSug, x];

To see how this controller approximates the desired goal function, we will evaluate it for the given test data set and plot the result.

recalledData = 
        Table[{testData[[i, 1]], inferenceSystem[testData[[i]]]}, {i, 
            Length@testData}];

plotOrig = 
        Plot[f[x], {x, 0, range}, DisplayFunction -> Identity, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0.7]}];
plotRecall = 
        ListPlot[recalledData, DisplayFunction -> Identity, 
          PlotJoined -> True, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0]}];
Show[plotOrig, Graphics@plotRecall, 
        DisplayFunction -> $DisplayFunction, 
        AspectRatio -> 0.2];

We can see the controller performs very well—except for the borders.

To compare the result, we will finally compute the average square error of the approximation.

Print["Average Squared Error: ", 
      Tr@Table[(recalledData[[x, 2]] - f[x - 1])^ 2, {x, 1, 
              Length@recalledData}]/(Length@recalledData + 1)]

Using SOM

We will now use a SOM (self-organizing map) to create a prediction function:

Options@CreateSOM

As the SOM algorithm normalizes the variances of all dimensions to 1, we have to use a higher weight for the first parameter.
Tip! Try using different weights and see what happens.

som = 
        CreateSOM[trainData, Size -> {1, 10}, 
          Weights -> {5, 1}];

The SOM interpolates the goal function with 10 supporting points in a nonlinear way.

GetData@som

som[{{15, Null}}]

We can recall the functions values using the SOM in a straightforward way.

recallData = 
        Table[{x, som[{{x, Null}}, NSize -> 3][[1, 2]]}, {x, 0, 
            range}];

To see how well our initial function is approximated, we create a plot.

plotOrig = 
        Plot[f[x], {x, 0, range}, DisplayFunction -> Identity, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0.7]}];
plotSOM = 
        PlotSOMGrid[som, Dims -> {1, 2}, PointSize -> 0.02, 
          DefaultColor -> Hue[0.1], 
          Thickness -> 0.005];
plotRecall = 
        ListPlot[recallData, DisplayFunction -> Identity, 
          PlotJoined -> True, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0]}];
Show[plotOrig, Graphics@plotSOM, Graphics@plotRecall, 
        DisplayFunction -> $DisplayFunction, 
        AspectRatio -> 0.2];

The average squared error is a good measure of how close our approximation is.

Print["Average Squared Error: ", 
      Tr@Table[(recallData[[x, 2]] - f[x - 1])^ 2, {x, 1, 
              Length@recallData}]/(Length@recallData + 1)]

Using a decision tree

Create a decision tree using ID3

To create a decision tree for the approximation problem, we will first define the test predicates.

testPreds = 
        CreatePredicates[trainDataSet, 1, 8, Overlap -> 1.0, 
          SetType -> "EXP", Border -> False, 
          Linguistic -> False];
goalPreds = 
        CreatePredicates[trainDataSet, 2, 3, Overlap -> 1.0, 
          Border -> True, 
          PredType -> "ISEX"];

    
testVars = DefPredicateVars[testPreds];
goalVars = DefPredicateVars[goalPreds];

PlotFuzzySet[Union@testPreds[[All, 2, 3]]];

PlotFuzzySet[goalPreds[[All, 2, 3]]];

If you use non-PWL membership functions, it might be necessary to increase the minimal entropy to 0.05.

id3tree = 
        CreateID3[trainDataSet, testVars, goalVars, MinEnt -> 0.05, 
          Logic -> LogicP, MinConf -> 0.8];

Show[
        Graphics[PlotID3[id3tree], AspectRatio -> 0.5]];

Sugeno controller

To create a controller using the previously computed decision tree, we will use the MakeSugenoInference[] command, which evaluates the given tree with respect to the specified set of goal predicates.

You can select different parameters to tune the evaluation process. We will use a majority-based evaluation first, which means that each node has only one goal class assigned.

testData = List /@ Range[0, 100];
recalledData = 
        MakeSugenoInference[id3tree, goalPreds, testData, 
          Type -> "MAJOR", Logic -> LogicP];

resultData = 
        Table[{testData[[i, 1]], recalledData[[i]]}, {i, 
            Length@testData}];

plotOrig = 
        Plot[f[x], {x, 0, range}, DisplayFunction -> Identity, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0.7]}];
plotRecall = 
        ListPlot[resultData, DisplayFunction -> Identity, 
          PlotJoined -> True, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0]}];
Show[plotOrig, Graphics@plotRecall, 
        DisplayFunction -> $DisplayFunction, 
        AspectRatio -> 0.2];

Print["Average Squared Error: ", 
      Tr@Table[(recalledData[[x]] - f[x - 1])^ 2, {x, 1, 
              Length@recalledData}]/(Length@recalledData + 1)]

We will now try a probabilistic evaluation, where a tree node can belong to more goal classes with a certain probability.

testData = List /@ Range[0, 100];
recalledData = 
        MakeSugenoInference[id3tree, goalPreds, testData, Type -> "PROB", 
          Logic -> LogicP];

resultData = 
        Table[{testData[[i, 1]], recalledData[[i]]}, {i, 
            Length@testData}];

plotOrig = 
        Plot[f[x], {x, 0, range}, DisplayFunction -> Identity, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0.7]}];
plotRecall = 
        ListPlot[resultData, DisplayFunction -> Identity, 
          PlotJoined -> True, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0]}];
Show[plotOrig, Graphics@plotRecall, 
        DisplayFunction -> $DisplayFunction, 
        AspectRatio -> 0.2];

Print["Average Squared Error: ", 
      Tr@Table[(recalledData[[x]] - f[x - 1])^ 2, {x, 1, 
              Length@recalledData}]/(Length@recalledData + 1)]

Mamdani controller (Test)

We will now take a look at a Mamdani controller. For a Mamdani controller a rule consists of a condition and a conclusion, where the conclusion is an (arbitrary) fuzzy set. Correctly, only piece-wise linear fuzzy sets are supported for the right-hand side.

To evaluate a decision tree using a Mamdani controller, we simply call MakeMamdaniInference[].

testData = List /@ Range[0, 100];
recalledData = 
        MakeMamdaniInference[id3tree, goalPreds, testData, 
          Type -> "PROB", Logic -> LogicP];

resultData = 
        Table[{testData[[i, 1]], recalledData[[i]]}, {i, 
            Length@testData}];

fuzzySet = 
        Table[PlotFuzzySet[testPreds[[i, 2, 3]], 
            DisplayFunction -> Identity], {i, 
            Length@testPreds}];

plotOrig = 
        Plot[f[x], {x, 0, range}, DisplayFunction -> Identity, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0.7]}];
plotRecall = 
        ListPlot[resultData, DisplayFunction -> Identity, 
          PlotJoined -> True, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0]}];
Show[plotOrig, plotRecall, DisplayFunction -> $DisplayFunction, 
        AspectRatio -> 0.2];

Print["Average Squared Error: ", 
      Tr@Table[(recalledData[[x]] - f[x - 1])^ 2, {x, 1, 
              Length@recalledData}]/(Length@recalledData + 1)]

Using a rule base

Compute rule set

To find a set of independent rules for the given approximation problem, we will now apply the FS-FOIL rule induction method.

Options@CreatePartition

testPreds = 
        CreatePredicates[trainDataSet, 1, 8, Overlap -> 1.0, 
          Border -> True, SetType -> "EXP", 
          Linguistic -> False];
goalPreds = 
        CreatePredicates[trainDataSet, 2, 3, Overlap -> 1.0, 
          Border -> True, 
          PredType -> "IS"];

    
testVars = DefPredicateVars[testPreds];
goalVars = DefPredicateVars[goalPreds];

PlotFuzzySet[Union@testPreds[[All, 2, 3]]];

PlotFuzzySet[goalPreds[[All, 2, 3]]];

Options@CreateFOIL

foilRules = 
        CreateFOIL[trainDataSet, testVars, goalVars, MinConf -> 0.6, 
          Logic -> LogicL];

PrintFOILRules[foilRules, Info -> False]

Sugeno controller

To apply the obtained set of rules on the test data, we can use the MakeSugenoInference[] command again.

testData = List /@ Range[0, 100];
recalledData = 
        MakeSugenoInference[foilRules, goalPreds, testData, 
          Logic -> LogicP];

resultData = 
        Table[{testData[[i, 1]], 
            recalledData[[i]] /. Null -> 0}, {i, 
            Length@testData}];

fuzzySet = {};

plotOrig = 
        Plot[f[x], {x, 0, range}, DisplayFunction -> Identity, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0.7]}];
plotRecall = 
        ListPlot[resultData, DisplayFunction -> Identity, 
          PlotJoined -> True, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0]}];
Show[fuzzySet, plotOrig, Graphics@plotRecall, 
        DisplayFunction -> $DisplayFunction, 
        AspectRatio -> 0.2];

Count[recalledData, NAN]

Print["Average Squared Error: ", 
      Tr@Table[(recalledData[[x]] - f[x - 1])^ 2, {x, 1, 
              Length@recalledData}]/(Length@recalledData + 1)]

RENO

Training a controller

The third method we will apply to solve the approximation problem combines (fuzzy) rule induction and numerical optimization to find the best solution. As the underlying sets of the fuzzy controller are tuned by the system, it is not necessary to initialize the predicates in advance.

Currently this method is limited to a small number of dimensions (< 6) and the obtained rule sets can become quite big. The approximation found with this method, however, is very accurate.

Options@CreateRENO

renoController  =  
        CreateRENO[trainDataSet, Dims -> {1}, Goal -> 2, 
          Controller -> 0,  NDims -> {6}, Orders -> {3},  
          StartingKnots -> 2];

PrintRules[renoController]

Sugeno controller

To apply the obtained set of rules on the test data, we can use the MakeSugenoInference[] command again.

testData = List /@ Range[0, 100];
recalledData = 
        MakeSugenoInference[renoController, testData];

resultData = 
        Table[{testData[[i, 1]], 
            recalledData[[i]] /. NAN -> 0}, {i, 
            Length@testData}];

plotOrig = 
        Plot[f[x], {x, 0, range}, DisplayFunction -> Identity, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0.7]}];
plotRecall = 
        ListPlot[resultData, DisplayFunction -> Identity, 
          PlotJoined -> True, 
          PlotStyle -> {Thickness[0.005], 
              Hue[0]}];
Show[plotOrig, Graphics@plotRecall, 
        DisplayFunction -> $DisplayFunction, 
        AspectRatio -> 0.2];

Print["Average Squared Error: ", 
      Tr@Table[(recalledData[[x]] - f[x - 1])^ 2, {x, 1, 
              Length@recalledData}]/(Length@recalledData + 1)]

We can see that the quality of the obtained model is much higher then from the other methods.

A more complex 2-dimensional example

In this example we will show how a two-dimensional function can be approximated using various methods.
The methods are similar to the one-dimensional example and will not be explained in detail again.

SetUp data

First we have to define the goal function and its range.

range = 100;
f[x_, y_] := 
        N@(Sin[x/range*2*Pi]*y);
Plot3D[f[x, y], {x, 0, range}, {y, 0, range}];

Now we can store a number of sample points in a matrix that is later used for training.

samples = 1000;
trainData = 
        Table[{x = Random[]*range, y = Random[]*range, 
            f[x, y]}, {samples}];

Min[trainData[[All, 1]]]

Show[Graphics3D@Map[Point, trainData], AspectRatio -> 0.5, 
        Axes -> True];

trainDataSet = 
        Def["DataSet", 
          DataSet[trainData, {"x", "y", "z"}]];

Using a decision tree

Create a decision tree using ID3

Options@CreatePartition

testPredsSep = {
CreatePredicates[trainDataSet, 1, 
            6, Width -> 1.0, Border -> True, 
            SetType -> "EXP"], 
CreatePredicates[
            trainDataSet, 2, 2, Overlap -> 0.7, Border -> True, 
            SetType -> "EXP"]
};
testPreds = Flatten[testPredsSep, 1];
goalPreds = 
        CreatePredicates[trainDataSet, 3, 3, Width -> 1.0, 
          Border -> True, PredType -> "ISEX", 
          SetType -> "PWL"];

    
testVars = DefPredicateVars[testPreds];
goalVars = DefPredicateVars[goalPreds];

PlotFuzzySetHistogram[testPredsSep[[1, All, 2, 3]], 
        GetData[trainDataSet, All, 1], HistogramScale -> 50, 
        HistogramCategories -> 20, PlotLabel -> "X1"];

PlotFuzzySetHistogram[testPredsSep[[2, All, 2, 3]], 
        GetData[trainDataSet, All, 2], HistogramCategories -> 10, 
        HistogramScale -> 50, PlotLabel -> "X2"];

PlotFuzzySetHistogram[goalPreds[[All, 2, 3]], 
        GetData[trainDataSet, All, 3], HistogramScale -> 500, 
        PlotLabel -> "X3"];

If you use non-PWL membership functions, it might be necessary to increase the minimal entropy to 0.05.

Options@CreateID3

id3tree = 
        CreateID3[trainDataSet, testVars, goalVars, MinConf -> 0.8, 
          MinEnt -> 0.01, MinIncr -> 0.01, Logic -> LogicL, 
          MaxLevel -> 4];

Show[
        Graphics[{PlotClassLegend[goalVars], PlotID3[id3tree]}, 
          AspectRatio -> 0.7]];

Sugeno controller

samplesTest = 20;
testData = 
        Flatten[Table[
            Table[{x*range/samplesTest, 
                y*range/samplesTest}, 
{x, 0, 
                samplesTest}], {y, 0, samplesTest}], 
          1];
recalledData = 
        MakeSugenoInference[id3tree, goalPreds, testData, Type -> "PROB", 
          Logic -> LogicP];

Count[recalledData, Null]

sumError = 
        Tr@Map[((f[#[[1]], #[[2]]] - #[[3]])^ 2 &), 
            Transpose@
              Append[Transpose@testData, 
                recalledData /. Null -> 0]];

Print["Average Squared Error: ", sumError/(Length@
                recalledData + 1)/(Max@testData - 
              Min@testData)^2]

trainMatrix = 
        Partition[Map[(f @@ # &), testData], 
          samplesTest + 1];
ListPlot3D[trainMatrix, ColorFunction -> Automatic, 
        ColorOutput -> GrayLevel, 
        Ticks -> {{{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 
                100}}, {{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 100}}, 
            Automatic}, 
PlotLabel -> "Original \
function"];

recallMatrix = 
        Partition[recalledData /. Null -> 0, 
          samplesTest + 1];
ListPlot3D[recallMatrix, ColorFunction -> Automatic, 
        ColorOutput -> GrayLevel, 
        PlotRange -> {Automatic, Automatic, {-100, 100}}, 
        Ticks -> {{{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 
                100}}, {{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 100}}, 
            Automatic}, 
PlotLabel -> "Recalled \
function"];

ListPlot3D[
        trainMatrix - 
          recallMatrix, 
PlotRange -> {Automatic, 
            Automatic, {-100, 100}}, 
        Ticks -> {{{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 
                100}}, {{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 100}}, 
            Automatic}, 
ColorFunction -> Automatic, 
        ColorOutput -> GrayLevel, 
        Axes -> 
          True, 
PlotLabel -> "Error surface \
function"];

cutNr = 15;
ListPlot[recallMatrix[[cutNr]], 
        PlotJoined -> True];
ListPlot[trainMatrix[[cutNr]], 
        PlotJoined -> True];

\(\(Show@
        Graphics[
          PlotInputRecall[Flatten@trainMatrix, Flatten@recallMatrix, 
            PointSize -> 
              0.01], 
AxesLabel -> {"Original", "Recall"}, 
Axes -> True];\)\)

Using a rule base

Compute rule set

Options@CreatePartition

testPredsSep = {
CreatePredicates[trainDataSet, 1, 
            6, Width -> 1.0, Border -> True, 
            SetType -> "EXP"], 
CreatePredicates[
            trainDataSet, 2, 2, Overlap -> 0.7, Border -> True, 
            SetType -> "EXP"]
};
testPreds = Flatten[testPredsSep, 1];
goalPreds = 
        CreatePredicates[trainDataSet, 3, 3, Width -> 1.0, 
          Border -> True, PredType -> "ISEX", 
          SetType -> "PWL"];

    
testVars = DefPredicateVars[testPreds];
goalVars = DefPredicateVars[goalPreds];

PlotFuzzySet[testPredsSep[[1, All, 2, 3]], 
        PlotLabel -> "X1"];

PlotFuzzySet[testPredsSep[[2, All, 2, 3]], 
        PlotLabel -> "X2"];

PlotFuzzySet[goalPreds[[All, 2, 3]], 
        PlotLabel -> "X3"];

Options@CreateFOIL

foilRules = 
        CreateFOIL[trainDataSet, testVars, goalVars, MinConf -> 0.6, 
          MinSup -> 0.05, Logic -> LogicP];

PrintFOILRules[foilRules, Info -> False]

Sugeno controller

samplesTest = 20;
testData = 
        Flatten[Table[
            Table[{x*range/samplesTest, y*range/samplesTest}, {x, 0, 
                samplesTest}], {y, 0, samplesTest}], 
          1];
recalledData = 
        MakeSugenoInference[foilRules, goalPreds, testData, 
          Logic -> LogicP];

Count[recalledData, Null]

sumError = 
        Tr@Map[((f[#[[1]], #[[2]]] - #[[3]])^ 2 &), 
            Transpose@
              Append[Transpose@testData, 
                recalledData /. NAN -> 0]];

Print["Average Squared Error: ", sumError/(Length@
                recalledData + 1)/(Max@recalledData - 
              Min@recalledData)^2]

Options@ListPlot3D

trainMatrix = 
        Partition[Map[(f @@ # &), testData], 
          samplesTest + 1];
ListPlot3D[trainMatrix, ColorFunction -> Automatic, 
        ColorOutput -> GrayLevel, 
        Ticks -> {{{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 
                100}}, {{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 100}}, 
            Automatic}, 
PlotLabel -> "Original \
function"];

recallMatrix = 
        Partition[recalledData /. NAN -> 0, 
          samplesTest + 1];
ListPlot3D[recallMatrix, ColorFunction -> Automatic, 
        ColorOutput -> GrayLevel, 
        PlotRange -> {Automatic, Automatic, {-100, 100}}, 
        Ticks -> {{{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 
                100}}, {{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 100}}, 
            Automatic}, 
PlotLabel -> "Recalled \
function"];

ListPlot3D[
        trainMatrix - 
          recallMatrix, 
PlotRange -> {Automatic, 
            Automatic, {-100, 100}}, 
        Ticks -> {{{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 
                100}}, {{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 100}}, 
            Automatic}, 
ColorFunction -> Automatic, 
        ColorOutput -> GrayLevel, 
        Axes -> 
          True, 
PlotLabel -> "Error surface \
function"];

cutNr = 20;
ListPlot[recallMatrix[[cutNr]], 
        PlotJoined -> True];
ListPlot[trainMatrix[[cutNr]], 
        PlotJoined -> True];

Show@
        Graphics[
          PlotInputRecall[trainMatrix[[cutNr]], 
            recallMatrix[[cutNr]], PointSize -> 0.01], 
          Axes -> True];

Using RENO

Compute Sugeno controller

Options@CreateRENO

reno = 
        CreateRENO[trainDataSet, Dims -> {1, 2}, Goal -> 3, 
          Orders -> {3, 3}];

reno[[1, 1, 1]]

PrintRules[reno]

Sugeno controller

samplesTest = 20;
testData = 
        Flatten[Table[
            Table[{x*range/samplesTest, y*range/samplesTest}, {x, 0, 
                samplesTest}], {y, 0, samplesTest}], 
          1];
recalledData = MakeSugenoInference[reno, testData];

sumError = 
        Tr@Map[((f[#[[1]], #[[2]]] - #[[3]])^ 2 &), 
            Transpose@Append[Transpose@testData, recalledData]];

Print["Average Squared Error: ", sumError/(Length@
                recalledData + 1)/(Max@recalledData - 
              Min@recalledData)^2]

trainMatrix = 
        Partition[Map[(f @@ # &), testData], 
          samplesTest + 1];
ListPlot3D[trainMatrix, ColorFunction -> Automatic, 
        ColorOutput -> GrayLevel, 
        Ticks -> {{{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 
                100}}, {{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 100}}, 
            Automatic}, 
PlotLabel -> "Original \
function"];

recallMatrix = 
        Partition[recalledData /. NAN -> 0, 
          samplesTest + 1];
ListPlot3D[recallMatrix, ColorFunction -> Automatic, 
        ColorOutput -> GrayLevel, 
        PlotRange -> {Automatic, Automatic, {-100, 100}}, 
        Ticks -> {{{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 
                100}}, {{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 100}}, 
            Automatic}, 
PlotLabel -> "Recalled \
function"];

ListPlot3D[
        trainMatrix - 
          recallMatrix, 
PlotRange -> {Automatic, 
            Automatic, {-100, 100}}, 
        Ticks -> {{{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 
                100}}, {{5, 25}, {10, 50}, {10, 50}, {15, 75}, {20, 100}}, 
            Automatic}, 
ColorFunction -> Automatic, 
        ColorOutput -> GrayLevel, 
        Axes -> 
          True, 
PlotLabel -> "Error surface \
function"];

cutNr = 15;
ListPlot[recallMatrix[[cutNr]], 
        PlotJoined -> True];
ListPlot[trainMatrix[[cutNr]], 
        PlotJoined -> True];

Show@
        Graphics[
          PlotInputRecall[trainMatrix[[cutNr]], 
            recallMatrix[[cutNr]], PointSize -> 0.01], 
          Axes -> True];