netAn3: Other Graphical Representations of Neural Network Behavior (bar and density plots), Notebook Version: 2.0
Steve Hunka, University of Alberta, steve.hunka@ualberta.ca
Mod. Date: Oct. 30/97; correction to function hwaxv

General Description of Notebook

The Notebook contains the following plotting functions:
1) array barplot of hidden cell responses
2) array barplot of output cell responses
3) density plot of responses (exemplar) by (exemplar attributes:hidden cells:outputcells:answers)
4) density plot of weights; (exemplars:hidden cells) by (exemplar attributes)
5) density plot of weighted attribute components of selected hidden cell
Data input to these functions are obtained from the files written by the neural network function bpnn and from the data matrix input to bpnn.Only data of the last epoch (final network state) is used. The examples use file TCLast written by the function bpnn in Neural.nb.

Functions in Notebook

hdrbarplot: for bar plot of hidden response for each exemplar
otrbarplot:  bar plot of output cell response for each exemplar
rspbarplot:  called by hrbarplot and otrbarplot to plot bars
araydis:     to arrange plots of mplotLists into a specified array format
araydis3:  for structuring array plot of bars 
dpltresp:   density plot of responses; exemplars by (attribute:hid cell:output
               cell:answer)
dpltwts:    density plot of weights; attributes by (exemplars:hidden cells)
hwxav:    density plots of the product values of hidden weights times attribute input values
logistic:   to convert weight to a 0 to 1 scale
clr3:        a RGB color function used by hwxav

Note: Functions hdrbarplot, otrbarplot, and rspbarplot are also in the Notebook netCls.nb

The following functions are provided by the Package netUtil.nb and documented in netUdoc.nb:
kbin3      input of numeric information from keyboard
kbinstrg  input of string information from keyboard
inpvec    checks the range of elements in a vector
pmat2     to display matrix
genrch    to generate row/column headings for pmat2
rndoff     round off function

Packages Required

<< "Graphics`Graphics`"<< "Graphics`MultipleListPlot`"<< "netUtil`"

araydis[m,in]; arranges the plots in m for GraphicsArray according to the arrangement specified by in. For example if m had 9 plots, and in={2,3,4} then the 1st row will contain 2 plots, the next 3 plots, and the last row 4 plots as displayed by Graphics Array. 
Length[in] <= Length[m].
Mod. Date: Feb 20/97

araydis[m_, in_] := 
      Module[{pa, ma, nt}, pa = {}; nt = Length[in]; ma = m; 
        Do[AppendTo[pa, 
            Take[ma, in[[i]]]]; 
          ma = Drop[ma, in[[i]]]; 
          , {i, 1, nt}]; Return[pa]; ];

logistic[p] returns p transformed by logistic function
Mod. Date: Mar 21/96

logistic[p_] := (1/(1 + (E^(-p))));

otrbarplot[d] creates a bar plot of the output cell response values for each exemplar. X-axis is output cell number; Y-axis is response value
d  is the matrix of output cell response values of order (#outcells,#exemplars)
User Input: See rspbarplot
Calls: rspbarplot, kbinstrg, kbin3, araydis
Mod. Date: Mar 11/97

otrbarplot[d_] := 
      Module[{nex}, 
        Print["Bar Plot of Output cell responses for each exemplar"]; 
        Print["Input data dimensions=", Dimensions[d]]; 
        Print["Number of output cells=", Length[d]]; 
        nex = Dimensions[d][[2]]; 
        Print["Number of exemplars=", nex]; 
        rspbarplot[d, nex, {"OutCell", "Resp"}]; ];

hdrbarplot[d] creates a bar plot of the hidden cell response values for each exemplar. X-axis is hidden cell number; Y-axis is response value
d  is the matrix of hidden cell response values of order (#exemplars,#hidden cells)
User Input: See rspbarplot
Calls: rspbarplot, kbinstrg, kbin3, araydis
Mod. Date: Mar 11/97

hdrbarplot[d_] := 
      Module[{nex}, 
        Print["Bar Plot of Hidden cell responses for each exemplar"]; 
        Print["Input data dimensions=", Dimensions[d]]; 
        Print["Number of hidden cells=", 
          Dimensions[d][[2]]]; 
        nex = Part[Length[d]]; Print["Number of exemplars=", nex]; 
        rspbarplot[Transpose[d], nex, {"HidCell", "Resp"}]; ];

rspbarplot[d,nx,xat] plots cell response to each exemplar as a bar chart. 
d is the matrix of output responses of order (#output cells or #hidden cells,#exemplars).
nx is the number of exemplars
xat contains x and y axis labels of the form {"xaxis","yaxis"}
X-axis is 1 to the number of cells; Y-axis is the value of the response.  Individual plots of all or selected exemplars can be made as well as an array plot.
Called by: hdrbarplot, otrbarplot
User Input:
a) plot all exemplars (y/n); if n plot selected exemplars
b) single plots for exemplars (y/n)
c) group single plots into an array (y/n)
d) if array plot selected, a vector of the number of plots per row
e) if selected exemplars plotted, a vector of the exemplar indices
Notes: The adequacy of the numbers on the X-axis will depend upon the number of cells. The font size of the X-axis digits is set in DefaultFont as 8. The font size of the PlotLabel is set to 12. These font values may need to be changed by the user to suite a particular plotting case. 
Calls: kbinstrg, kbin3, araydis,BarChart
Mod. Date: Mar 25/97

rspbarplot[d_, nx_, xat_] := 
      Module[{dsl, neplt, nex, exsl, pobj, sp, ap, i, err, npr}, 
        If["y" == 
            kbinstrg["Plot all exemplars? y/n", {{"y"}, {"n"}}], 
          Print["Plot all exemplars=y"]; dsl = Transpose[d]; neplt = nx; 
          exsl = Range[nx]; , Print["Plot all exemplars=n"]; 
          neplt = kbin3["How many exemplars to plot?", 1]; 
          exsl = inpvec[{1, nx}, neplt, "exemplars"]; 
          Print["Exemplars to be plotted=", exsl]; 
          dsl = Transpose[d][[exsl
              ]]; ]; 
        If["y" == 
            kbinstrg[
              "Single plots for each exemplar? y/n", {{"y"}, {
                  "n"}}], sp = $DisplayFunction; 
          Print["Single plots=y"], sp = Identity; 
          Print["Single plots=n"]]; 
        If["y" == 
            kbinstrg[
              "Group single plots into array? y/n", {{"y"}, {
                  "n"}}], ap = 1; Print["Group in array plot=y"], 
          ap = 0; Print["Group in array plot=n"]]; i = 0; 
        pobj = (
              BarChart[#1, AxesLabel -> xat, 
                  DefaultFont -> {"Times", 8}, 
                  DisplayFunction -> sp, PlotRange -> All, 
                  PlotLabel -> 
                    FontForm[
                      StringInsert["Exemplar ", 
                        ToString[
                          exsl[[i = 
                              i + 1]]], -1], {
                        "Times", 12}]]&)/@dsl; 
        If[1 == ap, err = 1; 
          While[err == 1, 
            npr = Input["Type vector of plots per array row"]; 
            If[nx < Plus@@npr, 
              Print["Error: sum plots per row > # plots"], err = 0]; ]; 
          Show[GraphicsArray[araydis[pobj, npr]], 
            DisplayFunction -> $DisplayFunction]]; ];

Data for otrbarplot and hdrbarplot

Final state data may be read from individual files by using <<filename, or by reading the data from the file containing only the final state data and the random seed. In the example below we first get a report of the contents of the file "TCLast", then using the data ID and date-time stamp, the specific data required are recovered using the function rcvrExpr

rprtFdata

Now recover the last epoch hidden cell response data from file TCLast.

tclhcr = rcvrExpr["TCL953hdr", 19971022214210];

hdrbarplot[tclhcr]

outr = rcvrExpr["TCL953rsp", 19971022214210];

otrbarplot[outr]

dpltresp[e,hr,otr] Density plot of exemplar attributes together with hidden cell responses, output cell responses, and target answers.
e: is the data matrix input to the network function bpnn; it is a structure for
   which Part[e,1] is of order (#exemplar,#attributes),and Part[e,2] is of order
    (#outcells,#exemplars) 
hr: is the matrix of hidden cell responses of order (#exemplars,#hidden cells); 
otr: is the output cell responses of order (#exemplars, #output cells). 
Density Display: 
a) each row represents an exemplar; the uppermost row is a color legend
b) the first i columns represent exemplar attributes (i=number of attributes)
c) the next j columns represent hidden cell responses (j=#hidden cells)
d) the next k columns represent output cell responses (k=#output cells)
e) the next k columns represent the target or correct answers at each output cell
To assist the user interpret columns the display attr(i):hdc(j):otc(k):ans(k) is displayed indicating the order and number of columns associated with the exemplar attributes, hidden cell responses, output cell responses, and answers. Since network responses are bounded by 0 and 1.0, no
transformation is made of the data. The values corresponding to the color legend are output. If 
the density plot is to be printed by a black/white printer, gray scale should be used.
Options: The user may select a color (c) or gray (g) scale density plot
Calls: kbinstrg, ListDensityPlot
Mod. Date: May 1/97

dpltresp[e_, hr_, otr_] := 
      Module[{comb3, dc, cs = 1, nat, nhc, noc, fl, lgnd}, 
        Print["Response to exemplars attributes by Hidden & Output \
cells"]; 
        Print["Dimensions exemplars attributes=", 
          Dimensions[e[[1]]]]; 
        Print["Dimensions answers=", 
          Dimensions[e[[2]]]]; 
        Print["Dimensions hidden responses=", Dimensions[hr]]; 
        Print["Dimensions out cell responses=", Dimensions[otr]]; 
        Print["Number of exemplars=", 
          Dimensions[hr][[1]]]; 
        nat = Dimensions[e[[1]]]
              [[2]]; 
        Print["Number of attributes=", nat]; 
        nhc = Dimensions[hr][[2]]; 
        Print["Number of hidden cells=", nhc]; 
        noc = Dimensions[otr][[1]]; 
        Print["Number of output cells=", noc]; 
        Print["Uppermost row is a legend low->high"]; 
        If["g" == 
            kbinstrg[
              "Color or Grey scale plot? c/g", {{"c"}, {"g"}}], 
          cs = 0]; 
        comb3 = Join[
            Join[Transpose[e[[1]]], 
              Join[Transpose[hr], otr]], 
            e[[2]]]; 
        dc = Dimensions[comb3][[1]]; 
        lgnd = Table[((-1 + Range[dc])/dc)]; 
        Print["Legend values=", rndoff[lgnd, 2]]; 
        comb3 = Append[Transpose[comb3], lgnd]; 
        fl = "attr():hdc():otc():ans()"; 
        fl = StringInsert[fl, ToString[nat], -20]; 
        fl = StringInsert[fl, ToString[nhc], -14]; 
        fl = StringInsert[fl, ToString[noc], -8]; 
        fl = StringInsert[fl, ToString[noc], -2]; 
        fl = Join[{fl}, {"exemplars"}]; 
        ListDensityPlot[comb3, FrameLabel -> fl, 
          If[cs == 1, ColorFunction -> (Hue[#1, #1, #(1^0.5)]&), 
            ColorFunction -> GrayLevel], 
          PlotLabel -> "Response to Exemplars"]; ];

Test data for functions dpltresp and dpltwts

The structure tcL is the data matrix input to the network simulation function bpnn. The last epoch hidden responses are read from file TCLhr; the last epoch output cell responses are read from file TCLor; the last epoch hidden weights are read from file TCLhw.

tcL = {{{ .9,  .9,  .9,  .1,  .9,  .1,  .1,  .9,  .1}, { .1,  .1, 
             .9,  .9,  .9,  .9,  .1,  .1,  .9}, { .1,  .9,  .1,  .1,  .9, 
             .1,  .9,  .9,  .9}, { .9,  .1,  .1,  .9,  .9,  .9,  .9,  .1, 
             .1}, { .9,  .9,  .9,  .9,  .1,  .1,  .9,  .9,  .9}, { .9,  .9, 
             .9,  .9,  .1,  .9,  .9,  .1,  .9}, { .9,  .9,  .9,  .1,  .1, 
             .9,  .9,  .9,  .9}, { .9,  .1,  .9,  .9,  .1,  .9,  .9,  .9, 
             .9}, { .9,  .1,  .1,  .9,  .1,  .1,  .9,  .9,  .9}, { .9,  .9, 
             .9,  .9,  .1,  .1,  .9,  .1,  .1}, { .9,  .9,  .9,  .1,  .1, 
             .9,  .1,  .1,  .9}, { .1,  .1,  .9,  .1,  .1,  .9,  .9,  .9, 
             .9}}, {{ .9,  .9,  .9,  .9,  .1,  .1,  .1,  .1,  .1,  .1,  .1, 
             .1}, { .1,  .1,  .1,  .1,  .9,  .9,  .9,  .9,  .1,  .1,  .1, 
             .1}, { .1,  .1,  .1,  .1,  .1,  .1,  .1,  .1,  .9,  .9,  .9, 
             .9}}};

Dimensions[tcL]

\(tclhr = rcvrExpr["TCL953hdr", 19971022214210]; 

    Dimensions[tclhr]\)

tclor = rcvrExpr["TCL953rsp", 19971022214210]; 
    Dimensions[tclor]

dpltresp[tcL, tclhr, tclor]

Explanaton of above density plot

In row 1, the first 9 colors represent the encoding of the character T, followed by 5 colored boxes representing the response from hidden cells 1 to 5, then 3 colored boxes representing the output from output cells 1 to 3, and the last 3 boxes representing the desired or target answer. Row 2 starts with the representation of the character T rotated clockwise 90 degrees and the boxes following in this row are interpreted as for the first row. Thus, the input and the final state of the hidden and output cell response as well as the target answer are all represented in the density plot.

dpltwts[e,hw] gives a density plot of attributes by exemplars+hidden wts. 
e is the input to the neural net and is a structure in which Part[e,1] is of order (#exemplars, #attributes), and Part[e,2] holds the answers of order (#outcells, #exemplars). 
hw is the hidden cell weight matrix of order (#hidden cells, #input cells). 
Density Display:
Columns of the density plot represent input cells. The first j rows (j=number of exemplars) are the exemplars, the next k rows refer to the k hidden cells. The uppermost row is a legend bounded by 0 and +1. Note that in transforming the weights by the logistic function, "large" 
negative weights approach 0, while the value of 0 is 0.5.  The matrix e is not rescaled by 
the logistic function.
Options: user can select a color (c) or gray (g) scale density plot.
Calls: kbinstrg, logistic (to scale wt values between 0 and 1), ListDensityPlot
Date: May 2/97

dpltwts[e_, hw_] := 
      Module[{ans, comb1, clrf, cs = 1, lgnd}, 
        Print["Hidden Cell weighting of Attributes"]; 
        Print["Dimensions exemplars attributes=", 
          Dimensions[e[[1]]]]; 
        Print["Dimensions answers=", 
          Dimensions[e[[2]]]]; 
        Print["Dimensions of wt matrix=", Dimensions[hw]]; 
        Print["Number Hidden cells=", 
          Dimensions[hw][[1]]]; 
        Print["Number Input cells=", 
          Dimensions[hw][[2]]]; 
        Print["Number of exemplars=", 
          Dimensions[e[[1]]]
              [[1]]]; 
        Print["Number of attributes=", 
          Dimensions[e[[1]]]
              [[2]]]; 
        Print["Min hidden wts=", Min[hw]]; 
        Print["Logistic(min)=", logistic[Min[hw]]]; 
        Print["Max hidden wts=", Max[hw]]; 
        Print["Logistic(max)=", logistic[Max[hw]]]; 
        Print["Uppermost row is legend low->high"]; 
        If["g" == 
            kbinstrg[
              "Color or Grey scale plot? c/g", {{"c"}, {"g"}}], 
          cs = 0]; 
        comb1 = logistic[
            Join[e[[1]], hw]]; 
        dc = Dimensions[comb1][[2]]; 
        lgnd = ((-1 + Range[dc])/dc); 
        Print["Legend values=", N[lgnd]]; comb1 = Append[comb1, lgnd]; 
        ListDensityPlot[comb1, 
          FrameLabel -> {"attributes", "exemplars+hidc"}, 
          If[cs == 1, ColorFunction -> (Hue[#1, #1, #(1^0.5)]&), 
            ColorFunction -> GrayLevel], 
          PlotLabel -> "Attribute Weighting"]; ];

\(tclhw = rcvrExpr["TCL953hwt", 19971022214210]; 

    Dimensions[tclhw]\)

dpltwts[tcL, tclhw]

Notes to the above density plot

Row 1 represents the encoding of the character T without rotation; row 2 is T rotated 90 degrees clockwise. Rows 13 to 17 represent the 5 hidden cells with the colored boxes representing the weights assigned to each exemplar attribute. For example, hidden cell 1 (row 13) gives a high weight (bright red) to attribute 5 (center cell of the 3x3  character matrix) of the exemplars.

Analysis of Hidden Wts x Input attributes

hwxa[h,x] calculates the weighted attribute components of a user selected hidden cell for each exemplar and displays the result as one or more density plots.
h: is the hidden weight matrices at last epoch of order (#hidden cells,#input cells)
x: is the input exemplar matrix used as input to the neural network function bpnn and is 
    of order {{#exemplar,#attributes},{#outcells,#exemplars}}. (Only the Part 1 is used.)
The hidden cell response is calculated as Logistic[Sum(W1X1,W2X2,....WjXk)] and hwxav
calculates each product WjXk for a selected hidden cell. The user may by option 
a)  display this matrix in its raw form or transformed by the logistic function
b)  display the resulting density plot  (#exemplars,#input cells (attributes))
c)  restructure each row (exemplars) separately of the matrix into an array from which the user can
d)  displaythe density plot of each such array individually, or collectively as a graphics array
e)  select another hidden cell for analysis. 
Notes: Option (a) is provided because the matrix of components is rescaled to the range of
0 to 1.0 before it is density plotted. This was done to ensure that the automatic scaling of the
data by ListDensityPlot is comparable over different ranges of data. Item (b) is optional because
the number of input cells may be very large. If the number of input cells is very large, option (c) 
allows restructuring each row into a separate density graphic. For example, if the number of 
input cells equal 1000 representing input from a pattern matrix of order (20,50), the user could 
restructure each exemplar density display to (20,50), rather than displaying under option (b)
a density matrix of order (#exemplars,1000). 
The user may wish to vary the cutting points for RGB color rendition in function clr3.
 The hidden cell response is calculated as a check since it should agree with that output by the
 function bpnn. This value is used as a PlotLabel for the row restructured density graphics. 
Calls: pmat2, rndoff,araydis,genrch,clr3,kbin3,kbinstrg, ListDensityPlot
Requires: netUtil
Mod. Date: Oct 16/97 correction to code;defn of rchd moved before If statement

\(\(hwxav[h_, x_] := 
      Module[{xa, nhc, nic, nx, shc, rchd, errf, agn, newwxc, wxc, sm, 
          neword, dspf, pobj, ppr}, 
        Print["Contribution of Exemplar attributes to Hc output"]; 
        xa = x[[1]]; 
        Print["Dimensions exemplar input=", Dimensions[xa]]; 
        nhc = \(Dimensions[h]\)[[1]]; 
        Print["Number hidden cells=", nhc]; 
        nic = \(Dimensions[h]\)[[2]]; 
        Print["Number of input cells (attributes)=", nic]; 
        nx = \(Dimensions[xa]\)[[1]]; 
        Print["Number of exemplars=", nx]; 
        If[nic ≠ 
            \(Dimensions[xa]\)[[2]], 
          Print["Error:h not conformable with x"]; Return[]]; 
        fn1[a_, b_] := a\ b; agn = True; 
        While[agn == True, shc = kbin3["Which hidden cell to use?", 0]; 
          Print["
Selected hidden cell=", shc]; 
          wxc = \((
                fn1[h[[shc]], 
                    xa[[#1]]]&)\)/@
              Range[nx]; 
          Print["Dimensions of product matrix", Dimensions[wxc]]; 
          If["y" == 
              kbinstrg[
                "Display product matrix? y/n", {{"y"}, {"n"}}], 

            				rchd = genrch["Ex", "Ic", Dimensions[wxc]]; 

            				If[
              "y" == 
                kbinstrg[
                  "Logistic transform on product matrix? y/n", {{
                      "y"}, {"n"}}], 
              Print["
Logistic:Product matrix for hidden cell=", shc]; 
              \(pmat2[rndoff[logistic[wxc], 3], 5, rchd]; \), 
              Print["
Product matrix for hidden cell=", shc]; 
              pmat2[rndoff[wxc, 3], 5, rchd]; ]; ]; 
          Print["
Calculated Hidden Cell Response"]; 
          sm = logistic[Plus@@Transpose[wxc]]; 
          rchd = genrch["Ex", "A", Reverse[Dimensions[{sm}]]]; 
          rchd = ReplacePart[
              rchd, {StringInsert["Hc", ToString[shc], \(-1\)]}, 2]; 
          pmat2[Transpose[{rndoff[sm, 3]}], 5, rchd]; wxc = logistic[wxc]; 
          If["y" == 
              kbinstrg[
                "Display Exemplar by Attribute Density Plot? y/n", {{
                    "y"}, {"n"}}], 
            \(ListDensityPlot[wxc, 
              FrameLabel -> {"attributes", "Exemplars"}, 
              ColorFunction -> \((clr3[#1]&)\), 
              PlotRange -> {0.0, 1.0}, 
              PlotLabel -> 
                StringInsert["Hidden Cell=", ToString[shc], \(-1\)]]; 
            \)]; If["y" == 
              kbinstrg[
                "Restructure each row of density plot? y/n", {{"y"}, {
                    "n"}}], errf = 1; 
            While[errf == 1, neword = kbin3["Type number of columns", 0]; 
              If[0 ≠ Mod[nic, neword], 
                Print[neword, ": Error:number of columns"], errf = 0]]; 
            If["y" == 
                kbinstrg[
                  "Display each individual plot? y/n", {{"y"}, {
                      "n"}}], dspf = $DisplayFunction, dspf = Identity]; 
            pobj = {}; 
            Do[newwxc = 
                Reverse[
                  Partition[wxc[[j]], 
                    neword]]; 
              \(AppendTo[pobj, 
                ListDensityPlot[newwxc, DisplayFunction -> dspf, 
                  FrameTicks -> None, 
                  ColorFunction -> \((clr3[#1]&)\), 
                  PlotLabel -> 
                    StringInsert["hr=", 
                      ToString[
                        rndoff[sm[[j]], 
                          3]], \(-1\)], PlotRange -> {0.0, 1.0}]]; 
              \), {j, 1, nx}]; 
            If["y" == 
                kbinstrg["Array graphics? y/n", {{"y"}, {"n"}}], 
              errf = 1; 
              While[errf == 1, 
                ppr = Input["Type number plots in each row as vector"]; 
                If[nx < Plus@@ppr, 
                  Print[ppr, ":Error:Plots in array exceeds ", nx], 
                  errf = 0]; ]; 
              Show[GraphicsArray[araydis[pobj, ppr]], 
                DisplayFunction -> $DisplayFunction]; ]; ]; 
          If["n" == 
              kbinstrg[
                "Analyze another cell? y/n", {{"y"}, {"n"}}], 
            agn = False]; ]; ]; \)\)

clr3[y] is a ColorFunction called by hwxav to set the RBG colors for density plots. The
notes below indicate the equivalence between the value of y  and the product value
WjXk, e.g., if WjXk=+2.0, after the logistic transform it is 0.88, and a value between
0.74 and 0.88 will be colored yellow.
Notes: Logistic[-2.0]=0.12; color=Black for values of y<=.12
           Logistic[-1.0]=0.27; color=Blue for values y>.12 and <=.27
           Logistic[1.0]=0.73; color=Green for values y>.27 and <=.73
           Logistic[2.0]=0.88; color=Yellow for values y>.73 and <=.88
           Logistic[3.0]=0.95; color=Red for values y>.88 and <=1.0
By changing values in the function clr3, the user can establish a different set of cutting points for establishing the colors/

clr3[y_] := 
      Which[y ≤  .12, RGBColor[0, 0, 0], y ≤  .27, 
        RGBColor[0, 0, 1], y ≤  .73, RGBColor[0, 1, 0], 
        y ≤  .88, RGBColor[1, 1, 0], y ≤ 1.0, 
        RGBColor[1, 0, 0]];

Test data for hwxav

rprtFdata

hdw = rcvrExpr["TCL953hwt", 19971022214210];

Dimensions[hdw]

hwxav[tclhw, tcL]

Notes to above density display

Row 1 represents the product components for exemplar 1 (T with no rotation) calculated by hidden cell 2 for each of the 9 input attributes (cells of the 3x3 character matrix). Low values result for input cell 5 (center cell of the 3x3 character matrix) for exemplars of T with the 0 and 180 degree, and 90 and 270 degree rotations showing slightly different pattern (position of the blue boxes).

Notes to above density  display

Each row of the large density plot of order (12,9) has been rearranged into individual square arrays of order (3,3) and follow the ordering of the (3,3) character matrix for each character. The title above each square array is the value of the response of  the hidden cell to a particular exemplar, which is the sum of the untransformed products represented by the colors.For example, for the character L with 0 degrees rotation, the title is hr=0.648 and is the response value of hidden cell 2 for exemplar 9 (see hidden response output for cell 2, exemplar 9, as output by the function bpnn in Neural.nb.)