Code Generation for a Symbolic Matrix

In this notebook we give an example of how MathCode C++ can be used to generate C++ code for efficient computation of a large matrix with symbolic entries. The matrix in this case is a Hessian of a given function which is an important entity to compute efficiently in for example optimization algorithms.

A Small Exampe

An Expression

We will use the following expression for generating a large matrix with symbolic entries.

expr = (((Sin[y z] + ArcTan[x]))/(Sqrt[1 +  .5  E^(-x^2 - y^2)]));

A Function for Computing the Gradient

Grad computes the gradient of a function w.r.t. a list of variables

Off[
        General::spell1] ;

Grad ;On[
        General::spell1] ;

\(Grad[fcn_, vars_] := Map[∂\_# fcn &, vars]\)

Grad[expr, {x, y, z}]

A Function for Computing the Hessian

Hessian computes the Hessian matrix of a function w.r.t. a list of variables. HessianMC should be used because Hessian[] already exists in Mathematica.

HessianMC[fcn_, vars_] := 
      Map[Grad[#, vars] &, Grad[fcn, vars]]

HessianMC[expr, {x, y, z}]

A Symbolic Matrix

We define a function that evaluates the Hessian of a given expression expr

SymbolicMatrix[x_, y_, z_] = HessianMC[expr, {x, y, z}];

A numerical evaluation

SymbolicMatrix[1. , 2. , 3. ] // MatrixForm

Code Generation

Load the MathCode C++ application package

Needs["MathCode`"]

SetDirectory[$MCRoot<>"/Demos/CompiledHessian"];

We need to declare types of the argument and return value of the function to be compiled. In this case 3 real parameters and a real 3 × 3 matrix.

Declare[
      SymbolicMatrix[Real x_, Real y_, Real z_] -> 
        Real[3, 3]]

We generate C++ code and compile it to an executable file using the BuildCode command. The option EvaluateFunctions makes optimizations such as common subexpression eliminations and symbolic evaluation of the function SymbolicMatrix before code generation.

BuildCode[EvaluateFunctions -> {SymbolicMatrix}]

Connect the executable to Mathematica

InstallCode[];

Now the external code is used instead

SymbolicMatrix[1. , 2. , 3. ] // MatrixForm

Remove the link to the external code (Mathematicas own definition of SymbolicMatrix is used again).

UninstallCode[];

Remove the definition of SymbolicMatrix

Remove[SymbolicMatrix]

The Generated Code

Here follows a listing of the generated code. Note the extensive use of temporary variables. The option EvaluateFunctions used above in the BuildCode command decomposes the large symbolic expression in such a way that common subexpressions only are evaluated once and then stored in temporary variables to be reused in the numeric evaluation of the rhs of the function. This gives extremely efficient code!

!! Global.cc

A Large Example

A New Expression

We now consider an expression with 10 parameters

expr = (((Sin[y z] + ArcTan[x]) (1 + Log[u v]) (E^( .5  d)))/((Sqrt[1 +  .5  (E^(-(x^2) - (y^2)))]) (2 + (v^(-1/w)) + (Sqrt[2 + Sinh[1/a] + b/c]))));

We redefine the function that evaluates the Hessian of expr

SymbolicMatrix[x_, y_, z_, u_, v_, w_, a_, b_, c_, d_] = 
        HessianMC[expr, {x, y, z, u, v, w, a, b, c, d}];

A numerical evaluation

SymbolicMatrix[1. , 2. , 3. , 4. , 5. , 6. , 7. , 8. , 9. , 10. ] // 
      MatrixForm

The symbolic form of the matrix occupies approximately 30 pages!

Code Generation

We need to declare types of the argument and returnvalue of the function to be compiled. In this case a list of 10 real parameters and a real 10 × 10 matrix.

Declare[
      SymbolicMatrix[Real x_, Real y_, Real z_, Real u_, Real v_, 
          Real w_, Real a_, Real b_, Real c_, Real d_] -> 
        Real[10, 10]]

We generate C++ code and compile it to an executable file using the BuildCode command. The option EvaluateFunctions makes optimiizations such as common subexpression eliminations and symbolic evaluation of the function SymbolicMatrix before code generation.

BuildCode[EvaluateFunctions -> {SymbolicMatrix}]

Connect the executable to Mathematica

InstallCode[];

Now the external code is used instead

SymbolicMatrix[1. , 2. , 3. , 4. , 5. , 6. , 7. , 8. , 9. , 10. ] // 
      MatrixForm

Remove the link to the external code (Mathematicas own definition of SymbolicMatrix is used again).

UninstallCode[];

A Performance Comparision

To compare performance between compiled and uncompiled code we define a function that uses SymbolicMatrix a specified number of times.

RepeatComputation[Integer n_] -> Null := 
      Do[SymbolicMatrix[1.  + 1/n, 2.  + 1/n, 3.  + 1/n, 4.  + 1/n, 
            5.  + 1/n, 6.  + 1/n, 7.  + 1/n, 8.  + 1/n, 9.  + 1/n, 
            10.  + 1/n];, {n}]

BuildCode[EvaluateFunctions -> {SymbolicMatrix}]

We do 100 iterations with the uncompiled code and compute the average time:

n = 100;
start = AbsoluteTime[];
RepeatComputation[n];
time1 = (AbsoluteTime[] - start)/n

Install the compiled code into Mathematica

InstallCode[];

We do 100000 iterations with the uncompiled code and compute the average time:

n = 100000;
start = AbsoluteTime[];
RepeatComputation[n];
time2 = (AbsoluteTime[] - start)/n

UninstallCode[];

Speed-up:

time1/time2

Clean Up

Remove the definition of SymbolicMatrix and all generated files.

Remove[SymbolicMatrix, RepeatComputation]

CleanMathCodeFiles[Confirm -> False, 
      CleanAllBut -> {}]