Data as Code/Code as Data

George E. Danner, Business Laboratory

Join the Conversation #WolframTechConf

Introduction

It isn’t often that we think of data we use in our models as comprised of large blocks of software.  However if we did so, the resulting analyses and visualizations we could perform would be very insightful, especially to those wanting to understand large interconnected software systems.  This is Data as Code.

Conversely, what if many thousands of code fragments were stored in a database such that a central system could assemble its logic automatically?  This is Code as Data.  In our talk we will expand on both these ideas using concrete examples and projects.

Two Parts to this Discussion

Code as Data

Treating a large batch of code or application as if it were data for analysis

Data as Code

Putting a large number of code fragments into a database that can be recalled and run on demand

Code as Data

A large code base could be thought of as a collection of strings separated by delimiters like “;”

effectiveSolarExposure=GetParameter["EffectiveSolarExposure",scenarioID];
solarRadiation="Insolation"/.GetWeather[regionID,periodID];
windowArea="WindowArea"/.archetypeData;
passiveSolarGain=solarRadiation*effectiveSolarExposure*windowArea;
flow=CalculateWXIndFlow[regionID,archetypeID,periodID,sectorID,behaviourID];
incGains=Total[Map[#[[2]]*GetUsefulnessFactor[sectorID,#[[1]]]&,flow]]+incGainOccupants+passiveSolarGain;
specificHeatLoss="SpecificHeatLoss"/.archetypeData;
tComfort="TComfort"/.archetypeData;
tAmbient="TemperatureAmbient"/.GetWeather[regionID,periodID];
netSpaceHeat=Table[Max[(((tComfort-tAmbient[[i]])*specificHeatLoss)-incGains[[i]]),0],{i,1,48}];
spaceHeat=netSpaceHeat*pm*pw*phh;

Actual user story : financial services firm

So what can you do by treating code as data?

Button["Open DAC Experiments",
NotebookOpen[NotebookDirectory[]<>"Data_As_Code_Experiments.nb"]
]

Implications :

We can look for patterns in code

Are there bottlenecks?

Unnecessary repitition?

Opportunities for code optimization

Malware, viruses, back doors, and vulnerabilities

Button["Open Bottlenecks",
NotebookOpen[NotebookDirectory[]<>"BottleNeck.nb"]
]

Button["Open System Implementation",NotebookOpen[NotebookDirectory[]<>"System_Implementation.nb"]]

Implications for other applications of the same technique

Documents with logic, like contracts

Button["Run Simulation", SystemOpen[NotebookDirectory[]<> "Drilling_Simulation.mov"]]

Automation of Ontologies (Graph Databases)

Data as Code

What if our code could reference a database of code fragments?

The central idea :

Code could be changed outside of the core

Code could be loaded on an as - needed basis and specific to an instance of use

A plausible application :

Think of a car reacting to the season and location (altitude, road conditions).  This goes far beyond the idea of "parameters" into intelligence that is imparted onto the device.  Without the Data as Code idea you would often be left creating a large and unwieldy code base.

A real application :

equations = Import[NotebookDirectory[] <> "ANOptimizationData.xlsx", {"Data", "Financial Sub Model"}];

equations//TableForm

Founders of Viv, talking about its underlying architecture :

"Siri, it can’t do anything that coders haven’t explicitly programmed it to do...Viv breaks through those constraints by generating its own code on the fly, no programmers required. Take a complicated command like “Give me a flight to Dallas with a seat that Shaq could fit in.” Viv will parse the sentence and then it will perform its best trick: automatically generating a quick, efficient program to link third-party sources of information together”

Wired Magazine, August 12, 2014

And then there' s IFTTT

Summary

Brilliant things can happen when we break out of our traditional notions of what constitutes data.

For more detail on this subject ...