Cloud Programming

Joel Klein, Wolfram Research

Join the Conversation #WolframTechConf

Abstract and Outline

How will you apply the Wolfram Language in the cloud? This talk will guide you through the process by looking at application patterns and the cloud programmer’s model from individual requests to complex interactions between cloud nodes. Topics such as performance, network latency, resource constraints and debugging techniques will be covered with real working examples.

Outline

• Properties of Applications
• Application Patterns
• Stateless Pattern: My Pi Day
• Stateless Pattern: ImageIdentify
• Stateful Pattern: Datadrop and Coffee Flavor-Rator
• Asynchronous: Tweet-a-Program
• Cloud Programmer’s Model
• Working example: Cloud Chat
• Working example: Accident Sign Generator
• Debugging

Properties of Applications

Independent variables that combine to give an application a certain shape.  What’s yours like?

1. Stateless/stateful

Is a request dependent on what requests have come before?

More about state

• State can be in:
		- client-side: cookies
		- server side: memory
		- server-side: persistent storage

Most web applications are stateless in terms of server-side memory, state comes from cookies and persistent storage. But server-side memory can be used for caching, to improve performance.

2. Size of code

Small code (under ~1MB) vs Large code

At issue: loading time at the beginning of an API call, affects how you structure your API

Note: “code” could include data, e.g. classification data

3. Length of computation

Fast (<100ms) vs slow

4. Size of data

How much data total? 

How much data is addressed in each request?

5. Contention

Is there contention to write data between simultaneous requests? 

(Database considerations, such as ACID properties, eventual )

6. Need for concurrent computation

When a single request/response model doesn’t cut it:
• Longer computations may need asynchronous computation
• Polling in a background task

7. Traffic

Number of requests per visit
Number of visitors

Application Pattern: Stateless function (MyPiDay)

http://mypiday.com/

Independent result for input of Oct 21, 2015

http://mypiday.com/results/?date=10-21-15&ck=d2935a21
(found my pi day @ π digit 1,331,079)

Result as a t-shirt

http://www.zazzle.com/my_pi_day_t_shirt-235580467374193131

How does MyPiDay.com work?

• The core calculation takes a few seconds

• Stateless - result comes from user input, nothing from persistent storage.

• There is some data that can be cached. Once a date is computed, its result can be cached. The nice thing about caching stateless functions is the results don't need to be invalidated -- the only concern is limiting space, solved by flushing seldom-used results (LRU replacement). But this cached data doesn't make it stateful, it only affects performance not the results.

Summary of properties

• Stateless
• Small code
• Length of computation: slow (seconds)
• Size of data: small
• Contention: none
• Need for asynchronous computation: none
• Traffic: small number of static and active requests per visit, large number of visitors

This is a simple, uncomplicated computation site that nevertheless gives an interesting result.

Application pattern: Stateful, data intensive (DataDrop & Flavor-Rator)

https://datadrop.wolframcloud.com/
http://blog.wolfram.com/2015/09/21/the-coffee-flavor-rator/

Wolfram DataDrop

Devices and programs can drop data into a data bin and retrieve or all or part of a bin.

DataDrop is generic enough to be a service, we'll look at an app built with it: the Coffee Flavor-Rator.

DataDrop is patently stateful: reading the contents of a databin can change from one call to the next as data is added to the bin, without any other apparent change in direct user input. (Put another way, the state of persistent storage is an implicit input.)

Summary of properties for DataDrop

• Stateful
• Small code
• Length of computation: fast
• Size of data: large pool of data, data returned is controlled by input to request (All vs part)
• Contention: none, typically (single writer is typical, but multiple writers can happen, handled through spin locks)
• Need for asynchronous computation: none
• Traffic: one API call per drop, varies on client/consumption side

Coffee Flavor-Rator

Gathers survey data from users and gives an aggregated result:

https://www.wolframcloud.com/objects/user-7053ce31-817f-4643-aec1-eda27051bba6/flavor-rator

The results have linear running time as the entire survey must be read into memory and summarized.

For small input sets, this is fine, but it quickly breaks down if you use this technique on a major web site.

A better approach is to summarize as you go, but this introduces contention. Another approach is to use asynchronous computation to do summarization in batches, which can remove the contention issue.

Application pattern: Asynchronous computation (Tweet a Program)

https://twitter.com/wolframtap

Tweet a Program

You tweet to @WolframTAP, the app runs the content of the tweet and tweets back the result:

How Tweet a Program works

A background task runs that listens for new new tweets. When a new tweet arrives, it calls a cloud API to handle the tweet itself.

Since tweets are handled asynchronously in separate API calls, the background task will never be delayed by a slow or malicious tweet.

This is made possible because identifying new tweets and processing them are independent, as is the handling of different tweets.

One possible feedback route might be to cache inputs that have already occurred, but this could also be built into the tweet handler and ignored by the background task.

Summary of properties for Tweet a Program

• Stateless from the user’s point of view, it may need internal state to work with the Twitter API
• Small code
• Length of computation: fast
• Size of data: small
• Contention: none (single background task)
• Need for concurrent computation: yes
• Traffic: one API call per tweet, web traffic, including serving results, is handled by Twitter

Cloud Programmer’s Model

Nodes, Sessions, and Kernel Types

• Cloud is comprised of multiple nodes (private cloud can have 1 node).

• The first web request from a user will be assigned a session.

• All requests in a session are routed to the same node.

• Number of nodes is autoscaled according to demand (in public cloud)

• Kernel types: 
   • public (for servicing APIs, forms, and other active content, as well as public CloudCDFs)
   • service (scheduled tasks and related, no web requests directly involved)
   • interactive (development environment, notebook editing)
   
   Web applications are concerned with public and service kernels.

• Static content is served without involving a kernel.

Files/Cloud objects

Kernel Sandbox

Request Model

Response Model

Cloud Programmer’s Model: Nodes, Session, and Kernel Types

• Cloud is comprised of multiple nodes (private cloud can have 1 node).

• The first web request from a user will be assigned a session.

• All requests in a session are routed to the same node.

• Number of nodes is autoscaled according to demand (in public cloud)

• Kernel types: 
   • public (for servicing APIs, forms, and other active content, as well as public CloudCDFs)
   • service (scheduled tasks and related, no web requests directly involved)
   • interactive (development environment, notebook editing)
   
   Web applications are concerned with public and service kernels.

• Static content is served without involving a kernel.

Debugging tip: what node and kernel am I on

{$MachineName, $ProcessID}

CloudEvaluate[{$MachineName, $ProcessID}]

Tally@Table[CloudEvaluate[{$MachineName, $ProcessID}], 20]

Performance note: network latency

Requests to the cloud from the internet: highly variable, 100-400ms typical values
   (calls from desktop Mathematica do this)

Requests between cloud nodes: much faster, <<100ms typical values
   (Tweet a Program does this)

These come into play as you plan webapps, e.g. Google Docs plugin.

Performance note: API overhead

An area of improvement over last year: time to run a trivial APIFunction, i.e. overhead added by the cloud to network latency. This is now in line with similar web frameworks, and is adding value.

Cloud Programmer’s Model: Files and Cloud Objects

When running code in the cloud, your home directory is visible through normal file operations.

Your home directory files are also accessible as cloud objects.

obj = CloudPut[Range[10], CloudObject["numbers.m"]]

CloudEvaluate[Read["numbers.m"]]

Other people’s cloud objects are only accessible as CloudObjects, not through normal file system.

Performance tip

Presently, direct file system access is faster than through a CloudObject. If you want to serve a file, it must be a CloudObject. So create it as a CloudObject first, and then if it needs to be read or written, you can do it as a file.

Debugging tip

Two things can go wrong with a cloud object: content can be wrong, or meta information can be wrong.

Content can be checked as normal with file system commands: FilePrint, Get, BinaryReadList, etc.

Meta information can be checked with Options[CloudObject[...]] and CloudObjectInformation[CloudObject[...]]

Cloud Programmer’s Model: Kernel Sandbox

In the cloud, your code is run in a sandbox mode.

Your home directory and some other essentials (Mathematica installation directory) are visible.

Files in directories outside the sandbox whitelist act as if they do not exist:

CloudEvaluate[Import["/etc/passwd","Text"]]

Generally, running user-provided binaries is not allowed.

Cloud Programmer’s Model: Request Model and User Metadata

https://reference.wolfram.com/language/guide/CloudExecutionMetadata.html

$CloudEvaluation

$EvaluationEnvironment

$WolframID

$WolframUUID

$RequesterWolframID

$RequesterWolframUUID

$RequesterAddress

Tip: WolframID is convenient, but WolframUUID is stable

Tip: be clear on the owner and the requester when talking about server side code

Server side code generally runs as the owner (current directory, what files are visible) but location-dependent variables are set from the perspective of the requester.

Localized variables

$GeoLocation

$GeoLocationCountry

$GeoLocationCity

$TimeZone

$DateStringFormat

CloudEvaluate[Map[# -> ToExpression[#]&, {"$GeoLocation", "$GeoLocationCountry", "$GeoLocationCity", "$TimeZone", "$DateStringFormat", "DateString[]"}]]

Everything else:

HTTPRequestData[]

CloudEvaluate[HTTPRequestData[]]

Cloud Programmer’s Model: Response Model

Three things at play:

1. ExportForm
2. ResponseForm
3. HTTPResponse

Ultimately, server side code boils down to an HTTPResponse, so study that.

But code is neater if you rely on ExportForm and ResponseForm.

ExportForm is useful because so often a result is filtered through an Export to be useful on the web.

ResponseForm is useful to provide a result and metadata, very valuable for debugging (messages and Print output)

ExportForm

$Permissions="Public";

CloudDeploy[Delayed[ClockGauge[], "Text"]]

CloudDeploy[Delayed[ClockGauge[], "PNG"]]

CloudDeploy[Delayed[ExportForm[ClockGauge[], "JPG"]]]

CloudDeploy[Delayed[ExportForm[ClockGauge[], "JPG"],"PNG"]]

ResponseForm

CloudDeploy[Delayed[Delayed[ClockGauge[1/0], {"PNG","JSON"}]]]

Debugging tip: EvaluationData

ResponseForm’s metadata comes from EvaluationData:

EvaluationData[Print["about to divide by zero"]; Pause[0.3]; 1/0]

Working Example: Cloud Chat

https://www.wolframcloud.com/app/objects/jfklein/cloudchat/chat.html

https://wolfr.am/7DeniupD

Give it a try now!

Mix of static and active (API) content.

Massive numbers of requests, polls every 3 seconds × #clients

Responsive (see API overfhead improvements)