Remote Services

Framework for Remote Kernel and Grid Management

Joel F. Klein

Wolfram Research, Inc.

Motivation

● Simplify the multi-vendor RSH/SSH/what-have-you configuration problem

● Provide a Wolfram solution designed to integrate with Mathematica

● Reclaim CPU cycles by building ad-hoc system (@home scenario)

● Any Mathematica users who want harness parallel computing

Outline

✓ Motivation 
● What is it and how does it work
● Remote calculation
● Service discovery
● Parallel calculations
● Monitoring and management
● Future directions and questions

What is it and how does it work?

● Alternative to ssh, etc. in launching kernels


But not just an alternative...

● Browse for available hardware

● Concept of "services": kernel launching command + configuration

● Monitoring and management

● Cross-platform

What is it and how does it work? (cont.)

● Put a daemon on each remote host

● Put a Mathematica add-on package where remote computation is controlled

What is it and how does it work? (cont.)

1000 words' worth of explanation:

What is it and how does it work? (cont.)

● Daemon is a Tomcat web server with Remote Services web application
   Acts as an agent for launching, monitoring, closing kernels

● Mathematica add-on package, Java talks HTTP to the server

● Web application sets up link from launched kernel back to itself
    Allows remote kernel to advise outside world of status

● Web server interactions are language independent XML calls -- can roll your own Remote Services client in C++, .Net, Python, etc.  If you can talk to the remote kernel via MathLink, you can use Remote Services kernels.

What is it and how does it work? (cont.)

Kernel launch flow

● Mathematica user requests particular kind of kernel (service) to be started on a particular host

● Mathematica package sends HTTP request "open kernel from service S" to agent (web server)

● Web server launches kernel via J/Link,
    sets up some MathLink connections, including a listening TCPIP link

● Web server returns linkname in HTTP response

● Package performs LinkConnect on the given linkname

● Mathematica user gets a LinkObject, and on we go

Outline

✓ Motivation 
✓ What is it and how does it work
● Remote calculation
● Service discovery
● Parallel calculations
● Monitoring and management
● Future directions and questions

Remote calculation

Needs["RemoteServices`"]

l =RemoteKernelOpen["jfkleinwin.wri.wolfram.com"]

LinkWriteHeld[l,Hold[{$MachineName, $ProcessID, 2^20}]]; LinkRead[l]

ReturnPacket[{"jfkleinwin",28452,1048576}]

RemoteKernelCloseAll[]

Outline

✓ Motivation 
✓ What is it and how does it work
✓ Remote calculation
● Service discovery
● Parallel calculations
● Monitoring and management
● Future directions and questions

Service discovery

● DNS-SD a component of Zeroconf networking, spearheaded by Stuart Cheshire at Apple
● Publish and browse model
● Allows users to find resources they didn't know about
● More information at http://zeroconf.org 
● Local link and wide-area versions of DNS-SD

Service discovery (cont.)

agents = Sort[RemoteKernelServices[]]

Service discovery (cont.)

agentinfo = RemoteServicesAgentInformation[agents[[1]]]

formatRule[lhs_ ->rhs_] := Row[{lhs,": ",rhs}];
formatRules[rules_] := Map[{formatRule[#]}&,rules];
cpuPlot[cpuHistory_] := ListPlot[cpuHistory,PlotJoined->True,PlotRange->{0,100},BaseStyle->Green,PlotStyle->Green,AxesStyle->Green,Background->Black];
nameOf[RemoteServices`Agent[{___,"ContactUrl"->u_,___}]] := StringReplace[u,{"http://"->"","/" ~~__~~ EndOfString ->""}];
formatAgent[Agent[allprops_]] := 
Module[{props,config,cpuHistory,propsrows,configrows,cpuplot},
props = DeleteCases[DeleteCases[allprops,"CpuHistory"->_],"Config"->_];
{config,cpuHistory} = {"Config", "CpuHistory" } /. allprops;
propsrows = formatRules[props];
configrows = Labeled[Framed[Grid[formatRules[config],Alignment->{Left,Top}]],"Configuration",{Top,Left}];
cpuplot = Labeled[Framed[cpuPlot[cpuHistory]],"CPU Usage History",Top];
Labeled[Framed[Grid[{{Grid[propsrows,Alignment->{Left,Top}]},
{Grid[{{configrows,cpuplot}},Alignment->{Left,Top}]}},
Alignment->{Left,Top}]],"Agent",{Top,Left}]];

formatAgent[agentinfo]//DisplayForm

Service discovery (cont.)

agentInfoList = Map[RemoteServicesAgentInformation,agents];

(* TabView *)
TabView[Map[nameOf[#]->formatAgent[#]&,agentInfoList],ControlPlacement->Left]

(* Simple column *)
Column[Map[formatAgent,agentInfoList]]//DisplayForm

Service discovery (cont.)

Finding services, not just hosts

services=RemoteKernelServiceInformation[];
services[[1]]

formatService[RemoteService[allprops:{___}]] :=
Module[{props,agent},
props = DeleteCases[allprops,"Agent"->_];
agent = "Agent" /. allprops;
Labeled[Framed[Grid[{{Grid[formatRules[props],Alignment->{Left,Top}]},{formatAgent[agent]}}]],"Service",Top]];

formatService[services[[1]]]

v6services = Cases[services,RemoteService[{___,"Name"->"6.0",___}]];
Length[v6services]

v6macServices = Cases[v6services,RemoteService[{___,"Agent"->Agent[{___,"OperatingSystem"->"Mac OS X",___}],___}]];
Length[v6macServices]

Service discovery (cont.)

Map[RemoteKernelOpen,v6macServices];
Map[RemoteKernelOpen,v6macServices];
RemoteKernels[]

Outline

✓ Motivation 
✓ What is it and how does it work
✓ Remote calculation
✓  Service discovery
● Parallel calculations
● Monitoring and management
● Future directions and questions

Parallel calculations

Needs["Parallel`"];
Map[ConnectSlave,RemoteKernels[]]

Now perform a parallel calculation, such as factoring integers of the form 111 ... 1 :

factors = Map[FactorInteger,(10^Range[20,55]-1)/9]; // AbsoluteTiming

factors2 = ParallelMap[FactorInteger,(10^Range[20,55]-1)/9]; // AbsoluteTiming

Parallel calculations (cont.)

Parallelizing high-latency, low-CPU tasks, 
or Mathematica helps you find a car

Get["/Developer/jfklein/workspace/AutoTraderSearch/AutoTraderSearch.m"]

candidates = autoTraderSearch["http://www.autotrader.com/fyc/searchresults.jsp?num_records=100&bkms=1159303316745&end_year=2006&make2=&address=61801&start_year=2003&fuel=&certified=&search_type=both&max_mileage=30000&body_code=0&drive=&min_price=10000&engine=4+Cylinder&distance=200&transmission=Automatic&rdpage=100&make=TOYOTA&sort_type=mileageASC&color=&style_flag=1&doors=Four+Door&advanced=y&max_price=17000&model=CAMRY&only_price=1"];
Length[candidates]

getPageTextInParallel[carLinks_]:=ParallelMap[(Print["Slave ",$ProcessorID,": Getting ",StringTake[#,50]];{"href"->#,"text"->Import[#]})&,carLinks];

pageTexts = getPageTextInParallel[candidates]; // AbsoluteTiming

StringTake["text"/.pageTexts[[1]],200]

carsMatchingSearchTerm[pageData_,searchText_]:=Map["href"/.#&,Cases[pageData,List[_,Rule[_,x_String]]/;StringMatchQ[x,"*"<>searchText<>"*",IgnoreCase->True]]];
matchingUrls = carsMatchingSearchTerm[pageTexts,"nroof"];//AbsoluteTiming
Length[matchingUrls]

Column[MapIndexed[Hyperlink["Match #"<>ToString[First[#2]],#]&,matchingUrls]]

Outline

✓ Motivation 
✓ What is it and how does it work
✓ Remote calculation
✓  Service discovery
✓ Parallel calculations
● Monitoring and management
● Future directions and questions

Monitoring and management

Status monitoring

RemoteEvaluate[matches={}]; (* set accumulator on workers *)

(* Queue an Import and string match for each candidate *)
pids = Map[Queue[Module[{pt},
RemoteServices`StartComputation[];
pt=Import[#];
If[StringMatchQ[pt,"*"<>"nroof"<>"*",IgnoreCase->True],
matches = Append[matches,#]];
RemoteServices`EndComputation[];]]&,
candidates]

(* Start it up *)
Wait[pids]; //AbsoluteTiming

allmatches = Flatten[RemoteEvaluate[matches]];
Length[allmatches]

Monitoring and management (cont.)

Progress metering

Row[{"url"," ",Round[100*"progress"],"%"}] /.RemoteKernelInformation[First[RemoteKernels[]],Renew->True]

pid = Queue[Module[{tot=0},
RemoteServices`StartComputation[];
Print["Slave ",$ProcessID, $MachineName, $ProcessID];
For[i=0,i<100,i++,RemoteServices`UpdateProgress[i/100.];tot+=N[Sin[i]];
Pause[1];];
RemoteServices`EndComputation[];
tot]];
QueueRun[]

Monitoring and management (cont.)

RemoteKernelCloseAll[]

Outline

✓ Motivation 
✓ What is it and how does it work
✓ Remote calculation
✓  Service discovery
✓ Parallel calculations
✓ Monitoring and management
● Future directions and questions

Future directions and questions

● One-stop shop installer bundle

● Full integration with Parallel Computing Toolkit

● Schedulable outages

● User tagging of services (saved queries)

Questions?

Outline

✓ Motivation 
✓ What is it and how does it work
✓ Remote calculation
✓  Service discovery
✓ Parallel calculations
✓ Monitoring and management
✓ Future directions and questions

Remote Services

● Cross-platform 
● Grid/cluster formation
● Monitoring and management
● Finding and grabbing hardware resources

You now know about Remote Services

● Find and grab hardware resources

● Concept of "services": kernel launching command + configuration

● Monitoring and management

● Cross-platform