Wolfram Library Archive

Courseware Demos MathSource Technical Notes
All Collections Articles Books Conference Proceedings

Star-P Language Extension for Fine-Grain Parallel: Computation on Distributed Data Sets

Eckart Jansen

2006 Wolfram Technology Conference
Conference location

Champaign IL

Simplified handling of terabyte-size distributed data sets is critical for many of today’s toughest computational problems. For example, a 10MB data file generated by an airborne radar today may swell under analysis into terabyte sizes. We demonstrate overloading of the Table function by using “*P” in the iterator to create a new type of data object that resides on a parallel HPC backend server. Data objects on the HPC backend server are manipulated using data-parallel versions of popular desktop Mathematica functions. Through type propagation, related variables also become parallel. Data objects created with the Star-P language extension are shown to be interoperable with ordinary Mathematica data objects and code. Data is automatically transferred between the Mathematica desktop and HPC backend server as required. A demonstration showing the creation of global distributed array objects on a remote HPC server backend and their manipulation using a series of parallel linear algebra operations will be shown.