The New Built-in Data Collections in Mathematica

Jamie Williams and Eric Weisstein

Wolfram Research, Inc.

The Wolfram Computable Data Initiative

Goal: Gather together data from a wide range of disciplines into a unified computational framework for immediate and direct use in calculations

Users of data are typically faced with the following difficulties:

Difficult to locate and extract from sources

Data from seemingly unrelated fields is scattered across the vastness of the internet

Different disciplines use different conventions for naming, units, methods, etc.

Data comes in many different file formats

Databases are not always easy to use

The Wolfram Computable Data Initiative

Goal: Gather together data from a wide range of disciplines into a unified computational framework for immediate and direct use in calculations

Plot[(x^(-1.5)),{x,.0001,10},Filling->Axis,PlotRange->{{0,10},{0,1}},FillingStyle->Yellow,Axes->False,Frame->True,FrameTicks->None,FrameLabel->{Style["Data Specialization",18],Style["Usage",18]}]

In the first stage, we are aiming for the head of the distribution.  We currently have 14 data functions covering a broad range of disciplines: mathematics, physics, chemistry, economics & finance, geopolitical, linguistic, and more.

The idea of a data function

A data function works much the same way as any other function in Mathematica, but returns stored data

The interface is much easier than standard database quering methods

CityData["Paris"]

CityData[{"Paris","IleDeFrance","France"},"Population"]

You can compute with them

((ElementData["Gold","SoundSpeed"])^2)/ElementData["Gold","Density"]

You can plot them

ListLogLogPlot[Tooltip[{CountryData[#,"MilitaryExpenditures"],CountryData[#,"GDP"]},CountryData[#,"Name"]]&/@CountryData["Countries"],Frame->True,FrameLabel->{"military expenditures","GDP"}]

You can manipulate them

nasdaqComputer=Cases[Join[FinancialData["Computers","Members"],FinancialData["ComputerProgrammingDataProcessing","Members"]]//Union,x_?(StringMatchQ[#,"NASDAQ:*"]&)];

Manipulate[Labeled[DateListPlot[FinancialData[s,"January"]],FinancialData[s,"Name"]],{s,nasdaqComputer,ControlType->PopupMenu}]

The data is extensible

Mathematica does not ship with (most of) the data: It is stored on our servers and pushed to the user over the internet.

The first time you use a data function, Mathematica contacts our data server, which sends basic indexing info

The first time you query for an entity/property in a session, Mathematica downloads the most updated data from our server, which gets stored locally on your hard drive in a compressed file

In a Mathematica session, after a data function is first used, data gets cached in memory

Initial offering in v6.0

Integrated Data Sources: Documentation

There are three different types of data coverage

Pre-computed: most of the mathematical data is pre-computed in-house

- e.g. GraphData, KnotData, LatticeData, PolyhedronData

Aggregated from publicly available sources

- e.g. AstronomicalData, CityData, CountryData, ChemicalData, ElementData, IsotopeData, ParticleData, WordData

Direct connection to web services-based provider

- e.g. FinancialData

Basic syntax

XXXXData["Entity", "Property"] returns any valid Mathematica expression

ElementData["Einsteinium","DiscoveryYear"]

CityData[{"Gaithersburg","Maryland","UnitedStates"},"Coordinates"]

FinancialData["^DJI","Members"]

Missing values

CountryData["Cuba","PovertyFraction"]

IsotopeData["Cu76","Parity"]

Missing values are ignored in plotting functions

ElementData["Helium","MohsHardness"]

mh=ElementData[#,"MohsHardness"]&/@ElementData[];

mh//Tally//Cases[#,{_Missing,_}]&

ListPlot[mh,PlotStyle->PointSize[.01]]

XXXXData["Entity", "Property", "Annotation"] gives additional info

Units

AstronomicalData["Earth","RotationPeriod"]

AstronomicalData["Earth","RotationPeriod","Units"]

ElementData["Gold","Density"]

ElementData["Gold","Density","Units"]

Significant digits using Mathematica's built-in precision tagging

d=ChemicalData["Aspirin","Density"]

Precision[d]

Uncertainty interval

ParticleData["Muon","Width"]

ParticleData["Muon","Width","Interval"]

Note

ElementData["Helium","MeltingPoint"]

ElementData["Helium","MeltingPoint","Note"]

XXXXData[All] gives a list of entities

AstronomicalData[All]//Length

XXXXData["Properties"] gives a list of properties

CityData["Properties"]

ChemicalData["Properties"]//Length

XXXXData["Classes"] gives a list of classes

AstronomicalData["Classes"]

AstronomicalData["Planet"]

CountryData

Data on over 230 countries throughout the world

Over 200 properties

CountryData["Properties"]//Short[#,6]&

Countries in a group

CountryData["Groups"]

CountryData["Africa"]

Plot the GDP of France from 1970 to 2005:

DateListPlot[CountryData["France",{{"GDP"},{1970,2005}}]]

Generate a world map using a Mollweide projection:

CountryData["World", {"Shape", "Mollweide"}]

Show countries in South America colored by their arable land fractions

Graphics[{Hue[CountryData[#,"ArableLandFraction"]],CountryData[#,"Polygon"]}&/@CountryData["SouthAmerica"]]

Plan a tour through every country of the world :

SC[{lat_,lon_}]:=r {Cos[lon °] Cos[lat °],Sin[lon  °] Cos[lat  °],Sin[lat °]};

r=6378.7;
places=CountryData["Countries"];
centers=Map[CountryData[#,"CenterCoordinates"]&,places];
distfun[{lat1_,lon1_},{lat2_,lon2_}]:=VectorAngle[SC[{lat1,lon1}],SC[{lat2,lon2}]]r;

{dist,route}=FindShortestTour[centers,DistanceFunction->distfun];

Visualize the tour:

surfaceCenters=Map[SC,centers[[route]]];

GreatCircleArc[{lat1_,lon1_},{lat2_,lon2_}]:=
Module[{u=SC[{lat1,lon1}],v=SC[{lat2,lon2}],a},a=VectorAngle[u,v]; Table[Evaluate[RotationTransform[θ,{u,v}][u]],{θ,0,a,a/Ceiling[10a]}]]

tourLine=Apply[GreatCircleArc,Partition[centers[[route]],2,1],{1}];

Graphics3D[{Sphere[{0,0,0},0.99 r],Map[Line[Map[SC,CountryData[#,"SchematicCoordinates"],{-2}]]&,places],{Red,Thick,Line[tourLine]},{Yellow,PointSize[Medium],Map[Tooltip[Point[SC[CountryData[#,"CenterCoordinates"]]],#]&,places]}},Boxed->False,SphericalRegion->True]

CityData

Over 163,000 cities covered throughout the world

CityData["Properties"]

Find city names that occur most frequently

Take[Sort[Tally[CityData[#,"Name"]&/@CityData[{All,"UnitedStates"}]],(#1[[2]]>=#2[[2]]&)],10]//TableForm

Find all cities in the US named Springfield

springfield=Select[CityData[{All,"UnitedStates"}],CityData[#,"Name"]=="Springfield"&]

TableForm[{CityData[#,"Population"],#}&/@springfield,TableDepth->2]

CityData[#,"Population"]&/@{"Champaign","Urbana","Savoy"}

Make a network of countries that share more than 50 city names :

d=First/@(Last[Reap[Sow[#[[1]],#[[3]]]&/@CityData[],CountryData[]]]/.{}->{{}});

m=ReplacePart[Outer[Length[Intersection[##]]&,d,d,1],{x_,x_}->0];

GraphPlot[Rule@@@Map[CountryData[][[#]]&,Union[Sort/@Position[m,x_/;x>50]],{2}],VertexLabeling->True]

AstronomicalData

Data covering over 100,000 objects, including the sun, planets, moons, stars, and galaxies

AstronomicalData["Classes"]

Over 70 unique properties, including orbital data, shape and appearance, identifiers

AstronomicalData["Properties"]

A list of solar system planets:

AstronomicalData["Planet"]

Text[Grid[{AstronomicalData[#,"Name"],AstronomicalData[#,"OrbitPeriod"]/3.1558*^7}&/@AstronomicalData["Planet"],Frame->All,Background->LightYellow]]

Make a graphic of solar system orbit paths:

Graphics3D[Tooltip[AstronomicalData[#,"OrbitPath"],AstronomicalData[#,"Image"]]&/@AstronomicalData["Planet"]]

Find the ten nearest stars and their distance in light years:

AstronomicalData["StarNearest10"]

AstronomicalData[#,"DistanceLightYears"]&/@%

ChemicalData

Currently over 18,000 compounds

Nearly 70 unique physical, structural, and safety properties, as well as chemical identifiers:

ChemicalData["Properties"]

Use string patterns to find a compound:

Short[ChemicalData["*chloro*"],10]

Find the melting point of ibuprofen

ChemicalData["Ibuprofen","MeltingPoint"]

Find the structure diagram for riboflavin:

ChemicalData["Riboflavin"]

ChemicalData["Riboflavin","EdgeRules"]

ChemicalData["Riboflavin","VertexTypes"]

Find the 3D molecular structure for the cholesterol:

ChemicalData["Cholesterol","MoleculePlot"]

ChemicalData["Cholesterol","AtomPositions"]

Obtain an NFPA safety label for nitroglycerin:

ChemicalData["Nitroglycerin","NFPALabel"]

ElementData

Covers the entire periodic table, from hydrogen to element 118:

ElementData[All]

Over 80 unique properties, including atomic weight, density, boiling & melting points, etc:

ElementData["Properties"]

Find the atomic weight of oxygen:

ElementData["Oxygen","AtomicMass"]

Find the density of lead:

ElementData["Pb","Density",#]&/@{"Value","UnitsNotation"}//Row

Use Manipulate to systematically explore the data:

Manipulate[ListPlot[Table[Tooltip[{ElementData[i,x],ElementData[i,y]},Row[{i," ",ElementData[i]}]],{i,118}],AxesLabel->{x,y}],{{x,"AtomicNumber"},ElementData["Properties"],ControlType->PopupMenu},{{y,"Density"},ElementData["Properties"],ControlType->PopupMenu}]

With[{props=Select[ElementData["Properties"],NumberQ[ElementData[4,#]]&]},Manipulate[ListPlot[Table[Tooltip[{ElementData[z,props[[p[[1]]]]],ElementData[z,props[[p[[2]]]]]},ElementData[z]],{z,118}],Frame->True,AspectRatio->1,FrameLabel->{{Row[{ElementData[1,props[[p[[2]]]],"Description"],"  ",(ElementData[1,props[[p[[2]]]],"UnitsNotation"]/._Missing|_ElementData->"")}],Null},{Row[{ElementData[1,props[[p[[1]]]],"Description"],"  ",(ElementData[1,props[[p[[1]]]],"UnitsNotation"]/._Missing|_ElementData->"")}],Null}},ImagePadding->{{90,15},{50,20}},PlotStyle->Directive[PointSize[Medium],Blue]],{{p,{17,45},"Properties"},{1,1},{Length[props],Length[props]},{1,1}}]]

IsotopeData

Find the distribution of lifetimes:

ListPlot[Transpose[{Table[i,{i,-30,30}],BinCounts[Cases[Log[10,IsotopeData[#,"Lifetime"]]&/@IsotopeData[],_Real],{-30.5,30.5,1}]}],Filling->Axis,Frame->True,Axes->None,PlotRange->All,FrameLabel->{"(Subscript[log, 10]) τ",None}]

Make an isotope chart of lifetimes:

g=Table[{a-z,z,Log[10,IsotopeData[{z,a},"Lifetime"]]},{z,1,118},{a,IsotopeData[#,"MassNumber"]&/@IsotopeData[z]}]//Flatten[#,1]&;

ListDensityPlot[g,FrameLabel->{"N","Z"},ColorFunction->"Rainbow"]

Draw a decay network:

is=Select[IsotopeData[],100≤IsotopeData[#,"MassNumber"]≤102&];

LayeredGraphPlot[Table[(IsotopeData[#,"Symbol"]&/@(i->#))&/@IsotopeData[i,"DaughterNuclides"],{i,is}]//Flatten,DirectedEdges->True,VertexLabeling->True]

Mathematical and "Social Science" Paclets

Linguistic

DictionaryLookup

WordData

Financial

FinancialData

Mathematical

GraphData

KnotData

LatticeData

PolyhedronData

DictionaryLookup

Usage

?DictionaryLookup

DictionaryLookup["*"]//Length

DictionaryLookup["lemo*"]

DictionaryLookup["lemo*",4]

DictionaryLookup["a"~~__~~"z"]

Simple Applications

Palindromes

Palindromes[n_]:=DictionaryLookup[x__/;(x==StringReverse[x] && StringLength[x]≥n)]

Palindromes[5]

Anagrams

Anagrams[s_String, 1] := Module[{c = Sort[Characters[s]]}, DictionaryLookup[x__ /; Sort[Characters[x]] == c]]

Anagrams["lemon",1]

WordData

Usage

?WordData

WordData[]//Length

WordData["Properties"]

Examples

WordData["spiky"]

WordData["spiky","PartsOfSpeech"]

WordData["innovation"]

WordData

Simple Applications

De-Britishification

TableForm[Outer[WordData,{"color","curb","metre","tire"},{"AmericanSpelling","BritishSpelling"}],TableHeadings->{None,{"American","British"}}]

Shallow[brit=Select[WordData[],WordData[#,"AmericanSpelling"]≠WordData[#,"BritishSpelling"]&]]

Select[brit,StringMatchQ[#,"colo*"]&]

brit2amer=Rule@@@Union[{WordData[#,"BritishSpelling"],WordData[#,"AmericanSpelling"]}&/@brit];

str="When I parked my car at the freshly painted kerb next to the parking metre I got paint on my tyre and changed its colour";

StringJoin@@Riffle[StringSplit[str]/.brit2amer," "]

Hyphenation

WordData["innovation","Hyphenation"]

StringJoin@@Riffle[%,"-"]

FinancialData

Usage

?FinancialData

FinancialData[]//Length

FinancialData["Properties"]

Examples

FinancialData["ZEUS"]

FinancialData["ZEUS","Symbol"]

FinancialData["ZEUS","StandardName"]

FinancialData["ZEUS","Name"]

DateListPlot[FinancialData["GE","Jan. 1, 2000"]]

FinancialData

Simple Application

401K

V401KFunds = {"VASIX","VSCGX","VSMGX","VASGX","VFSTX","VBMFX","VWINX","VFINX","VWNFX","VWUSX","VEXMX","VEXPX","VWIGX"};

(*
data1996=
{FinancialData[#,"CumulativeReturn",{{1996,1,1},{2007,6,6}}],#<>": "FinancialData[#,"StandardName"]}&/@V401KFunds;
*)

data1996=
{FinancialData[#,"CumulativeReturn",{{1996,1,1},{2007,6,6}}],#}&/@V401KFunds;

DateListPlot[Tooltip@@@data1996,Joined->True,Filling->Bottom]

GraphData

Usage

?GraphData

Lots of computation at your fingertips

Characteristic Polynomial

209397.46655100002`/3600/24 CPUDays

GraphData["F998A","CharacteristicPolynomial"][x]//TraditionalForm

Chromatic Polynomial

ChromaticPolynomial[Graph["ErreraGraph"],x]//Timing

1.3924862584669997`*^6 /3600/24 CPUDays

(this sort of thing is a good stress test for Mathematica grid technology ;)

GraphData

Interactive Exploration

CollapseFriends[x_]:=StringSplit[Flatten[{x}][[1]],DigitCharacter][[1]]

Friends=First/@Split[GraphData[],CollapseFriends[#1]==CollapseFriends[#2]&];

Manipulate[Column[{GraphData[g],GraphData[g,p]}],{g,Friends},{p,Complement@@GraphData/@{"Properties","Classes"}}]

GraphData

Nice examples

Embeddings

GraphicsGrid[Partition[GraphData["TesseractGraph","AllImages"],6,6,{1,1},{}],ImageSize->600]

Find out more

GraphData["TesseractGraph","Information"]

Tables of graphs with known embeddings

GraphicsGrid[Partition[(Tooltip@@GraphData[#,{"Image","Name"}])&/@Take[GraphData["VertexCoordinates","Defined"],60],10],ImageSize->600]

GraphData

New in V6

V6.0

GraphData[All]//Length

V6.1

GraphData[All]//Length

GraphData["Properties"]

v61=GraphData["Properties"];

Complement[v61,v60]

KnotData

Examples

KnotData[]

KnotData[All]//Short

KnotData["Properties"]

KnotData["FigureEight", "AlexanderPolynomial"][x]//TraditionalForm

KnotData["FigureEight", "Image"]

KnotData["FigureEight","Information"]

KnotData

Nice Examples

TabView[Graphics[KnotData[#,"KnotDiagramData"], Epilog -> Inset[Style[KnotData[#,"AlexanderBriggsNotation"]],{Right,Bottom},{Right,Bottom}]] & /@ Take[KnotData[All],5],2]

Graphics[{Thickness[.01],Transpose[{Table[RGBColor @@ RandomReal[{0,1},3],{Length[#]}], #}]}]& @KnotData[{"TorusKnot",{7,30}},"KnotDiagramData"]

LatticeData

Examples

LatticeData[All]

LatticeData["Properties"]

LatticeData["FaceCenteredCubic"]

LatticeData["FaceCenteredCubic","Image"]

PolyhedronData

Usage

?PolyhedronData

Enumeration

PolyhedronData[]//Length

Solids

PolyhedronData["Platonic"]

Show[PolyhedronData[#,"Image"],PlotLabel->Style[PolyhedronData[#,"Name"],FontSlant->"Italic"]]&/@%

PolyhedronData

Interactive Exploration

CollapseFriends[x_]:=StringSplit[Flatten[{x}][[1]],DigitCharacter][[1]]

Friends=First/@Split[PolyhedronData[],CollapseFriends[#1]==CollapseFriends[#2]&];

Manipulate[Column[{PolyhedronData[g],PolyhedronData[g,p]}],{g,Friends},{p,Complement@@PolyhedronData/@{"Properties","Classes"}}]

Other Nice Examples

Dual compounds

Graphics3D[PolyhedronData[{"PentagonalHexecontahedron",#},"Faces"]&/@{"Laevo","Dextro"}]

Nets

Show[Graphics[PolyhedronData[#,"NetImage"]],PlotLabel->Style[PolyhedronData[#,"Name"],FontSlant->"Italic"]]&/@PolyhedronData["Platonic"]

GraphicsGrid[Partition[Graphics[Tooltip[PolyhedronData[#,"NetImage"][[1]],PolyhedronData[#,"Name"]]]&/@PolyhedronData["NetImage","Defined"],20]]

Skeletons

GraphicsGrid[Partition[Graphics[Tooltip[PolyhedronData[#,"SkeletonImage"][[1]],PolyhedronData[#,"Name"]]]&/@PolyhedronData["SkeletonImage","Defined"],20]]

Send us your ideas for data coverage

datasources@wolfram.com

Developers

Data Paclet Infrastructure Developers:

Mike Pilat: data paclet infrastructure and development tools

Import["https://internal.wolfram.com/admin/personnel/people/images/mpilat.jpg"]

Todd Gayley: paclet system

Import["https://internal.wolfram.com/admin/personnel/people/images/tgayley.jpg"]

Data Function Developers

Charles Pooh: KnotData & LatticeData

Import["https://internal.wolfram.com/admin/personnel/people/images/charlesp.jpg"]

Chris French: ExampleData

Import["https://internal.wolfram.com/admin/personnel/people/images/cfrench.jpg"]

Eric Weisstein: GraphData & PolyhedronData

Import["https://internal.wolfram.com/admin/personnel/people/images/eww.jpg"]

Fred Meinberg: FinancialData & WordData

Import["https://internal.wolfram.com/admin/personnel/people/images/fredm.jpg"]

Jamie Williams: ChemicalData, ElementData, IsotopeData, & ParticleData

Import["https://internal.wolfram.com/admin/personnel/people/images/jamiew.jpg"]

Jason Cawley: CountryData & CityData

Import["http://www.wolframscience.com/summerschool/2003/images/cawley.jpg"]

Jeff Bryant: AstronomicalData

Import["https://internal.wolfram.com/admin/personnel/people/images/jeffb.jpg"]

Joshua Martell: WeatherData

Import["https://internal.wolfram.com/admin/personnel/people/images/joshuam.jpg"]

Kovas Boguta: CityData

Import["https://internal.wolfram.com/admin/personnel/people/images/kovasb.jpg"]

Paul-Jean Letourneau: GenomeData, GenomeLookup, ProteinData, SpeciesData

Import["https://internal.wolfram.com/admin/personnel/people/images/paul-jean.jpg"]