Machine Learning

Etienne Bernard

Wolfram Research - Advanced Research Group

Introduction

Learning a task from examples

Image recognition

->cat,  ->dog, ...

Text classification

"What a nice day"->positive,"I got fired"->negative,...

Prediction from features

(6 yo,female)->1.15 m,(11 yo,male)->1.48 m,...

...

Automatization

Preprocessing

Feature extraction

->{1.4, 5.7, ...}

“What a nice day” -> {“What”, “a”, “nice”, “day”}

Input missing

{6, Missing[], 4} -> {6, 7, 4}

Nominal encoding

"B" -> {0., 1., 0., ...}

Standardization

Dimensionality reduction

Feature selection

Model selection

Linear Regression

Logistic Regression

Nearest Neighbors

Random Forest

, ...

Naive Bayes classifier

Markov models

Feedforward Neural Network

Support Vector Machine

Hyperparameter selection

Number of neighbors, number of trees, number of layers, regularization parameters, ...

Cross validation

Predict & Classify in practice

Basic use

p=Predict[{1->1,2->3,3->5,4->8}]

p[2.5]

c=Classify[{1->"A", 2->"A",3->"B",4->"B"}]

c[2.3]

c=Classify[{{1,"red"}->"A",{2,"blue"}->"A",{3,"blue"}->"B",{4,"red"}->"B"}]

c[{1.5, "red"}]

Titanic survival

titanicdata = ExampleData[{"MachineLearning","Titanic"},"Data"];

RandomSample[titanicdata,10]

c=Classify[titanicdata]

c[{"1st", 20,"female"}]

c[{"3rd", 50,"male"}]

c[{"1st", 20,"female"}, "Probabilities"]

c[{"3rd", 50,"male"}, "Probabilities"]

ClassifierInformation[c]

ClassifierInformation[c, FeatureTypes]

c=Classify[titanicdata, PerformanceGoal->"TrainingSpeed"]

c=Classify[titanicdata, Method->"LogisticRegression"]

ClassifierInformation[c]

proba[class_,age_,sex_]:=c[{class,age,sex},{"Probability","survived"}];

Plot[{proba["1st",x,"female"],proba["3rd",x,"female"],proba["1st",x,"male"],proba["3rd",x,"male"]}, {x,0,100},PlotLegends->{"female, 1st class", "female, 3rd class", "male, 1st class", "male, 3rd class"}, Frame->True,FrameLabel-> {"Age (years)", "Survival probability"}]

Digit recognition

trainingset = ExampleData[{"MachineLearning","MNIST"},"TrainingData"];

RandomSample[trainingset, 10]

c=Classify[RandomSample[trainingset, 10000], PerformanceGoal->"TrainingSpeed"]

testset = ExampleData[{"MachineLearning","MNIST"},"TestData"];

cm=ClassifierMeasurements[c,testset]

cm["Accuracy"]

cm["ConfusionMatrixPlot"]

cm[{"Examples", 2->5}]

Text

catordog=Classify[{"the cat is grey"->,"my cat is fast"->,"this dog is scary"-> ,"the big dog"->}]

catordog[{"nice cat","what a dog"}]

Gauge value predictor

(*gaugeset=Image[AngularGauge[#]]->#&/@RandomReal[1,10];*)

gaugeset = {->0.908,->0.378,->0.575,->0.265,->0.607`,->0.139,->0.171,->0.280,->0.047,->0.032,->0.510,->0.211`,->0.409,->0.105,->0.624,->0.845,->0.620,->0.940,->0.892,->0.471,->0.111,->0.525,->0.629,->0.792,->0.187,->0.731,->0.370,->0.377,->0.710,->0.597,->0.444,->0.719,->0.840,->0.764,->0.771,->0.257,->0.813,->0.153,->0.607,->0.936,->0.743,->0.690,->0.893,->0.197,->0.147,->0.718,->0.066,->0.957,->0.464,->0.574};

gaugevalue=Predict[gaugeset]

gaugevalue[]

gaugevalue[, "Distribution"]

DynamicModule[{x=0},Row[{AngularGauge[Dynamic[x]],Dynamic[AngularGauge@gaugevalue[Image[AngularGauge[x]]]]}]]

Built-in functions

Language

Classify["Language",{"the house is blue", "la maison est bleu", "la casa es azul", "das Haus ist blau", "房子是蓝色的", "المنزل باللون الأزرق", "будинок синій"}]

DynamicModule[{text = "" },Column[{InputField[Dynamic[text], String, ContinuousAction-> True, FieldHint->"Enter a string"],
Dynamic[Classify["Language",text, "TopProbabilities"]]}
]
]

FacebookTopic

Classify["FacebookTopic", {"I bought a new computer", "happy birthday!", "this skirt looks nice"}]

Classify["FacebookTopic","I love carrots", "TopProbabilities"]

NameAge

Predict["NameAge","Claire"]

distribution = Predict["NameAge", "Claire", "Distribution"]

Plot[PDF[distribution,x],{x,0,100},Exclusions->None,PlotRange->All, Filling->Bottom]

Classify & Predict - Conclusion

Classification, regression

Automatic

Simple

Custom + Buit-in functions

More Information:

http://reference.wolfram.com/language/ref/Classify.html

http://reference.wolfram.com/language/ref/Predict.html

http://reference.wolfram.com/language/guide/MachineLearning.html

http://www.wolfram.com/mathematica/new-in-10/highly-automated-machine-learning/

http://www.wolfram.com/mathematica/new-in-10/built-in-classifier-collection/

Come to the workshop!

Future...

Classify & Predict

Models (decision tree, gaussian processes, graphical models ...)

Semi-supervised learning

Online learning

Extend[ClassifierFunction[...], {...}]

Model combination

Classify[{{"the cat is grey",}->,{"my cat is fast", }->,{"this dog is scary",}-> ,{"the big dog", }->}]

Deep learning (e.g. Convolutional Neural Network)

...

Other functions

Multi-label classification/prediction

Recommender system (Recommend)

Segmentation, tagging (Segment, Tag)

Distribution estimation (LearnDistribution)

...

Loading

Quit[]

PacletDirectoryAdd["~/Work/eclipse/Mathematica/ARG"];
PacletDirectoryAdd["~/Work/eclipse/Mathematica/ARG/Classifiers"];
PacletDirectoryAdd["~/Work/eclipse/Mathematica/ARG/Predictors"];

Quiet[Get["MachineLearning`"]]