Unsupervised Analysis

machine learning framework for Mathematica

Unsupervised analysis is concerned with analysis of data without an explicit goal parameter. This covers tasks like clustering and finding general relations within the data. Often unsupervised analysis is performed prior to supervised analysis, in order to identify groups of interest, which are analyzed afterwards, or to clean and/or reduce the amount of data.

In this notebook we will describe several methods and scenarios where unsupervised analysis can help to understand the underlying structure of the data. First we will present a special kind of neural nets—so called self-organizing maps or SOMs—which are used to visualize high dimensional data and additionally to clean and reduce the available data. Then we will describe different clustering algorithms, which can be used to identify regions of similarity within the data. We will also show how supervised methods can be used to create cluster descriptions.

Initialization of the system

Needs["Graphics`Graphics`"]
Needs["mlf`"]

Basics—artificial data set

In this section we will briefly describe the methods available for unsupervised learning. First we explain how self-organizing maps can be computed. Then we show how different clustering methods can be used to find regions of similarity in the data. Finally we apply supervised learning methods to create descriptions for the clusters found.

Set-up data

samples = 
        1000;

    dims = 2;

    data1 = 
        Table[{Random[]*0.25, Random[]*0.25, 1}, {i, 
            samples*0.25}];

    data2 = 
        Table[{0.25 + Random[]*0.4, 0.5 + Random[]*0.5, 2}, {i, 
            samples}];

    data3 = 
        Table[{0.75 + Random[]*0.25, Random[], 3}, {i, 
            samples}];

    sampleData = 
        DataSet[Join[data1, data2, 
            data3], {"x", "y", "class"}];

Show@
        Graphics[
          Join[
PlotAttributes[sampleData, Dims -> {1, 2}, 
              Goal -> 3, PointSize -> 0.01]
], 
          Axes -> True, PlotRange -> {{0, 1}, {0, 1}}, 
          AspectRatio -> 1];

Compute the SOM

Self-organizing maps (SOM) were introduced by T. Kohonen in 1982. His idea was to create a neural network to represent the input space using the topological structure of a grid to store neighborhood relations. In contrast to most neural network methods that use the desired result to compute the weights of the network, SOMs need no reference output to tune their parameters, but are able to organize themselves (unsupervised learning). 
An SOM defines a mapping of an n-dimensional input space R to an m-dimensional output space C (we use m = 2). The output space consists of N neurons. They are embedded in the topological structure of C, which may be a m-dimensional grid or any other graph structure (e.g., a hexagonal grid). To each neuron s in C of the output space, a parametric weight vector in the input space is associated. If a new sample is presented to the map, it is associated with that node, which is closest to it.

To compute a SOM the command CreateSOM[] is used. We now compute a 5 x 5 SOM, considering only the two data dimensions.

\(\(som = 
        CreateSOM[
sampleData, 
Weights -> {1, 1, 0}, 
Size -> {5, 
              5}, 
Errorlevel -> 
            Automatic, 
Type -> "BATCH"
];\)\)

Now we have computed an SOM with 5 x 5 = 25 node vectors. The net has been trained with the BATCH algorithm and weights of {1,1,0} ( i.e., the third attribute was not taken into account for training).

To inspect the weight vectors of the SOM, the GetData[] command can be used.

GetData[som]

We can plot the result easily. Notice that we have trained the SOM implicitly with the class information. This means that each node has the class information, however, it was not used to compute the neighborhood relations.

Show@
        Graphics[
          Join[
PlotAttributes[sampleData, Dims -> {1, 2}, 
              Goal -> 3, 
              PointSize -> 0.005], 
{GrayLevel[0]}, 
            PlotSOMGrid[som, Dims -> {1, 2}, Goal -> 3, 
              PointSize -> 0.02, 
              Thickness -> 0.0025]
], 
          Axes -> True, PlotRange -> {{0, 1}, {0, 1}}, 
          AspectRatio -> 1];

Recall data using a SOM

Data can be recalled by just calling the SOM with the test data

recallData = som[sampleData];

or by calling the RecallBM[] function:

recallData = 
        RecallBM[sampleData, som, NStrength -> 5.0, 
          NSize -> 5];

Show@
        Graphics[
          Join[
PlotAttributes[recallData, Dims -> {1, 2}, 
              Goal -> 3, 
              PointSize -> 0.005], 
{GrayLevel[0]}, 
            PlotSOMGrid[som, Dims -> {1, 2}, Goal -> 3, 
              PointSize -> 0.02, 
              Thickness -> 0.0025]
], 
          Axes -> True, PlotRange -> {{0, 1}, {0, 1}}, 
          AspectRatio -> 1];

We can see how the SOM covers the data space. As the recall mechanism includes interpolation, the samples are distributed uniformly within the boundaries of the net. On the edges, however, we can see that samples lying outside are mapped into it.

Compute clusters with WARD

Then we compute a new clustering using the WARD method. As we do not know the optimal number of clusters in advance, we will try to identify this by specifying Size->1. This feature is only available for WARD clustering. As WARD clustering is very computational expansive, we suggest only appling it to SOMs!

centersWARDinit = 
        CreateWARD[
som, 
Weights -> {1, 
              1, 0}, 
Size -> 
            1
];

To plot the clustering index, we have to extract the WIndex parameter from the obtained clustering first. Afterwards, we can plot the result using the PlotLineGraph[] command.

wIndex = GetParam[
            centersWARDinit, "WIndex"][[1]];
PlotLineGraph[wIndex];

This plot shows us that the most likely number of clusters are 2, 4, 6, and 9. We can now try out these clusters to obtain the optimal number of clusters for our purpose.

An alternative way of displaying the cluster index is to use a dentogram. A dentogram describes how clusters are joined. This feature is also only available for WARD clustering.

Show@Graphics[PlotClusteringTree[centersWARDinit]];

The numbers used in the dentogram correspond to the numbering of the nodes. We can plot the SOM with the corresponding labels by specifying the Labels option for PlotSOMGrid[].

labels = Range[25] - 1;

Show@Graphics@
          PlotSOMGrid[som, Dims -> {1, 2}, Thickness -> 0. , 
            Labels -> labels, PointSize -> 0.04, 
            AspectRatio -> 1];

Although the WARD index suggested we should use 2 or 4 clusters, we will now compute the final clustering of the training data for three clusters.

centersWARD = 
        CreateWARD[
som, 
Weights -> {1, 
              1, 0}, 
Size -> 
            3
];

Now we can plot the new clusters.

Show@
        Graphics[
          Join[
PlotAttributes[sampleData, Dims -> {1, 2}, 
              PointSize -> 0.01], 
{GrayLevel[0]}, 
            PlotSOMGrid[som, Dims -> {1, 2}, Goal -> centersWARD, 
              PointSize -> 0.05, 
              Thickness -> 0.0025]
], 
          Axes -> True, PlotRange -> {{0, 1}, {0, 1}}, 
          AspectRatio -> 1];

We can see that the blue cluster is very sustained. This is the reason the WARD index failed to indicate the correct number of clusters.

Compute clusters with k-means

When looking on the final clustering found by the WARD method, we can see that one node is in the center of two clusters. As the WARD clustering method assigns each node to only one cluster, this node can cause misinterpretation. For such cases it is much more appropriate to use fuzzy clusters, where each node can belong to multiple clusters, too.

To compute a fuzzy clustering, we use the fuzzy k-means method. It is initialized by the CreateKMeans[] command. Note that the k-means clustering method is sensitive to initialization. Running the algorithm multiple times might give different results. See the next section to learn how to overcome this problem.

centersKMeans = 
        CreateKMeans[
sampleData, 
Weights -> {1, 1, 0}, 
Size -> 
            4, 
Fuzziness -> 
            50.0, 
SomFactor -> 
            10.0
];

As the k-means clusters are described by their cluster centers, we can print out the representative of each cluster.

GetParam[
          GetParam[
            centersKMeans, "Clusters"], "DataMatrix"][[1]] // 
      TableForm

Show@
        Graphics[
          Join[
PlotAttributes[sampleData, Dims -> {1, 2}, 
              Goal -> centersKMeans, 
              PointSize -> 0.01]
], Axes -> True, 
          PlotRange -> {{0, 1}, {0, 1}}];

Compute clusters by combining different methods

As it is often a critical task to identify the number of clusters and the initial cluster centers, it is possible to combine different methods.

In the following example, we use the WARD method to initialize the k-means clustering to overcome its sensitivity to initialization. This is done by adding the InitClusters->GetParam[centersWARD,"Clusters"] tag to the CreateKMeans[] command.

centersWARD = 
        CreateWARD[
som, 
Weights -> {1, 
              1, 0}, 
Size -> 
            3
];
\(\(centersKMeans = 
        CreateKMeans[
som, 
InitClusters -> GetParam[
              centersWARD, "Clusters"], 
Weights -> {1, 
              1, 0}, 
Size -> 
            3
];\)\)

Now we can plot the new clusters.

Show@
        Graphics[
          Join[
PlotAttributes[sampleData, Dims -> {1, 2}, 
              Goal -> 3, 
              PointSize -> 0.005], 
{GrayLevel[0]}, 
            PlotSOMGrid[som, Dims -> {1, 2}, Goal -> centersKMeans, 
              PointSize -> 0.05, 
              Thickness -> 0.0025]
], 
          Axes -> True, PlotRange -> {{0, 1}, {0, 1}}, 
          AspectRatio -> 1];

Create descriptions

centersKMeans = 
        CreateKMeans[
sampleData, 
Weights -> {1, 1, 0}, 
Size -> 
            4, 
Fuzziness -> 
            50.0, 
SomFactor -> 
            10.0
];

testPreds = 
        CreatePredicates[sampleData, {1, 2}, 6];
testVars = DefPredicateVars[testPreds];

descr = 
        CreateClusterDescriptions[sampleData, centersKMeans, testVars, 
          MaxRules -> 5];
PrintDescriptions[descr, Info -> False]

Simple analysis—iris data set

In this example we will analyze a simple data file that contains data about 150 iris flowers of different types. Each flower is described by five attributes. The first four characterize the blossom of the flower: sepal length, sepal width, pedal length, and pedal width.The last attribute specifies the category of irises to which the flower belongs.

Set-up data

First we load the data set from a file.

\(\(dataRaw = 
        Import[$TopDirectory <> "/AddOns/Applications/mlf/data/iris.txt", "Table"];\)\)
headers = dataRaw[[1]];
idata = 
        dataRaw[[1 + 
              Range[Length@dataRaw - 1]]];
irisData = 
        Def["IrisData", 
          DataSet[idata,  headers]];
Clear[dataRaw];

Computing the SOM

Then we will compute a SOM using CreateSOM[] and train it with the data about the flowers. For training we take only the first four attributes into account. The fifth attribute, however, is also present in the map and can be used for visualization or for recall.

\(\(somIris = 
        CreateSOM[
irisData, 
Size -> {5, 
              5}, 
Cooldown -> 
            0.5, 
Type -> "BATCH", 
Weights -> {1, 1, 1, 1, 
              0}
];\)\)

To visualize the distribution of the data, we create a simple contour plot using PlotSOMContour[]. In these plots, low values indicated as blue, high values of the attributes as red.

Each grid point in these plots represents a typical sample of the original data. Neighboring points are more similar then far distant ones.
A grid point belongs always to all dimensions (i.e., the upper-left corner is the same grid point in all plots). This allows us to draw conclusions like if sepal_length is low, then petal_length is also likely to be low. We can also see that the class attribute is well clustered and behaves similarly to the petal attributes. This indicates that there is a good chance to predict the class membership by inspecting only the flower blossoms.

PlotSOMContour[somIris];

Recall class membership

We can use the SOM to recall the class membership of a new data set by applying the SOM to the new data set or record.

recDS = somIris[irisData];

somIris[{{0, 0, 1.4, 0.2}}]

somIris[{{0, 0, 1.4, 0.2},{0,0,1,1}}]

We can also use the more flexible command RecallBM[] to ensure that only the first four attributes are used for recall.

recallData = 
        GetData@RecallBM[irisData, somIris, 
            Weights -> {1, 1, 1, 1, 0}];

origCl = GetData[irisData];

To compare the prediction and the correct classification, we will compute a evaluation matrix using CompEvalMatrix[].

recallClass = 
        Map[(If[
                Abs@Subtract[#[[1]], #[[2]]] < 
                  0.5, #[[1]], -1] &), 
          Transpose[{origCl[[All, 5]], 
              recallData[[All, 5]]}], {1}];

evalMatrix = 
        CompEvalMatrix[GetData[irisData, All, 5], recallClass];
N /@ evalMatrix // MatrixForm
PlotEvalChart3D@evalMatrix;

Similar to the supervised case, we can also isolate those samples that have not been classified correctly.

failedS = 
        Extract[GetData@irisData, 
          ExtractNonMatchingRecs[irisData, 5, 
            recallClass]];
failedS // TableForm

This plot shows which of the samples have not been classified correctly (gray circles).

Show@
        Graphics[
          Join[
PlotAttributes[failedS, Dims -> {3, 4}, 
              PointSize -> 0.02, 
              Thickness -> 0.0025], 
PlotAttributes[
              irisData, Dims -> {3, 4}, Goal -> 5, 
              PointSize -> 0.01, 
              Thickness -> 0.0025]
], 
          Axes -> True];

Compute clusters with WARD

To identify groups of similarity in the data, we will first use the WARD method. This method allows us to also identify the optimal number of clusters.

centersWARD = 
        CreateWARD[
somIris, 
Size -> 1, 
Weights -> {1, 1, 1, 1, 
              1}
];

wIndex = ("WIndex" /. 
              centersWARD)[[1]];
PlotLineGraph[wIndex];

centersWARD = 
        CreateWARD[
somIris, 
Size -> 3, 
Weights -> {1, 1, 1, 1, 
              1}
];

PlotSOMContour[somIris, 5];
PlotSOMContour[somIris, centersWARD];

Note that when we recall the node information according to a clustering, we only get the values of the best matching cluster center!

recallData = 
        GetData[RecallBM[irisData, centersWARD, 
            Weights -> {1, 1, 1, 1, 0}]];

recallClass = 
        Map[(If[Abs@Subtract[#[[1]], #[[2]]] < 0.5, #[[1]], 
                0] &), 
          Transpose[{origCl[[All, 5]], 
              recallData[[All, 5]]}], {1}];

evalMatrix = 
        CompEvalMatrix[GetData[irisData, All, 5], recallClass];
N /@ evalMatrix // MatrixForm
PlotEvalChart3D@evalMatrix;

failedS = 
        Extract[GetData@irisData, 
          ExtractNonMatchingRecs[irisData, 5, 
            recallClass]];
failedS // TableForm

Show@
        Graphics[
          Join[
PlotAttributes[failedS, Dims -> {3, 4}, 
              PointSize -> 0.02, 
              Thickness -> 0.0025], 
PlotAttributes[
              irisData, Dims -> {3, 4}, Goal -> 5, 
              PointSize -> 0.01, 
              Thickness -> 0.0025]
], 
          Axes -> True];

Create descriptions

testPreds = 
        CreatePredicates[irisData, {1, 2, 3, 4}, 
          3];
testVars = DefPredicateVars[testPreds];

descr = 
        CreateClusterDescriptions[somIris, centersWARD, 
          testVars];
PrintDescriptions[descr, Info -> False]

Discrete data set—animals

In this example we will analyze different animals based on a list of Boolean attributes for each of them.

Set up data

animalData = 
        Import[$TopDirectory <> "/AddOns/Applications/mlf/data/animals.data\
", "Table"];

TableForm[Transpose@animalData, TableSpacing -> {1, 1}]

animalLabels = 
        animalData[[Range[Length@animalData - 1] + 1, 
            1]];
featureLabels = 
        animalData[[1, 
            Range[Length@animalData[[1]] - 2] + 
              1]];
trainData = 
        DataSet[
animalData[[Range[
                  Length@animalData - 1] + 1, 
              Range[Length@animalData[[1]] - 2] + 
                1]], 
featureLabels];
testData = trainData;
Clear[tmp];
somSize = {5, 5};

Clustering the data set

To see which groups of animals there are, we compute a clustering first and create a set of descriptions for these clusters afterwards.

clustersData = 
        CreateWARD[
trainData, 
Size -> 4
];

testPreds = 
        CreatePredicatesBin[trainData, 
          Range@Length@featureLabels];
testVars = DefPredicateVars[testPreds];

To create a set of descriptions for the clusters, we use the CreateClusterDescriptions[] method.

descr = 
        CreateClusterDescriptions[trainData, clustersData, 
          testVars];
PrintDescriptions[descr, Info -> False]

To compare these descriptions with the animals in each cluster, we can recall the labels (animal names) belonging to each cluster using GetClusterLabels[].

GetClusterLabels[clustersData, animalLabels] // TableForm

Create SOM

To find out similarities between the animals, we can also compute a SOM and assign the animal names to the corresponding parts of the SOM afterwards using the GetSOMLabels[] and the PrintLabeledSOM[] commands.

somAnimals = 
        CreateSOM[trainData, Size -> somSize, Type -> "ONLINE", 
          Cooldown -> 2];

somLabels = 
        GetSOMLabels[somAnimals, testData, animalLabels];

PrintLabeledSOM[somLabels]

Now that we have projected the animals on the two-dimensional grid, we can compute a clustering.

We decide to compute 4 clusters.

centersWARD = 
        CreateWARD[
somAnimals, 
Size -> 4
];

$TextStyle = {FontWeight -> "Bold", 
          FontSize -> 16};
Show@
        Graphics[
{Hue[0.6, 0.2, 
              1], 
PlotSOMUGrid[somAnimals, 
              Goal -> centersWARD, Thickness -> 0.03, 
              Labels -> somLabels, 
              PointSize -> 0.04]
}, 
          PlotRange -> All];
Clear[$TextStyle];

Create descriptions for SOM clusters

To find out which common properties the animals in one cluster have, we will create a set of descriptions.

testPreds = 
        CreatePredicatesBin[somAnimals, 
          Range@Length@featureLabels];
testVars = DefPredicateVars[testPreds];

descr = 
        CreateClusterDescriptions[somAnimals, centersWARD, 
          testVars];
PrintDescriptions[descr, Info -> False]

It shows that these descriptions are more detailed then the previous ones. This is because the SOM also contains nodes with no animals assigned. These nodes, however, are also part of the descriptions.

GetClusterLabels[centersWARD, somLabels] // TableForm

Application example—project portfolio

In this example we will analyze the project portfolio of a software company. For that purpose the project managers have been asked to describe their project with respect to a list of 240 characteristic topics. In total we collected the descriptions of 31 projects.

Set up data

\(\(projDataRaw = 
        Import[$TopDirectory <> "/AddOns/Applications/mlf/data/projCharacteristic.txt", "Table"];\)\)
tmpData = Transpose[projDataRaw];
projData = 
        tmpData[[Range[Length@tmpData - 1] + 1, 
            Range[Length@tmpData[[1]] - 1] + 
              1]];
projLabels = 
        tmpData[[Range[Length@projData] + 1, 
            1]];
featureLabels = 
        tmpData[[1, 
            Range[Length@projDataRaw - 1] + 
              1]];
Clear[projDataRaw];
Clear[tmpData];

This is a list of the project names we are using.

projLabels

These topics are used to describe two projects.

featureLabels

A typical project description consists of a vector of zeros and ones, where one indicates that the project is concerned with the corresponding topic.

projData[[1]]

Create SOM

Next we train a SOM with the data. As we have only 31 records, we increase the Cooldown level.

trainData = projData;
trainDataSet = 
        Def["scchProjects", 
          DataSet[trainData, featureLabels]];
testDataSet = 
        DataSet[projData, featureLabels];
somSize = {10, 10};

somProjects = 
        CreateSOM[trainDataSet, Size -> somSize, 
          Type -> "ONLINE", Cooldown -> 2.0];

Now that we have computed a SOM, we can recall the labels according to our original data set and labels using GetSOMLabels[] and print the result using PrintLabeledSOM[].

somLabels = 
        GetSOMLabels[somProjects, testDataSet, projLabels];

PrintLabeledSOM[somLabels]

It is also possible to add the labels to the graphical plot of the SOM using the Labels option.

$TextStyle = {FontWeight -> "Bold", 
          FontSize -> 12};
Show@
        Graphics[
{Hue[0.6, 0.2, 
              1], 
PlotSOMUGrid[somProjects, 
              Labels -> somLabels, 
              PointSize -> 0.020]
}, 
          PlotRange -> All];
Clear[$TextStyle];

Identify position of a new project

If we want to know how a new project is related to the project already done (e.g., to contact the corresponding project manager), we can describe the new project using the previously defined topic list and map it onto the SOM. Then we can investigate the neighborhood of the node to find out relationships to other projects.

newProject = {1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
          0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 
          0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 
          0, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 
          0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 
          0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
          0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 0, 
          0, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 
          1, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 
          1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 
          0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 
          1, 1};

newLabels = 
        GetSOMLabels[somProjects, 
          DataSet@{newProject}, {"NewProject"}];

$TextStyle = {FontWeight -> "Bold", 
          FontSize -> 12};
Show@
        Graphics[
{Hue[0.6, 0.2, 
              1], 
PlotSOMUGrid[somProjects, 
              Labels -> newLabels, 
              PointSize -> 0.020]
}, 
          PlotRange -> All];
Clear[$TextStyle];

Projects concerning a specific topic

We can also look at which projects are related to a specific topic (e.g., knowledge).

actPos = Position[featureLabels, "knowledge"][[1, 
            1]];

First we will check which projects are directly related to the topic specified.

actProj = 
      Extract[projLabels, 
        Position[projData[[All, actPos]], 1]]

Next we will plot the SOM and mark those projects related to the topic with red and those not related with blue.

$TextStyle = {FontWeight -> "Bold", 
          FontSize -> 12};
Show@
        Graphics[
{Hue[0.6, 0.2, 
              1], 
PlotSOMUGrid[somProjects, 
              Goal -> actPos, Labels -> somLabels, 
              PointSize -> 0.020]
}, 
          PlotRange -> All];
Clear[$TextStyle];

Clusters in the data

To find out how the projects can be grouped together, we compute a WARD clustering.

Then we identify the most useful number of clusters.

centersWARD = 
        CreateWARD[
somProjects, 
Size -> 1
];

wIndex = ("WIndex" /. 
              centersWARD)[[1]];
PlotLineGraph[wIndex];

centersWARD = 
        CreateWARD[
somProjects, 
Size -> 8
];

$TextStyle = {FontWeight -> "Bold", 
          FontSize -> 12};
Show@
        Graphics[
{Hue[0.6, 0.2, 
              1], 
PlotSOMUGrid[somProjects, 
              Goal -> centersWARD, Labels -> somLabels, 
              PointSize -> 0.020]
}, 
          PlotRange -> All];
Clear[$TextStyle];

Create descriptions

To better understand what the obtained clusters are about, we will now compute a set of descriptions for each cluster.

testPreds = 
        CreatePredicatesBin[somProjects, Range@Length@featureLabels, 
          PredType -> "IS"];
testVars = DefPredicateVars[testPreds];

Options@CreateClusterDescriptions

descr = 
        CreateClusterDescriptions[somProjects, centersWARD, testVars, 
          MinSup -> 0.1];

PrintDescriptions[descr, Info -> False]

GetClusterLabels[centersWARD, somLabels] // TableForm