Activity 20: Neural Networks

In this activity, we use neural networks as another method to classify our objects into groups depending on the values of the extracted features. Similar to activity 19, we use 2 classes: kwekkwek and pillows. However, only 2 features are used to differentiate the 2 groups for simplicity. These are the length to height ratio and the red component of color.

A neural network is a computational model of how neurons in the brain work. It is a preferred alternative to linear discriminant analysis in the pattern recognition because one does not need heuristics and recognition rules to perform classification. Instead neural networks “learn” the rules of a mapping by example. Although it may take a long time for a network to train, its recognition processing speed is faster once it has learned.

source: M. Soriano, A20 - Neural Networks.pdf
____________________________________________________________________

A code was already prepared by J. Tugaff and it only needs to be modified depending on the number and values of the input features. For our purposes, we have:

2 input features
4 training objects from each class (kwekkwek and pillows)
4 test objects from each class

We first train our neural network using the training set.

Length/Height, Red component --(kwek kwek; pillows)
0.97, 0.74
1.00, 0.77
0.96, 0.73
0.91, 0.74
0.96, 0.30
1.00, 0.30
1.02, 0.33
1.02, 0.33
____________________________________________________________________

//original code c/o Jeric Tugaff

//ensure the same starting point each time
rand('seed',0);

//network def.
//neurons per layer, including input
//2 neurons in the input layer, 2 in the hidden layer and 1 in the output layer
N = [2,2,1];

//inputs, 1st column = length/height, 2nd column = red component
//training set
x = [0.97, 0.74;
1.00, 0.77;
0.96, 0.73;
0.91, 0.74;
0.96, 0.30;
1.00, 0.30;
1.02, 0.33;
1.02, 0.33]';

//targets, 0 if kwekkwek and 1 if pillows
t = [0 0 0 0 1 1 1 1];
//learning rate is 0.1 and 0 is the threshold for the error tolerated by the network
lp = [0.1,0];

W = ann_FF_init(N);

//training cycles
T = 1000;
W = ann_FF_Std_online(x,t,N,W,lp,T);
//x is the training, t is the output, W is the initialized weights,
//N is the NN architecture, lp is the learning rate and T is the number of iterations

//full run
ann_FF_run(x,N,W)
//the network N was tested using x as the test set, and W as the weights of the connections
____________________________________________________________________

We expect that this code will output values close to (0, 0, 0, 0, 1, 1, 1, 1), meaning that the first 4 objects will be classified as members of the 'kwekkwek' class while the last 4 objects will be classified as members of the 'pillows' class. The actual results are (0.1448030, 0.1291128, 0.1506565, 0.1342528, 0.8876302, 0.8956291, 0.8738883, 0.8738883). We can now say that our neural network has "learned" the features of each class and has sorted the objects into the correct groups. We now use the test set to see whether the network can classify the other objects accurately as well.

Length/Height, Red component --(kwek kwek; pillows)
0.67, 0.77
0.81, 0.79
1.08, 0.78
1.00, 0.81
1.09, 0.35
1.12, 0.35
1.05, 0.35
0.94, 0.32

Using the training parameters, the resulting values are (0.0900170, 0.0954682, 0.1355222, 0.1071037, 0.8711712, 0.8782258, 0.8610966, 0.8642343). A 100% correct classification was achieved. Next, we try to tweak the parameters (learning rate and training cycles) to see whether the values will go closer to 0 and 1 or whether the clasification will still remain 100% correct.

learning rate = from 0.1 --> change to 1.0
training cycles = from 1000 --> change to 400
result: (0.0201587, 0.0220962, 0.0384002, 0.0265042, 0.9631881, 0.9668789, 0.9575001, 0.9582095)

The values are closer to 0 and 1 using the new training parameters and the classification is still 100% correct. Now, let us try to change the order of the test set to make sure that the 100% correct classification does not depend on the order.

Length/Height, Red component --(kwek kwek; pillows)
0.67, 0.77
1.05, 0.35
1.08, 0.78
1.09, 0.35
0.81, 0.79
1.12, 0.35
0.94, 0.32
1.00, 0.81

learning rate = 1.0
training cycles = 400
result: (0.0201587, 0.9575001, 0.0384002, 0.9631881, 0.0220962, 0.9668789, 0.9582095, 0.0265042)

The classification is still 100% correct and we have shown that the order of the objects in the input matrix is not important.
____________________________________________________________________

I give myself 10 points for this activity because of the 100% correct classification of the objects into their respective classes. Thanks to Jeric Tugaff for the ANN toolbox and the code and Mark Leo Bejemino for helping me to apply the code to my objects and extracted features.