6) Activating a Classification Model and Appraising Performance.

This Blog entry is from the Neural Networks section in Learn R.

To recall the neural network, return a value between 0 and 1 depending on the likelihood that the record is fraudulent:

FraudRiskPredictions <- FraudRiskNeuralNetwork$net.result
1.png

Run the line of script to console:

2.png

Peeking the results with the head() function:

head(FraudRiskPredictions)
3.png

Run the line of script to console:

4.png

It can be seen that numeric values, between 0 and 1, have been returned.  The closer to one, the more likely that the record is fraudulent.   To assert a proper classification, so that a confusion matrix may be plotted to appraise performance of the model, create a vector contains a 1 where the value of FraudRiskPredictions > 0.5, else 0, yet wrapping FraudRiskPrediction with the unlist() function to transform the list output to a vector:

IsFraud <- ifelse(unlist(FraudRiskPredictions) > 0.5,1,0)
5.png

Run the line of script to console:

6.png

As has become customary, use a confusion matrix to appraise the value of the classifier:

library("gmodels")
CrossTable(FraudRisk$Dependent, IsFraud)
7.png

Run the block of script to console:

8.png

In this example, it can be seen that 720 records were classified as being fraudulent correctly.  In total, it can be seen that 901 records were classified, so the accuracy rate on predicting fraud is 79.9%, a substantial uplift on the logistic regression models created in procedure 93.  It is well worth mentioning, that for classification problems, less is very often more and rather than increase network complexity by adding more and more hidden layers and processing elements, it is often more efficient to create many more abstracted variables backed by intuitive judgement and domain expertise.