Thursday, December 11, 2008

Sample code for CvANN_MLP

Here is a sample for using CvANN_MLP:

// CvANN_MLPDemo.cpp : Defines the entry point for the console application.
//

// Sample code for CvANN_MLP Artificial Neural Network module of OpenCV.
// We stick pretty much to the basics here. See full reference at
// http://www.seas.upenn.edu/~bensapp/opencvdocs/ref/opencvref_ml.htm
//
// Author : Feroz M Basheer
//
// DISCLAIMER:THIS CODE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, WHETHER
// EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE WARRANTIES OF
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. THE
// ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THIS CODE IS WITH YOU.

#include "stdafx.h"
#include "cv.h"
#include "ml.h"

// The neural network
CvANN_MLP machineBrain;

// Read the training data and train the network.
void trainMachine()
{
int i;
//The number of training samples.
int train_sample_count;

//The training data matrix.
//Note that we are limiting the number of training data samples to 1000 here.
//The data sample consists of two inputs and an output. That's why 3.
float td[1000][3];

//Read the training file
/*
A sample file contents(say we are training the network for generating
the mean given two numbers) would be:

5
12 16 14
10 5 7.5
8 10 9
5 4 4.5
12 6 9

*/
FILE *fin;
fin = fopen("train.txt", "r");

//Get the number of samples.
fscanf(fin, "%d", &train_sample_count);
printf("Found training file with %d samples...\n", train_sample_count);

//Create the matrices

//Input data samples. Matrix of order (train_sample_count x 2)
CvMat* trainData = cvCreateMat(train_sample_count, 2, CV_32FC1);

//Output data samples. Matrix of order (train_sample_count x 1)
CvMat* trainClasses = cvCreateMat(train_sample_count, 1, CV_32FC1);

//The weight of each training data sample. We'll later set all to equal weights.
CvMat* sampleWts = cvCreateMat(train_sample_count, 1, CV_32FC1);

//The matrix representation of our ANN. We'll have four layers.
CvMat* neuralLayers = cvCreateMat(4, 1, CV_32SC1);

CvMat trainData1, trainClasses1, neuralLayers1, sampleWts1;

cvGetRows(trainData, &trainData1, 0, train_sample_count);
cvGetRows(trainClasses, &trainClasses1, 0, train_sample_count);
cvGetRows(trainClasses, &trainClasses1, 0, train_sample_count);
cvGetRows(sampleWts, &sampleWts1, 0, train_sample_count);
cvGetRows(neuralLayers, &neuralLayers1, 0, 4);

//Setting the number of neurons on each layer of the ANN
/*
We have in Layer 1: 2 neurons (2 inputs)
Layer 2: 3 neurons (hidden layer)
Layer 3: 3 neurons (hidden layer)
Layer 4: 1 neurons (1 output)
*/
cvSet1D(&neuralLayers1, 0, cvScalar(2));
cvSet1D(&neuralLayers1, 1, cvScalar(3));
cvSet1D(&neuralLayers1, 2, cvScalar(3));
cvSet1D(&neuralLayers1, 3, cvScalar(1));

//Read and populate the samples.
for (i=0;i<train_sample_count;i++)
fscanf(fin,"%f %f %f",&td[i][0],&td[i][1],&td[i][2]);

fclose(fin);

//Assemble the ML training data.
for (i=0; i<train_sample_count; i++)
{
//Input 1
cvSetReal2D(&trainData1, i, 0, td[i][0]);
//Input 2
cvSetReal2D(&trainData1, i, 1, td[i][1]);
//Output
cvSet1D(&trainClasses1, i, cvScalar(td[i][2]));
//Weight (setting everything to 1)
cvSet1D(&sampleWts1, i, cvScalar(1));
}

//Create our ANN.
machineBrain.create(neuralLayers);

//Train it with our data.
//See the Machine learning reference at http://www.seas.upenn.edu/~bensapp/opencvdocs/ref/opencvref_ml.htm#ch_ann
machineBrain.train(
trainData,
trainClasses,
sampleWts,
0,
CvANN_MLP_TrainParams(
cvTermCriteria(
CV_TERMCRIT_ITER+CV_TERMCRIT_EPS,
100000,
1.0
),
CvANN_MLP_TrainParams::BACKPROP,
0.01,
0.05
)
);
}

// Predict the output with the trained ANN given the two inputs.
void Predict(float data1, float data2)
{
float _sample[2];
CvMat sample = cvMat(1, 2, CV_32FC1, _sample);
float _predout[1];
CvMat predout = cvMat(1, 1, CV_32FC1, _predout);
sample.data.fl[0] = data1;
sample.data.fl[1] = data2;

machineBrain.predict(&sample, &predout);

printf("%f \n",predout.data.fl[0]);

}

int _tmain(int argc, _TCHAR* argv[])
{
int wait;

// Train the neural network with the samples
trainMachine();

// Now try predicting some values with the trained network
Predict(15.0,20.0);
Predict(1.0,5.0);
Predict(12.0,3.0);

//I'll wait for an integer. :)
scanf("%d",&wait);
return 0;
}

15 comments:

Unknown said...

Why this example leads to wrong prediction, when I run the program.
The result on the screen is that:
Found training file with 5 samples...
12.689177
7.732689
10.838544

I try to get the return of the method train, i.e.
int num =machineBrain.train(...)
which is the number of done iterations, and I found num = 1.

I wants to know where the problem is. Thank you.

Unknown said...

sample.data.fl[0] = data1;
sample.data.fl[1] = data2; // changed to data2

Feroz M Basheer said...

@Agat50
Thanks! I had missed it. :) changed it now.

@dkai
I hope this solves the issue.

Unknown said...

Hello, I have newbie questions. Why are you using:

CvMat trainData1, trainClasses1, neuralLayers1, sampleWts1;

cvGetRows(trainData, &trainData1, 0, train_sample_count);
cvGetRows(trainClasses, &trainClasses1, 0, train_sample_count);
cvGetRows(trainClasses, &trainClasses1, 0, train_sample_count);
cvGetRows(sampleWts, &sampleWts1, 0, train_sample_count);
cvGetRows(neuralLayers, &neuralLayers1, 0, 4);

?

And could you explain what does 0.01 and 0.05 in train() method mean?
I've turned your example into XOR network, and have had some problems, but playing with those 2 values helped.

dguerra said...

I doesn't predict well the values with the given data training set. does anyone know why?
Cheers

dguerra said...
This comment has been removed by the author.
ginadcv said...

I have trained it with 81 samples and still its not giving the desired result. Moreover in this code the use of back propagation is being done thus is there a need to initialize the sample weights?? As per the reference there is no need. Also I would like to know the output you have received.

ginadcv said...
This comment has been removed by the author.
ginadcv said...

I think the epsilon value of 1.0 is responsible for errors in the calculation. If we alter it then the number of iterations will change. It has worked for my code.

Konstantin said...

Yes, play with epsilon value (0.001 should be Ok) and 5 training samples is not enough. I think samples count should exceed total amount of neurons. With 9 samples and smaller epsilon accurancy icrease up to ~10%.

TWright said...

For those trying this out and playing with it:
CvANN_MLP_TrainParams::BACKPROP,
0.01, //bp_dw_scale
0.5 //bp_moment_scale

Those numbers should be smaller. .1 and .1 work well.

CV_TERMCRIT_ITER+CV_TERMCRIT_EPS,
100000, //not totally certain what this does
0.01 //epsilon
.01 is too large, try .001

I have 10 training samples and output looks much better than when I first copy pasted it.

Unknown said...

I've tried this code, and I have this kind of error: "The array of layer neuron counters must be an integer vector". Why?
The only difference that I have introduced is that the number of my inputs is 59.

Unknown said...
This comment has been removed by the author.
Unknown said...
This comment has been removed by the author.
DanyaMariePerez said...

I tried this code but my Codeblocks 10.05 crashes whenever the machineBrain.train() is called. Why is that?