neural networks - overtraining

[bolpx001]

Hope somebody can help, I have used BA to create a large sample of data and I am now trying to train a Neural Network to use it. Does anybody know or have any suggestion how as to how to know if it is over-trained - any help much appreciated - Paul

[osknows]

With most forms of multivariate analysis it's better to split the data into a training set, testing set and one or more validation sets. The training and testing set are used to build the model and tested against the validation sets. If the results are worse against the validation set it can indicate overfitting.

If the sample size is small, cross-validation can sometimes help.

[bolpx001]

Thanks for the reply, I have done that and have a training set of 385,a testing set of 125 and a validation set of 6600. I have tried various ideas and training sizes but have no idea of how big to make the training iterations. 200, 2000, 20000. I was hoping for some ideas on how to spot over training. The real problem I suspect is that I am out of my depth - but then God loves a trier - Paul

[rourkem]

Hi. I'm trying out things on my end too. But so far, no luck as of yet. I think there's a way to do this but just a tough one to come up.

[doris_day]

I think it also depends on the number of parameters you have within the sets and their degree of independence from one another. Its surprising how many parameters can have underlying dependence which you can be completely unaware of.
I did a lot of work with neural nets in the past and had a lot of fun.