C++ Neural Networks and Fuzzy Logic by Valluru B. Rao M&T Books, IDG Books Worldwide, Inc. ISBN: 1558515526   Pub Date: 06/01/95

Operation

The first thing to do with your simulator is to train a network with an architecture you choose. You can select the number of layers and the number of hidden layers for your network. Keep in mind that the input and output layer sizes are dictated by the input patterns you are presenting to the network and the outputs you seek from the network. Once you decide on an architecture, perhaps a simple three-layer network with one hidden layer, you prepare training data for it and save the data in the training.dat file. After this you are ready to train. You provide the simulator with the following information:

•  The mode (select 1 for training)

•  The values for the error tolerance and the learning rate parameter, lambda or beta

•  The maximum number of cycles, or passes through the training data you’d like to try

•  The number of layers (between three and five, three implies one hidden layer, while five implies three hidden layers)

•  The size for each layer, from the input to the output

The simulator then begins training and reports the current cycle number and the average error for each cycle. You should watch the error to see that it is on the whole decreasing with time. If it is not, you should restart the simulation, because this will start with a brand new set of random weights and give you another, possibly better, solution. Note that there will be legitimate periods where the error may increase for some time. Once the simulation is done you will see information about the number of cycles and patterns used, and the total and average error that resulted. The weights are saved in the weights.dat file. You can rename this file to use this particular state of the network later. You can infer the size and number of layers from the information in this file, as will be shown in the next section for the weights.dat file format. You can have a peek at the output.dat file to see the kind of training result you have achieved. To get a full-blown accounting of each pattern and the match to that pattern, copy the training file to the test file and delete the output information from it. You can then run Test mode to get a full list of all the input stimuli and responses in the output.dat file.

Summary of Files Used in the Backpropagation Simulator

Here is a list of the files for your reference, as well as what they are used for.

•  weights.dat You can look at this file to see the weights for the network. It shows the layer number followed by the weights that feed into the layer. The first layer, or input layer, layer zero, does not have any weights associated with it. An example of the weights.dat file is shown as follows for a network with three layers of sizes 3, 5, and 2. Note that the row width for layer n matches the column length for layer n + 1:

1 -0.199660 -0.859660 -0.339660 -0.25966 0.520340
1  0.292860 -0.487140 0.212860 -0.967140 -0.427140
1  0.542106 -0.177894 0.322106 -0.977894 0.562106
2 -0.175350 -0.835350
2 -0.330167 -0.250167
2  0.503317 0.283317
2 -0.477158 0.222842
2 -0.928322 -0.388322

In this weights file the row width for layer 1 is 5, corresponding to the output of that (middle) layer. The input for the layer is the column length, which is 3, just as specified. For layer 2, the output size is the row width, which is 2, and the input size is the column length, 5, which is the same as the output for the middle layer. You can read the weights file to find out how things look.

•  training.dat This file contains the input patterns for training. You can have as large a file as you’d like without degrading the performance of the simulator. The simulator caches data in memory for processing. This is to improve the speed of the simulation since disk accesses are expensive in time. A data buffer, which has a maximum size specified in a #define statement in the program, is filled with data from the training.dat file whenever data is needed. The format for the training.dat file has been shown in the Training mode section.

•  test.dat The test.dat file is just like the training.dat file but without expected outputs. You use this file with a trained neural network in Test mode to see what responses you get for untrained data.

•  output.dat The output.dat file contains the results of the simulation. In Test mode, the input and output vectors are shown for all pattern vectors. In the Simulator mode, the expected output is also shown, but only the last vector in the training set is presented, since the training set is usually quite large.
Shown here is an example of an output file in Training mode:

for input vector:
0.400000  -0.400000
output vector is:
0.880095
expected output vector is:
0.900000