Cnns On Keras Converge To The Same Value No Matter The Input

I've been learning Keras recently and I tried my hand at the CIFAR10 dataset with CNNs. However, the model I trained (you can run the code here) returns the same answer for every i

Solution 1:

You have forgotten to normalize the images. Currently, the values in x_train are in the range [0,255]. This causes large gradient updates and stalls training process. One simple normalization scheme in this case would be:

x_train = x_train.astype('float32') / 255
x_test = x_test.astype('float32') / 255

This causes the values to fall in the range [0,1]. Then you would surely see that the training progresses.

---

A more sophisticated normalization scheme involves feature-wise (i.e. pixel-wise) normalization or centering. In this approach we normalize all the images such that each pixel in all the images have a mean of zero and a standard deviation of one (i.e. they mostly fall in the range [-1,1]):

# make sure values are float
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')

x_mean = x_train.mean(axis=0)
x_train -= x_mean
x_std = x_train.std(axis=0)
x_train /= x_std + 1e-8# add a small constant to prevent division by zero# normalize test data using the mean and std of training data
x_test -= x_mean
x_test /= x_std + 1e-8

Note the last part: NEVER EVER normalize test data by its own mean and std. Use the training mean and std instead.

Solution 2:

you are doing prediction on x_test

predictions = model.predict_classes(x_test, batch_size=50)

and than comparing them with y_train

comparison = [(predictions[i], y_train_[i][0]) for i in range(0, len(predictions))]

I think it should be y_test