Keras: Optimal Epoch Selection

I'm trying to write some logic that selects the best epoch to run a neural network in Keras. My code saves the training loss and the test loss for a set number of epochs and then p

Solution 1:

Use EarlyStopping which is available in Keras. Early stopping is basically stopping the training once your loss starts to increase (or in other words validation accuracy starts to decrease). use ModelCheckpoint to save the model wherever you want.

from keras.callbacks import EarlyStopping, ModelCheckpoint

STAMP = 'simple_lstm_glove_vectors_%.2f_%.2f'%(rate_drop_lstm,rate_drop_dense)
early_stopping =EarlyStopping(monitor='val_loss', patience=5)
bst_model_path = STAMP + '.h5'
model_checkpoint = ModelCheckpoint(bst_model_path, save_best_only=True, save_weights_only=True)

hist = model.fit(data_train, labels_train, \
        validation_data=(data_val, labels_val), \
        epochs=50, batch_size=256, shuffle=True, \
         callbacks=[early_stopping, model_checkpoint])

model.load_weights(bst_model_path)

refer to this link for more info

Solution 2:

Here is a simple example illustrate how to use early stooping in Keras:

• First necessarily import:

  from keras.callbacksimportEarlyStopping, ModelCheckpoint

• Setup Early Stopping

  # Set callback functions to early stop training and save the best model so farcallbacks = [EarlyStopping(monitor='val_loss', patience=2),
           ModelCheckpoint(filepath='best_model.h5', monitor='val_loss', save_best_only=True)]
  

• Train neural network

  history = network.fit(train_features, # Features
                    train_target, # Target vector
                    epochs=20, # Number of epochs
                    callbacks=callbacks, # Early stopping
                    verbose=0, # Print description after each epoch
                    batch_size=100, # Number of observations per batch
                    validation_data=(test_features, test_target)) # Data for evaluation

See the full example here.

Please also check :Stop Keras Training when the network has fully converge; the best answer of Daniel.