How To Save Weights Of Keras Model For Each Epoch?

I want to save keras model and I want to save weights of each epoch to have best weights. How I do that? Any help would be appreciated. code: def createModel(): input_shape=(1,

Solution 1:

model.get_weights() will return a tensor as a numpy array. You can save those weights in a file with extension .npy using np.save().

To save weights every epoch, you can use something known as callbacks in Keras.

from keras.callbacks import ModelCheckpoint

before you do model.fit, define a checkpoint as below

checkpoint = ModelCheckpoint(.....), assign the argument 'period' as 1 which assigns the periodicity of epochs. This should do it.

Solution 2:

You should use model.get_weights() and LambdaCallback function together:

1. model.get_weights(): Returns a list of all weight tensors in the model, as Numpy arrays.

   model = Sequential()
   weights = model.get_weights()
   

2. LambdaCallback: This callback is constructed with anonymous functions that will be called at the appropriate time

   import json
   json_log = open('loss_log.json', mode='wt', buffering=1)
   json_logging_callback = LambdaCallback(
               on_epoch_end=lambda epoch, logs: json_log.write(
                   json.dumps({'epoch': epoch, 'loss': logs['loss']}) + '\n'),
               on_train_end=lambda logs: json_log.close()
   )
   
   model.fit(...,
             callbacks=[json_logging_callback])
   

When your code is considered, you should write callback function and add to your model:

import json
from keras.callbacks import LambdaCallback

json_log = open('loss_log.json', mode='wt', buffering=1)
json_logging_callback = LambdaCallback(
            on_epoch_end=lambda epoch, logs: json_log.write(
                json.dumps({'epoch': epoch, 
                            'loss': logs['loss'],
                            'weights': model.get_weights()}) + '\n'),
            on_train_end=lambda logs: json_log.close()
)

model.compile(loss='categorical_crossentropy',
              optimizer=opt_adam, 
              metrics=['accuracy'])

model.fit_generator(..., callbacks=[json_logging_callback])

This code write your all weights in all layers to json file. If you want to save weights in specific layer, just change the code with

model.layers[0].get_weights()

Solution 3:

I am not sure it will work but you can try writing callback and inside callback you can save the weights.

Eg.

checkpoint = ModelCheckpoint("best_model.hdf5", monitor='loss', verbose=1,
    save_best_only=True, mode='auto', period=1)

model.fit(x_train, y_train,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test, y_test),
          callbacks=[checkpoint])

source = https://medium.com/@italojs/saving-your-weights-for-each-epoch-keras-callbacks-b494d9648202

Solution 4:

You can write a ModelCheckpoint callback using tf.keras.callbacks.ModelCheckpoint to save weights every epoch. If you are using recent Tensorflow like TF2.1 or later, then You need to use save_freq='epoch' to save weights every epoch instead of using period=1 as other answer mentioned. Please check entire example here

callback is as follows

checkpoint_path = "./training_2/cp-{epoch:04d}.ckpt"
checkpoint_dir = os.path.dirname(checkpoint_path)

cp_callback = tf.keras.callbacks.ModelCheckpoint(
   checkpoint_path, verbose=1, save_weights_only=True,
   # Save weights, every epoch.
   save_freq='epoch')

calling the model training

# Create a basic model instance
model=create_model()
model.save_weights(checkpoint_path.format(epoch=0))
model.fit(x_train, y_train,
         epochs = 50, callbacks = [cp_callback],
         validation_data = (x_test,y_test),
         verbose=0)

Hope this helps. Thanks!