Tensorflow Estimator - High Evaluation Values On Training Data

I'm using Tensorflow 1.10 with a custom Estimator. To test my training/evaluation loop, I just feed the same image/label into the network every time, so I expected the network to c

Solution 1:

I've found, that the handling of BatchNormalization can cause such errors, like described here.

The usage of the control_dependencies in the model-fn solved the issue for me (see here).

if mode == tf.estimator.ModeKeys.TRAIN:
    optimizer = tf.train.MomentumOptimizer(learning_rate=1e-4, momentum=0.9)
    with tf.control_dependencies(model.get_updates_for(features)):
        train_op = optimizer.minimize(loss, global_step=tf.train.get_global_step())
    return tf.estimator.EstimatorSpec(mode=mode, loss=loss, train_op=train_op)