Cost Function Outputs 'nan' In Tensorflow

While studying the tensorflow, I faced a problem. The cost function output 'nan'. And, if you find any other wrong in source code let me know the links for it. I am trying to sen

Solution 1:

You use a cross entropy loss without a sigmoid activation function to hypothesis, thus your values are not bounded in ]0,1]. The log function is not defined for negative values and it most likely get somes. Add a sigmoid and epsilon factor to avoid negative or 0 values and you should be fine.

Solution 2:

As I know,

Cross entropy cost function assumes that the hypothesis which you want to predict is stochastic value. Because cross entropy uses log function and (1-Y_) formula. Therefore, cross entropy loss should be used only for stochastic cases.

So you have to use the softmax function to make the results of the hypothesis probability.

W2 = tf.get_variable('W2', shape=[10 * 10 * 32, 1], 
initializer=tf.contrib.layers.xavier_initializer())        
b = tf.Variable(tf.random_normal([1]))

# hypothesis = tf.matmul(L1, W2) + bhypothesis = tf.nn.softmax(tf.add(tf.matmul(L1, W2), b))
cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) * tf.log(1 - hypothesis))

Or you can use this code

W2 = tf.get_variable('W2', shape=[10 * 10 * 32, 1], 
initializer=tf.contrib.layers.xavier_initializer())        
b = tf.Variable(tf.random_normal([1]))

hypothesis = tf.matmul(L1, W2) + b
cost = tf.nn.softmax_cross_entropy_with_logits(labels=Y, logits=hypothesis)