How To Get Correct Acccuracy For Multi Label Prediction?

I am trying to get a tensorflow network that does multi-label predictions. Using softmax with one-hot (single label) predictions works correctly. The accuracy get's calculated perf

Solution 1:

At the end, after countless tries to get this fixed, it turned out that everything was ok, except for the optimizer. When using the AdamOptimizer() instead of the GradientDescentOptimizer(learning_rate) the network started learning fast, from 0.7 to 0.97 accuracy in 40 epochs.

Maybe some tweaking of the learning rate would also make it work, but for now this is finally solved!