Getting Batch Predictions For Tfrecords Via Cloudml

I followed this great tutorial and successfully trained a model (on CloudML). My code also makes predictions offline, but now I am trying to use Cloud ML to make predictions and ha

Solution 1:

When you export your model, you need to make sure that it is "batchable", i.e., the outer dimension of the input Placeholder has shape=[None], e.g.,

input = tf.Placeholder(dtype=tf.string, shape=[None])
...

That may require reworking the graph a bit. For instance, I see that the shape of your output is hard-coded to [1,1]. The outermost dimension should be None, this may happen automatically when you fix the placeholder, or it may require other changes.

Given that the name of the output is probabilities, I would also expect the innermost dimension to be >1, i.e. the number of classes being predicted, so something like [None, NUM_CLASSES].