Training Multiple Input And Output Keras Model (valueerror: Failed To Convert A Numpy Array To A Tensor (unsupported Object Type List))

I have six different Sequnstial Keras models and I desire to concatenate them like the following: Models activation = 'tanh' model1 = Sequential() model1.add(Dense(3, input_dim=3,

Solution 1:

As far as I can tell we can't just pass all variable-size inputs together in fit for the multi-input model. The way you pass your training pairs to the model, it surely unable to unpack for concern input layers. The related post that you mentioned is also an important fact to consider.

However, in tensorflow, we can use tf.ragged.RaggedTensor for variable-length input sequence, discussed here. But not sure if there is any workaround possible by converting to the ragged tensor. It probably would possible if a single input layer takes a different length of the input sequence.

If you read the fit method's training pairs input you would see keras expects the x paramer as follows:

Argumentsx: Inputdata. Itcouldbe:
    1. ANumpyarray (or array-like), oralistofarrays (in case the model 
       has multiple inputs).
    2. ATensorFlowtensor, oralistoftensors (in case the model 
       has multiple inputs).
    3. Adictmappinginputnamestothecorrespondingarray/tensors, 
       ifthemodelhasnamedinputs.
...
...

For your case, option 3 is pretty convenient to choose from, which is passing the dictionary mapped input names with training pairs. Here is one way we can do this. First set some names to the input layer of each model. We set model1, model2, ... etc.

from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense 

activation = 'tanh'
model1 = Sequential()
model1.add(Dense(3, input_dim=3, activation=activation, name='model1'))
model1.add(Dense(3, activation=activation))
model1.add(Dense(1))

model2 = Sequential()
model2.add(Dense(3, input_dim=3, activation=activation, name='model2'))
model2.add(Dense(3, activation=activation))
model2.add(Dense(1))

model3 = Sequential()
model3.add(Dense(1, input_dim=1, activation=activation, name='model3'))
model3.add(Dense(1, activation=activation))
model3.add(Dense(1))

model4 = Sequential()
model4.add(Dense(6, input_dim=6, activation=activation, name='model4'))
model4.add(Dense(3, activation=activation))
model4.add(Dense(1))

model5 = Sequential()
model5.add(Dense(2, input_dim=2, activation=activation, name='model5'))
model5.add(Dense(2, activation=activation))
model5.add(Dense(1))

model6 = Sequential()
model6.add(Dense(6, input_dim=6, activation=activation, name='model6'))
model6.add(Dense(4, activation=activation))
model6.add(Dense(1))

Now, build whole the final model where we also set the last layer name, which is set here as target_concatenate.

import tensorflow as tf 

model_concat = tf.keras.layers.concatenate(
    [
     model1.output, model2.output, model3.output, 
     model4.output, model5.output, model6.output
    ], name='target_concatenate')

composed_model = tf.keras.Model(
    inputs=[model1.input, model2.input, model3.input, 
            model4.input, model5.input, model6.input],
    outputs=model_concat
)
composed_model.compile(optimizer='sgd', loss='mse')

DataSet

The sample data you provided is not legal for model training as we mentioned above. Firstly it should not be a list but numpy and secondly, for multi-input variable size, it convenient to pass them separately.

import numpy as np 

# Six Inputs 
one = np.random.randint(-50, 10, size=(1, 3))
two = np.random.randint(-10, 30, size=(1, 3))
thr = np.random.randint(20, size=(1))
fur = np.random.randint(10, size=(1, 6))
fiv = np.random.randint(50, size=(1,2))
six = np.random.randint(10, size=(1, 6))

# One Target
tar = np.random.randint(2, size=(1, 6))

print(one, one.shape)
print(two, two.shape)
print(thr, thr.shape)
print(fur, fur.shape)
print(fiv, fiv.shape)
print(six, six.shape)
print(tar, tar.shape)

[[-42   9 -34]] (1, 3)
[[28 22  7]] (1, 3)
[19] (1,)
[[4 4 1 7 4 7]] (1, 6)
[[40 35]] (1, 2)
[[3 6 1 1 3 8]] (1, 6)
[[0 0 1 1 0 0]] (1, 6)

When we call .fit, we will pass these datasets as dict mapping model's input and output names to the corresponding array. So, let's check to get the names of the composed model.

# viewing the model shapes and layers name # tf.keras.utils.plot_model(composed_model, #                            show_shapes=True, show_layer_names=True)
composed_model.input_names, composed_model.output_names

(['model1_input',
  'model2_input',
  'model3_input',
  'model4_input',
  'model5_input',
  'model6_input'],
 ['target_concatenate'])

Great, now we can pass training Paris as follow conveniently to the fit method.

composed_model.fit(
    {
         "model1_input": one, "model2_input": two, "model3_input": thr,
         "model4_input": fur, "model5_input": fiv, "model6_input": six
     },
     {
         "target_concatenate": tar
     },
    epochs=10,
    batch_size=32,
    verbose=2
)

Epoch 1/10
496ms/step - loss: 0.6022
Epoch 2/10
5ms/step - loss: 0.5428
Epoch 3/10
7ms/step - loss: 0.4888
Epoch 4/10
16ms/step - loss: 0.4426
Epoch 5/10
8ms/step - loss: 0.4041
Epoch 6/10
8ms/step - loss: 0.3719
Epoch 7/10
9ms/step - loss: 0.3448
Epoch 8/10
5ms/step - loss: 0.3215
Epoch 9/10
7ms/step - loss: 0.3011
Epoch 10/10
5ms/step - loss: 0.2830