ValueError: Only instances of keras.Layer can be added to a Sequential model. Received: <tensorflow_probability.python.layers.dense_variational_v2.DenseVariational object at 0x000001C67AF5B4D0> (of type <class 'tensorflow_probability.python.layers.dense_variational_v2.DenseVariational'>)

[Suryansh-patidar]

While implementing BNN using tensorflow, i'm getting the below error while using DenseVariational and not sure how to resolve it
I have also tried using Google collab/ Jupyter notebook/ VS code/ Kaggle, but the issue is the same
The tensorflow version that i tried to use are both 2.15.0/ 2.16.1 and python version is 3.11.3

from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow import keras
import keras_tuner
import tensorflow_probability as tfp
from tensorflow_probability import distributions as tfd
from tensorflow.keras.models import Model
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dropout,Input,Dense
from tensorflow.keras.callbacks import EarlyStopping
from tensorflow.keras.optimizers import Adam
from keras import layers
from keras import regularizers

best_model_2 = tf.keras.Sequential([
    layers.Dense(181,activation='relu',input_shape=(21,)),
    tfp.layers.DenseVariational(181,activation='relu',make_posterior_fn=posterior_mean_field,
                                        make_prior_fn=prior_trainable,
                                        kl_weight=1/train_size,),
    layers.Dense(1,activation='linear')
])

ValueError                                Traceback (most recent call last)
Cell In[42], line 7
      1 # tf.__version__
      2 # %pip uninstall tensorflow
      3 # import tensorflow
      4 # %pip install tensorflow==2.14
      5 # keras.__version__
----> 7 best_model_2 = tf.keras.Sequential([
      8     layers.Dense(181,activation='relu',input_shape=(21,)),
      9     tfp.layers.DenseVariational(181,activation='relu',make_posterior_fn=posterior_mean_field,
     10                                         make_prior_fn=prior_trainable,
     11                                         kl_weight=1/train_size,),
     12     layers.Dense(1,activation='linear')
     13 ])

File ~\anaconda3\Lib\site-packages\keras\src\models\sequential.py:73, in Sequential.__init__(self, layers, trainable, name)
     71 if layers:
     72     for layer in layers:
---> 73         self.add(layer, rebuild=False)
     74     self._maybe_rebuild()

File ~\anaconda3\Lib\site-packages\keras\src\models\sequential.py:95, in Sequential.add(self, layer, rebuild)
     93         layer = origin_layer
     94 if not isinstance(layer, Layer):
---> 95     raise ValueError(
     96         "Only instances of `keras.Layer` can be "
     97         f"added to a Sequential model. Received: {layer} "
     98         f"(of type {type(layer)})"
     99     )
    100 if not self._is_layer_name_unique(layer):
    101     raise ValueError(
    102         "All layers added to a Sequential model "
    103         f"should have unique names. Name '{layer.name}' is already "
    104         "the name of a layer in this model. Update the `name` argument "
    105         "to pass a unique name."
    106     )

ValueError: Only instances of `keras.Layer` can be added to a Sequential model. Received: <tensorflow_probability.python.layers.dense_variational_v2.DenseVariational object at 0x000001C663D64350> (of type <class 'tensorflow_probability.python.layers.dense_variational_v2.DenseVariational'>)

[mehtamansi29]

Hi @Suryansh-patidar-

Could you please share posterior_mean_field,prior_trainable and train_size to reproduce the issue ?

[Suryansh-patidar]

Hi @Suryansh-patidar-

Could you please share posterior_mean_field,prior_trainable and train_size to reproduce the issue ?

def posterior_mean_field(kernel_size,bias_size,dtype: any):
    n = kernel_size + bias_size
    c = np.log(np.expm1(1.))

    return tf.keras.Sequential([
        tfp.layers.VariableLayer(2*n, dtype=dtype),
        tfp.layers.DistributionLambda(lambda t: tfd.Independent(
            tfd.Normal(loc=t[..., :n],
                       scale = 1e-5 + tf.nn.softplus(c + t[..., n:])),
                       reinterpreted_batch_ndims=1)),
        ])


def prior_trainable(kernel_size,bias_size,dtype: any):
    n = kernel_size + bias_size
    return tf.keras.Sequential([
        tfp.layers.VariableLayer(n, dtype=dtype),
        tfp.layers.DistributionLambda(lambda t: tfd.Independent(
            tfd.Normal(loc=t,scale=1),
            reinterpreted_batch_ndims = 1)),
    ])

And train_size = 98455

[mehtamansi29]

Hi @Suryansh-patidar -

Thanks for the code snippet. I tested the code snippet with keras 2.15 and tensorflow 2.15 and it is working fine. Attached gist for your reference.