TensorFlow 2.12: Difficulty with tf.keras.callbacks.LearningRateScheduler Not Updating Learning Rate
I just started working with This might be a silly question, but I'm trying to implement a custom learning rate schedule using the `tf.keras.callbacks.LearningRateScheduler` in TensorFlow 2.12, but it seems that the learning rate is not updating as expected during training. I've set up my `LearningRateScheduler` to adjust the learning rate based on the epoch number, but when I check the learning rate at the end of each epoch, it's always the same as the initial value. Here’s the relevant portion of my code: ```python import tensorflow as tf # Define a simple model model = tf.keras.Sequential([ tf.keras.layers.Dense(64, activation='relu', input_shape=(32,)), tf.keras.layers.Dense(10) ]) model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001), loss='sparse_categorical_crossentropy', metrics=['accuracy']) # Custom learning rate schedule function def schedule(epoch, lr): if epoch > 5: return lr * tf.math.exp(-0.1) # Exponential decay after epoch 5 return lr # Set up the LearningRateScheduler callback lr_scheduler = tf.keras.callbacks.LearningRateScheduler(schedule) # Dummy data import numpy as np x_train = np.random.random((1000, 32)) y_train = np.random.randint(10, size=(1000,)) # Train the model model.fit(x_train, y_train, epochs=10, callbacks=[lr_scheduler]) ``` I've tried printing the learning rate within the `schedule` function, and it seems to be calculated correctly based on the epoch, but when I check the model's optimizer state after training, the learning rate remains at 0.001. I even tried adjusting the `verbose` parameter of `LearningRateScheduler` to see its output during training, but there were no helpful messages indicating any changes or updates. Is there something I'm missing in the implementation, or a specific configuration I need to ensure that the learning rate gets updated correctly? Any suggestions would be appreciated! My development environment is Ubuntu. My development environment is Debian. For context: I'm using Python on Linux. Cheers for any assistance!