Unexpected Performance Drop with tf.data.Dataset and Image Augmentations in TensorFlow 2.10
I'm optimizing some code but I recently switched to I'm not sure how to approach I'm upgrading from an older version and I'm wondering if anyone has experience with I've been banging my head against this for hours..... I'm working on a convolutional neural network (CNN) using TensorFlow 2.10 and tf.data.Dataset for data loading. I implemented a series of image augmentations using the TensorFlow Image API, but I've noticed a significant drop in training performance, particularly in the first few epochs. My training accuracy is stagnating at around 30% after 10 epochs, and the loss is not decreasing as expected. Here’s a simplified version of what my data pipeline looks like: ```python import tensorflow as tf AUTOTUNE = tf.data.AUTOTUNE def load_and_preprocess_image(path): img = tf.io.read_file(path) img = tf.image.decode_jpeg(img, 3) img = tf.image.resize(img, [224, 224]) img = tf.image.random_flip_left_right(img) img = img / 255.0 # Normalize to [0,1] return img file_paths = tf.constant(['image1.jpg', 'image2.jpg']) # Replace with actual paths labels = tf.constant([0, 1]) dataset = tf.data.Dataset.from_tensor_slices((file_paths, labels)) dataset = dataset.map(lambda x, y: (load_and_preprocess_image(x), y), num_parallel_calls=AUTOTUNE) dataset = dataset.batch(32).prefetch(AUTOTUNE) ``` I’ve tried different augmentations and even removed them completely, but the model still struggles to learn. Additionally, when I run the dataset without the augmentations, the accuracy jumps to 80% after the same number of epochs. I’m currently using a ResNet50 model with the following compilation settings: ```python model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy']) ``` Is there something I could be missing in the image processing part or in the dataset configuration that could cause this unexpected drop in performance? Any insights or suggestions would be greatly appreciated! This is happening in both development and production on Ubuntu 20.04. I'd be grateful for any help. Cheers for any assistance! The project is a service built with Python. What's the correct way to implement this? For reference, this is a production service. What's the best practice here? The stack includes Python and several other technologies.