👀 Views: 2 💬 Answers: 1 📅 Created: 2025-06-10

machine-learning scikit-learn hyperparameter-tuning Python

I'm stuck on something that should probably be simple. I'm stuck trying to I've been working on this all day and Quick question that's been bugging me - I'm currently using `GridSearchCV` from `scikit-learn` version 0.24.2 to tune the hyperparameters of a `RandomForestClassifier` for a binary classification question. My dataset is significantly imbalanced with a 90:10 ratio of positive to negative samples. I've set up my parameter grid as follows: ```python param_grid = { 'n_estimators': [50, 100, 200], 'max_depth': [None, 10, 20, 30], 'min_samples_split': [2, 5, 10], 'min_samples_leaf': [1, 2, 4] } ``` After running `GridSearchCV` with a 5-fold cross-validation, I noticed that while it returns a high accuracy on the training set, the validation scores are significantly lower, indicating potential overfitting. Here’s how I’m currently initializing it: ```python from sklearn.model_selection import GridSearchCV from sklearn.ensemble import RandomForestClassifier from sklearn.metrics import classification_report rf = RandomForestClassifier(random_state=42) grid_search = GridSearchCV(estimator=rf, param_grid=param_grid, cv=5, scoring='f1', verbose=2, n_jobs=-1) ``` I’ve tried implementing techniques such as class weighting by adding `class_weight='balanced'` to the classifier, but I’m still concerned about overfitting when I use the best parameters for final model training. Additionally, I attempted to use `StratifiedKFold` instead of the default cross-validation, but the results haven't changed much. Here's the behavior I’m working with in terms of performance: when I evaluate the model on a separate validation set (split from the original data), the F1 score drops from around 0.92 in cross-validation to 0.75. Given the imbalanced nature of my dataset, this performance drop is alarming. What can I do to better tune the model parameters while also preventing overfitting and ensuring my model generalizes well on unseen data? Are there specific strategies or best practices I should consider for hyperparameter tuning with imbalanced datasets? I'm coming from a different tech stack and learning Python. Thanks in advance!