Pandas DataFrame filtering results in missing rows after using .loc with a date range on a datetime index
I am working with an scenario where filtering my DataFrame using a date range with `.loc` appears to skip rows that I expect to see. I'm using Pandas version 1.5.1, and my DataFrame has a datetime index. Here’s a simplified version of what I’m doing: ```python import pandas as pd import numpy as np dates = pd.date_range('2023-01-01', periods=100, freq='D') data = np.random.randn(100) df = pd.DataFrame(data, index=dates, columns=['values']) # Introduce some NaN values randomly nan_indices = np.random.choice(df.index, size=10, replace=False) df.loc[nan_indices] = np.nan # Attempting to filter using a date range filtered_df = df.loc['2023-01-05':'2023-01-15'] print(filtered_df) ``` After executing the filtering, I find that some rows that I expect to be included are somehow missing from the result, especially those containing NaN values. The `filtered_df` should show values for every day from January 5 to January 15, 2023, but it seems to skip over any row with NaN. I expected that rows with NaN would still be included in the output. I've tried using `df.loc['2023-01-05':'2023-01-15'].fillna(method='ffill')` to handle NaNs, but this doesn't resolve the filtering scenario itself. Is there a specific behavior in Pandas that causes this, or am I missing something in my approach? Any insights would be appreciated!