👀 Views: 0 💬 Answers: 1 📅 Created: 2025-08-27

pandas dataframe timeseries Python

I'm working on a project and hit a roadblock... I'm working with a multi-index DataFrame in Pandas where the outer index represents different regions and the inner index represents time. After performing a resampling operation to get daily averages, I'm running into issues with NaN values that arise due to missing data for some regions on certain days. The question is that when I try to fill these NaNs using `fillna(method='ffill')`, it doesn't seem to propagate the values correctly across the multi-index. Here's a small sample of my DataFrame: ```python import pandas as pd import numpy as np # Sample data arrays = [ ['North', 'North', 'South', 'South'], pd.date_range('2023-01-01', periods=3, freq='D'), ] index = pd.MultiIndex.from_arrays(arrays, names=('Region', 'Date')) df = pd.DataFrame({ 'Value': [1, np.nan, 2, np.nan] }, index=index) print(df) ``` After resampling: ```python daily_avg = df.resample('D').mean() print(daily_avg) ``` Now, when I try to fill the NaNs: ```python daily_avg_filled = daily_avg.fillna(method='ffill') print(daily_avg_filled) ``` I'm finding that the 'North' region's final NaN for '2023-01-03' is not filled as I expected. It seems like the forward fill only works within each individual region and doesn't apply across the multi-index. I've tried using `groupby` on the outer index before filling, but it still doesn't yield the desired results. Any suggestions on how to properly handle NaN values in this context? I’m using Pandas version 1.3.5. I'm working on a CLI tool that needs to handle this. Is there a better approach?