Pandas moving average where the window expands/contracts at the ends?

84 Views Asked by At

This has probably been answered somewhere but I am having a right brain fade on what you actually call this.

In Python/Pandas, if I use:

df['sginal'] = df['signal'].rolling(window=9).mean()

Then I get a sliding window, size 9, which averages to the central position in the window before moving to the next sample point. This obviously doesn't work well at the ends because there is not enough data points on one side of the window. So you get "nan" as an output and also when you compare to the original input signal, the output is shifted (lagging by 8).

In MATLAB I often used:

signal = smooth(signal, 9, 'moving');

The way this works is it will grow the window from the start and then shrink at the end. So until I get to 5th element in the data, the moving average window is not 9. Instead it started at 1, then 3, then 5, then 7 and then finally 9. At the end of the data, the window shrinks down again.

This way you do not get "nan" and you do not get any lagging output. I don't care that the initial and final handful of values essentially haven't been averaged at the full 9 samples. I don't know what you call this though.

Does Pandas have anything similar? should I look in NumPy or SciPy instead?

2

There are 2 best solutions below

1
amance On BEST ANSWER

You could define min_periods to eliminate the nans.

df['signal'] = df['signal'].rolling(window=9, min_periods=1).mean()
0
Otispunkmeyer86 On

Never mind, I found it in:

https://pandas.pydata.org/docs/reference/api/pandas.Series.rolling.html

I need to set: min_periods=1 and I also need to set center=True. This way it will cinch down to window size of 1 and also average around the center and not the right hand edge.