How to use numba with pandas.Rolling.Apply

485 views Asked by At

I want to speed up my following code:

values_list = []
indices_list = []

offset_by_w = pd.tseries.frequencies.to_offset(135)

_ = signal[::-1].rolling(offset_by_w, min_periods=4, closed='both')./
apply(lambda x: [0, indices_list.append((x.index[-1], x.index[0])), values_list.append(x[::-1])][0], raw=True, engine='numba')

but I get an error:

numba.core.errors.TypingError: Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Failed in nopython mode pipeline (step: nopython frontend)
Untyped global name 'indices_list': Cannot type empty list

How can I solve this problem with saving values_list , indices_list ?

signal is pd.Series where indexies' type is Timestamp and values' type is float

I have pd.Series like this:

example of input data

And I want to roll with moving window through the data (by reverse order) to get two lists with apply method:

  1. values_list

where the 1st element looks like

1st element of values_list

the 2nd element looks like

2nd element of values_list

  1. indices_list

where the 1st element looks like

1st element of indices_list

the 2nd element looks like

2nd element of indices_list

1

There are 1 answers

1
BeRT2me On

I see you have the pandarellel tag, to use it here you'd change your code to:

from pandarallel import pandarallel
pandarallel.initialize()

values_list = []
indices_list = []

offset_by_w = pd.tseries.frequencies.to_offset(135)

_ = signal[::-1].rolling(offset_by_w, min_periods=4, closed='both').\
    parallel_apply(lambda x: [0, indices_list.append((x.index[-1], x.index[0])), values_list.append(x[::-1])][0], raw=True)