Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Memory Usage when calling e.g. btalib.rsi(df) in a loop #11

Open
moritzgun opened this issue Jan 28, 2021 · 5 comments
Open

Memory Usage when calling e.g. btalib.rsi(df) in a loop #11

moritzgun opened this issue Jan 28, 2021 · 5 comments

Comments

@moritzgun
Copy link

I am running a MultiplexSocket on Binance API to get 5 minute klines of around 200 symbols. On every new completed candle I call a "run_technical_analysis" function. Inside that function I calculate indicators for each symbol like this:

period = 14
value = btalib.ema(candlesticks_df.tail(period + 1), period=period) ## Version A
#value = candlesticks_df.Close.tail(14).ewm(span=14).mean() ## Version B

After that I append value.df.tail(1) to a list. On Version A my RAM goes up almost 10 MB every 5 minutes. It seems like the allocated memory for 'value' is not beeing freed. When I use Version B instead there is no such issue.

What can I do to reduce that memory usage?

@reedajohns
Copy link

Same problem on my end!

@drhighliner
Copy link

drhighliner commented Mar 30, 2021

Same problem on my end as well.
Does anybody have a fix for this? Maybe we can create a fork and publish it on pypi ... seems like the developer is not updating this library anymore. :(

@PapoKarlo
Copy link

with btalib.stochastic(df) same problem, memory leak very fast.

@sam1946
Copy link

sam1946 commented Oct 8, 2021

https://stackoverflow.com/questions/13784192/creating-an-empty-pandas-dataframe-then-filling-it

This may solve the issue?
Never append a row to a dataframe

@Davesmacer
Copy link

Every indicator with btalib inside a loop is filling memory, even if we use del to free variables returned. This is a big issue.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants