updateMktDepthL2 is slowish

[lkaino]

I have spent time optimizing my code for faster execution. Currently the slowest link in is the updateMktDepthL2 function in the ib_async.

A single call at decoder level takes 56us in my measurements. There are so many updates to the order book that with a numRows=40, a usual update loop takes around 10-15 ms.

This is not the end of the world, but when I have subscribed to order book from 3 stocks, it starts to add up to the latency.

I have been thinking of moving the order book updating to a separate process, but maybe there's some optimizations that could be done on the update logic or data structures being used?

[gnzsnz]

this is not realliy addressing your point directly, but it might help

have you tried with uvloop

[gnzsnz]

I tried with an algo i'm running on paper account and indeed uvloop did not work.

however if i run a simple test it works.

import asyncio
import uvloop

import ib_async

uvloop.install()
#asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

async def main():
    ib = await ib_async.IB().connectAsync("tws", 7499, clientId=210)
    
    await asyncio.sleep(1)
    print('I'm connected!')
    pos = ib.positions()
    print(len(pos), pos[-1])
    await asyncio.sleep(1)
    ib.disconnect()
    print('bye bye')


if __name__ == '__main__':
    print('before start')
    asyncio.run(main())
    print("the end")

I would need to test more, but i think it should work. in theory uvloop is transparent it implements the asyncio loop interface.

i read updateMktDepthL2 and assuming that list/dictionaries performance are to blamme (it was a quick look so it might be something else) i'm not sure there is much more room to improve, there are alternatives like frozeendict and immutables but the performance improvements are slim

https://github.com/Marco-Sulla/python-frozendict#benchmarks
https://gist.github.com/1st1/292e3f0bbe43bd65ff3256f80aa2637d

[lkaino]

I tried the uvloop yesterday, required a small change to ib_async. It is difficult to tell if there are any performance improvements. It would require setting up a similar scenario to compare with.

[maxicus]

May be just replace updateMktDepthL2 method body with just a callback to you and handle it in your code?
There you'll be able to optimize data to your algo needs.

At least to know it's ever possible to get noticeably better timings.

I expect you at least may omit part since
values = list(dom.values()) line there

[lkaino]

May be just replace updateMktDepthL2 method body with just a callback to you and handle it in your code? There you'll be able to optimize data to your algo needs.

At least to know it's ever possible to get noticeably better timings.

I expect you at least may omit part since values = list(dom.values()) line there

That's a good idea. I forgot that you can do that with Python.