Slowdown with two indices (using tmpfs)

[eddie-scio]

I've been mounting a tmpfs in a Google App Engine Flex instance (after looking through a couple of issues in this repo that suggested this) to ensure everything is fully mmapped.

Recently, I made a change where my server is loading two Annoy indices -- I've tried both allocating two tmpfs and one larger tmpfs, the total volume of which is much larger than it needs to be to load both indices into memory.

However, simply keeping index 2 loaded has a ~2x slowdown at 50% on .retrieve() calls to index 1 (whether they're loaded from the same tmpfs or two separate ones). Is is possible there are some library-level locks or logic that prevent multiple indices from being fully mmapped on the same machine?

[erikbern]

It's possible the page cache in the kernel simply keeps things on disk until they are fully "warmed up". You can try to use the prefault flag, or as another option, hit the second index with a bunch of random queries when you load it to force the kernel to put it in RAM as opposed to disk

[eddie-scio]

I'm using tmpfs for the whole purpose of keeping it in RAM (as opposed to doing the various warmup and prefault strategies described in other issues) -- taking this advice. Are you saying that it's possible simply loading the index from tmpfs might still not be enough to keep it fully loaded in RAM?

I'll also shoot @sonots an email, seems like he might have a better idea.

[erikbern]

I don't know how tmpfs works, but if your computer has swap enabled, then I think it's hard to know whether it will "truly" be in RAM.

There's definitely no locking going on inside Annoy!

[eddie-scio]

There's definitely no locking going on inside Annoy!

That's useful to know, thanks!