Single Server | GpuIndexFlatL2 Write Strategy

[dexception]

Running on:

• CPU
• [yes] GPU

Interface:

• [ yes] C++
• Python

About my app:

1. Multi threaded http server based application
2. Accepts id and vector for /add request
3. Provide GpuIndexFlatL2 search functionality

However all the adding and search is happending in memory and if the application closes or crashes the data is lost. My question is since faiss supports writing index via:

    const char *name = "index.bin";
    faiss::write_index(faiss::gpu::index_gpu_to_cpu(index), name);

how do i implement the most efficient index saving strategy ?

1. Block all requests while index is being written to file for every new vector added
   This will lead to decrease in performance.

2. Periodically update the index in the background after every 10,000 new vectors
   if application crashes unwritten new vectors will be lost

3. Other strategy ?

Please help me. I have been scratching my head for the last 2 weeks regarding this problem.

[mdouze]

It really depends on the operating conditions.
One approach is with two indexes: one big one with most of the vectors, and one in which you add new vectors. At search time, you search in both.

Then you can save every 10k adds with:

1. save small index with (fast) with incremental file names

2. merge small index into big one (fast, in RAM)

3. clear small index.

At recover time, you then need to load the small indexes to reconstruct the big one. You could have a background job that merges the small indexes on disk.

[dexception]

Thanks i think this would work without data loss in case of failure.

Other question is how do you handle meta data for the vectors because when the results for distance search are achieved that might not be revelant. For example,

In our application we have clientId,categoryId for each vector and other attributes as well. So when the topK results are returned that might not be for that clientID. Is there an Index that suports adding attributes for vectors inside the index as well ?

[mdouze]

No, you need to put metadata in a separate conventional table.
Rationale in #641

[dexception]

Resolved.