Knowing when library has been reloaded

[rgaudin]

This is related to #728 but independent.

Use case we have is that kiwix-serve is behind a varnish cache for library.kiwix.org because it's not able to handle the load.
Our cache sure has a time-based expiration for all its entries but it's not relevant here.

Because we frequently publish new ZIM files, we frequently (at most once per hour ATM) regenerate the library XML file.
When we do, we want to invalidate our cache entries related to the Catalog.

Our problem is that we don't know when kiwix-serve has actually reloaded the library and is ready to serve new data.

If we invalidate cache too soon, then chances are an incoming request happens before the refresh and we'd be storing old data into the cache instead of the new one.

To workaround this, we are now waiting 10s after writing the XML file on disk and purging the cache. Of course, that's arbitrary, ugly and fragile.

How could we be informed that the library has been reloaded?

[evrial]

No library - no problem.
#735

[rgaudin]

@evrial:

• library allows customizing/overriding a number of metadata. Without it we'd be stuck with in-ZIM metadata.
• watching a directory would absolutely not solve this issue. An external program would not be able to know when kiwix-serve has detected and reloaded for a specific ZIM. For the cache example above, there would be no improvement.