MDB improvements

[argakiig]

MDB_NOSYNC could provide improvements with disk usage by not calling fsync after commit
MDB_NORDAHEAD improves memory usage by not trying to read ahead in the DB, This has no effect on windows but other OSes should benefit. Windows does not read ahead so it is not affected

[SergiySW]

MDB_NOSYNC Don't flush system buffers to disk when committing a transaction. This optimization means a system crash can corrupt the database or lose the last transactions if buffers are not yet flushed to disk. The risk is governed by how often the system flushes dirty buffers to disk and how often mdb_env_sync() is called. However, if the filesystem preserves write order and the MDB_WRITEMAP flag is not used, transactions exhibit ACI (atomicity, consistency, isolation) properties and only lose D (durability). I.e. database integrity is maintained, but a system crash may undo the final transactions. Note that (MDB_NOSYNC | MDB_WRITEMAP) leaves the system with no hint for when to write transactions to disk, unless mdb_env_sync() is called. (MDB_MAPASYNC | MDB_WRITEMAP) may be preferable. This flag may be changed at any time using mdb_env_set_flags().
http://www.lmdb.tech/doc/group__mdb.html

[cryptocode]

Yeah I think this would need to be a config flag since it can more easily cause corruption. A pattern I've seen is "fast flags", e.g. MDB_NOSYNC is fast and MDB_NOSYNC | MDB_WRITEMAP | MDB_MAPASYNC is fastest, but the default should be neither.

[argakiig]

if MDB_NOSYNC is enabled without MDB_WRITEMAP you still maintain the ACI, at the cost of durability. the risk is you lose some final transactions and have to bootstrap them later. How is this different from the node being offline for extended periods.

[cryptocode]

MDB_NOSYNC can corrupt the database on system crashes/outage. That's relatively low risk and may be a fine tradeoff for some nodes, but I do think it should be a config option.

[argakiig]

ah, I see, let me see about getting a config flag option, up

[cryptocode]

@argakiig seems like you're right about not using MDB_WRITEMAP still maintaining ACI according to docs, a config flag would be nice though.

[clemahieu]

I think the first order of business is to figure out if there's any performance gain from these flags, if not we should document it and not make a change.

[argakiig]

Will create a separate PR for testing MDB_NOSYNC/MDB_WRITEMAP with config options This is now just for MDB_NORDAHEAD which, when supported, will stop the DB from being loaded