Modifying the block in the middle of the file requires copying the following block

[Johnxjj]

In the line 398 of DESIGN.md you mentioned：

To avoid a full copy during appends, we can store the data backwards. Appending blocks just requires adding the new block and no other blocks need to be updated. If we update a block in the middle, we still need to copy the following blocks, but can reuse any blocks before it. Since most file writes are linear, this design gambles that appends are the most common type of data update.

So suppose this is a very large file, you need to modify the block in the middle of the file, then you need to copy the following blocks of this large file every time, so will it take a lot of time to copy? And Is there any good solution?

[earlephilhower]

@Johnxjj, it's like you were reading my mind! We've recently included LittleFS and found some cases like this (update in middle of a file) which seemed too slow to be real.

In our case a "very large file" is only 64-128KB, and with the flash we're using erasing blocks and writing a new copy after an update near the beginning is taking 1.5s. Using SPIFFS (which has its own issues) such an update is 200-300ms.

I'm not sure it's a bug, honestly, as it is a design assumption that might not be so true in general cases.

[Johnxjj]

@earlephilhower and TD-er，Thank you for your advice. This seems to be a good solution.

Or, you can use separate files (or a suibdir since they work fine on LittleFS) for each structure and then just overwrite the file-per-structure and you'll have blazing speeds.

I think I can make it a separate handler on LittleFS and write a separate import/export handler for it to make it look like a single file.

[geky]

I've been thinking about this limitation ever since #27.

It is an intentional design tradeoff, but not a great one. And it catches people by surprise.

So far, every time this crops up there's been a successful workaround, but that may just be a result of the determination of embedded developers.

Improving random writes is a bit complicated though. So it's not a problem I've been able to get to yet.

My opinion right now is that adopting proper B-trees is the correct solution. Though how they interact with the rest of the filesystem raises a number of questions.

One of the more interesting questions: How do you traverse a read-only tree with constant RAM?

• Recursive solutions use O(height) stack
• You can't swap pointers during traversal
• Introducing a threaded-list (B+-tree) brings back the O(n) update cost, effectively making the structure a skip-list