lfs_file_seek very slowly

[nizhq]

I used Littlefs(version is V2.2) in a 16 Megabyte SPI flash drive with a block size of 4096 bytes.
The file system's read and write cache size is 256 bytes, and the lfs_lookahead_buff size is 32 bytes.
malloc is not used.
The cache of file is 256 bytes.

I created a file in the root directory with 16K bytes at the beginning, but the file will remain open and write data continuously, and the final file size may exceed 300k bytes.
When the file is ready to be close, I need to modify the first 7 bytes of the file. use lfs_file_seek to jump the file pointer to the file header, where lfs_file_seek is extremely slow and takes about 4 seconds.

I found operation flush in lfs_file_seek, In units of one byte, 300K bytes were copied, causing slow execution.
The slow code is located on lines 2678-2697 of lfs.c

I'd like to know how to avoid this problem of slow execution.

Looking forward to your reply
Thanks!

[nizhq]

I looked at the source code.Discover that every time the contents of an old file are changed, the file system first creates a new space and copies the entire modified file into the new space.This should be the mechanism of wear equalization.I wonder if My understanding is correct?

[geky]

Hi @nizhq, thanks for creating an issue.

This looks related to #27. The short explanation is that littlefs does not handle random-writes well, and a write to a random location in a file will require copying all data after that location.

Unfortunately this means littlefs performs terribly on files that contain a header. As a workaround, a footer (store the header at the end of the file) would be more performant, though I realize it isn't a great solution if you have an existing format and existing tools.

lfs_file_seek is extremely slow and takes about 4 seconds.

The reason it looks like lfs_file_seek is slow is because lfs_file_seek needs to save the current state of the file to disk before moving the file position. If you seek then write, littlefs doesn't save this to disk immediately, instead it keeps it in the file cache in case it can combine multiple file writes into one disk write. But littlefs only keeps one cache per file, so when you seek it needs to flush the cache to disk, which is what lfs_file_flush is doing.

This should be the mechanism of wear equalization.

littlefs only provides dynamic wear-leveling (it only moves blocks that are written to), so in theory it shouldn't need to rewrite the entire file. This is more a limitation of the file data-structure. I've been looking into possibly using B-trees instead, however the current CTZ skip-list data structure has some valuable properties, such as traversal, that are proving difficult to replace.

[nizhq]

Hi @geky ,Thank you for your suggestion and explanation

I'm going to try to put the header information at the end of the file, which might involve some changes, but that's acceptable.
And I will continue to focus on LittleFS and look forward to better performance in future releases