Removing first block of file

[DrArmaggedon]

First of all I should thank you for the great work, saving me a lot of headache.

I use LittleFS in a project to store logs in LIFO style. The issue is that I need the log file to be bounded, that means when the log file size reaches a specific limit, first block of file should be removed before adding new entries. I understand that it's possible to implement a Circular Buffer over a pre-allocated file to do the job but with its own overhead. Also worth mentioning that entries are added to file in a block size chunk.

I wonder if it's possible to simply change meta data to remove the first block of file? Any hints expect for reading the whole spec? ;)

[DrArmaggedon]

Ok, I have read the Spec, seams that due to backward nature of CTZ-Skip list, removing first block requires a large amount of update (COW) to structure. Is that right?

[geky]

Hi @DrArmaggedon thanks for creating an issue, sorry for the late response.

This is actually a feature I'm personally quite interested in exploring once other pressing issues in the filesystem are sorted out. It would be a nice feature for a common use case of LittleFS : )

Unfortunately the CTZ-skip makes this a bit difficult. You could in theory keep an extra "offset" field that indicates low-numbered blocks are unused and return them to the block allocator, though there are some assumptions about how large the file can ever get and this would break that.

Unrelated to this, #27 indicates that the current data-structure is insufficient for the general case, so I'm planning to adopt a more general purpose B-tree structure in the future. With this I'm hoping to sneak in this feature of dropping the first n-bytes of a file cheaply.

---

For now the most efficient way to store logs in littlefs is to use a system of rotating files, common in Linux and others. That is to write a file to some fix size, and when it's full rename it to log.2 and start writing a new file. This works well with littlefs's append-friendly file data-structure.