LittleFS poor write performance

[LaurentLouf]

Description

• Target : STM32F103RCT6
• Toolchain : GNU ARM 7-2017-q4-major
• Tools : mbed-cli 1.7.3
• Mbed OS : e1bea44 (HEAD -> mbed-os-5.10, tag: mbed-os-5.10.2, tag: latest)

I encounter a problem, which I think is mostly a configuration problem with LittleFS. In my application, I store some small files on a NOR Flash and I realized today that those write are quite slow. It's two files, around 10 bytes each, and the write takes ~3s to write both on the file system. The thing that makes me thing it's mostly a configuration problem, is that saving only one takes around 2.9s, so there's not a huge difference between one or two files write. And I guess that reusing the file system object to write on another sets of files, after these initial writes, would be quite fast too (though not tested, since not needed in my application).

To be very specific :

• Flash ref is W25Q64JVSSIQ
• I'm using SPIFBlockDevice as block device (mbed-spiffs 8f4aaec (HEAD -> master, origin/master, origin/HEAD, spif-driver 5c49300 (HEAD -> master) )
• SPI clock frequency is 16MHz
• I use default configuration for LittleFS, so : MBED_LFS_READ_SIZE=64, MBED_LFS_PROG_SIZE=64, MBED_LFS_BLOCK_SIZE=512, MBED_LFS_LOOKAHEAD=512 and MBED_LFS_INTRINSICS=true

So I think I should tweak those, but haven't found any short read (reading quickly design.md and spec.md did not give me any info on that) that would give some insights on how to quickly do that for my application, and I feel that it could be a nice addition to the documentation. My gut feeling is telling me I probably should decrease the lookahead parameter, but with no certitude (though I will test that tomorrow). A small rule of thumb would be nice to cover the most common cases.

Issue request type

[X] Question
[ ] Enhancement
[ ] Bug

[ciarmcom]

Internal Jira reference: https://jira.arm.com/browse/MBOCUSTRIA-301

[geky]

Hi @LaurentLouf, sorry about the late response, I know it has been a long time. I had forgotten about this issue until @linlingao reminded me.

LittleFS v1 has several issues with small files. It doesn't surprise me that they are expensive.

We're working on fixing this in v2. It's experimental at the moment, but if you want to try it out you can find it here:
https://github.com/armmbed/mbed-littlefs

To use v2 you will need to change any instances of LittleFileSystem to LittleFileSystem2.

[LaurentLouf]

Unfortunately, I don't have that much time to test things and I can't change of version of LittleFS easily (I also would have to update a port for the ESP32 of the lib that I made). Thanks for getting back though.

I've re-tested some things today and got a very surprising result for the creation of directories (both using the native, retargeted function, or directly the LittleFS function), with up to 5s+ to create a directory, really not so deep in the tree (in case it has some importance). I don't know the exact time because it triggered a reset because of the watchdog in my example. Does this ring a bell to you @geky ?

[geky]

Hmm, it's possible it's triggering a lookahead scan, which currently has some performance issues. (related littlefs-project/littlefs#75). Unfortunately there isn't an API to trigger the lookahead manually.

These scans were slightly improved in v2, but results may vary. (moves and orphans were significantly improved, but the lookahead itself was not).

Scalability is the next big area I'm looking into in littlefs, and there are a number of improvements that are possible. But it's going to take a while for us to get ahead of current number of issues.

Hopefully that helps, feel free to ask any other questions, though sorry my response time is so slow : )

[LaurentLouf]

Your response time is perfectly fine. Thanks for the link, it's both very long and very instructive (though I'd wish to know more about file systems to fully grasp what is going on) ! I'll try fiddling with the parameters and I'll probably move the file system object in my state machine implementation instead of directly in the states, which should be fine enough

[geky]

We do have a design document that may be helpful if you're interested in learning more:
https://github.com/ARMmbed/littlefs/blob/master/DESIGN.md

Unfortunately how a filesystem behaves can get a bit complicated at times.

[ciarmcom]

Thank you for raising this issue. Please note we have updated our policies and
now only defects should be raised directly in GitHub. Going forward questions and
enhancements will be considered in our forums, https://forums.mbed.com/ . If this
issue is still relevant please re-raise it there.
This GitHub issue will now be closed.